2024-01-11T17:31:23.460 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-11T17:31:23.460 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480 --owner scheduled_vshankar@teuthology --verbose --name vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/orig.config.yaml 2024-01-11T17:31:23.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-11T17:31:23.617 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480 branch: wip-vshankar-testing-20240103.072409-1 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: vshankar@redhat.com first_in_suite: false job_id: '7511480' kernel: client: branch: testing flavor: default kdb: true sha1: c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20240103.072409-1 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: c27531add9155519f390b228e498f0a2c5eae117 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: c27531add9155519f390b228e498f0a2c5eae117 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240103.072409-1 sha1: c27531add9155519f390b228e498f0a2c5eae117 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4792 sha1: c27531add9155519f390b228e498f0a2c5eae117 sleep_before_teardown: 0 subset: 414/1024 suite: fs suite_branch: wip-vshankar-testing-20240103.072409-1 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c27531add9155519f390b228e498f0a2c5eae117 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUxAXgF7Uz1hRGAu0N2ik0dSn7OJa3/bAmjoGACrbzZl+U+CY4bMsuLpWV//adPrDYu7cZq670RzaMGa6anUYs= smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKW56sAh1/aT9PL4UA9l9+2lC/6OkMwIfx/Fs+Ncf/f+DdUMaC8HzkGNfq7EKlzd6mXt9WSTQecBYwFGnSH1OQw= smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKOnrNojzKmJ4frmXVXgx8FtpXvRrJRpgot+brkW8uFuelrNSuPXdazsVnH79I3ILG7iDrmIr9UFqZmBNhyAkNQ= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-10_15:00:23 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-11T17:31:23.618 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa; will attempt to use it 2024-01-11T17:31:23.618 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks 2024-01-11T17:31:23.619 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-11T17:31:23.620 INFO:teuthology.task.internal:Checking packages... 2024-01-11T17:31:23.646 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'c27531add9155519f390b228e498f0a2c5eae117' 2024-01-11T17:31:23.647 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-11T17:31:23.647 INFO:teuthology.packaging:ref: None 2024-01-11T17:31:23.647 INFO:teuthology.packaging:tag: None 2024-01-11T17:31:23.647 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240103.072409-1 2024-01-11T17:31:23.647 INFO:teuthology.packaging:sha1: c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:31:23.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20240103.072409-1 2024-01-11T17:31:23.877 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-446.gc27531ad 2024-01-11T17:31:23.878 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-11T17:31:23.884 INFO:teuthology.task.internal:no buildpackages task found 2024-01-11T17:31:23.884 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-11T17:31:23.889 INFO:teuthology.task.internal:Saving configuration 2024-01-11T17:31:23.912 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-11T17:31:23.918 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T17:31:23.943 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 17:26:24.097693', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUxAXgF7Uz1hRGAu0N2ik0dSn7OJa3/bAmjoGACrbzZl+U+CY4bMsuLpWV//adPrDYu7cZq670RzaMGa6anUYs='} 2024-01-11T17:31:23.968 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 17:26:24.099694', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKW56sAh1/aT9PL4UA9l9+2lC/6OkMwIfx/Fs+Ncf/f+DdUMaC8HzkGNfq7EKlzd6mXt9WSTQecBYwFGnSH1OQw='} 2024-01-11T17:31:23.990 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 17:26:24.101444', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKOnrNojzKmJ4frmXVXgx8FtpXvRrJRpgot+brkW8uFuelrNSuPXdazsVnH79I3ILG7iDrmIr9UFqZmBNhyAkNQ='} 2024-01-11T17:31:23.990 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-11T17:31:23.995 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-01-11T17:31:23.995 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-01-11T17:31:23.995 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-01-11T17:31:23.995 INFO:teuthology.run_tasks:Running task console_log... 2024-01-11T17:31:24.098 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f439b77b160>, signals=[15]) 2024-01-11T17:31:24.099 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-11T17:31:24.104 INFO:teuthology.task.internal:Opening connections... 2024-01-11T17:31:24.104 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:31:24.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:31:24.181 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-01-11T17:31:24.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:31:24.260 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-01-11T17:31:24.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:31:24.333 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-11T17:31:24.340 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-01-11T17:31:24.362 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-01-11T17:31:24.363 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Red Hat Enterprise Linux" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8.6 (Ootpa)" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:ID="rhel" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="fedora" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8.6" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-11T17:31:24.424 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.redhat.com/" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-11T17:31:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-11T17:31:24.426 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-01-11T17:31:24.451 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-01-11T17:31:24.473 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-01-11T17:31:24.473 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-01-11T17:31:24.534 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-11T17:31:24.535 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-11T17:31:24.536 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-11T17:31:24.536 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-11T17:31:24.536 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-01-11T17:31:24.559 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-01-11T17:31:24.581 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-01-11T17:31:24.581 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-01-11T17:31:24.642 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Red Hat Enterprise Linux" 2024-01-11T17:31:24.642 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8.6 (Ootpa)" 2024-01-11T17:31:24.642 INFO:teuthology.orchestra.run.smithi038.stdout:ID="rhel" 2024-01-11T17:31:24.642 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="fedora" 2024-01-11T17:31:24.642 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8.6" 2024-01-11T17:31:24.642 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.redhat.com/" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-11T17:31:24.643 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-11T17:31:24.643 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-01-11T17:31:24.668 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-11T17:31:24.678 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-11T17:31:24.682 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-11T17:31:24.683 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-01-11T17:31:24.685 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-01-11T17:31:24.688 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-01-11T17:31:24.708 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-11T17:31:24.714 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-11T17:31:24.714 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-01-11T17:31:24.747 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-01-11T17:31:24.749 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-01-11T17:31:24.788 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-11T17:31:24.865 INFO:teuthology.run_tasks:Running task kernel... 2024-01-11T17:31:24.880 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b'} 2024-01-11T17:31:24.880 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-01-11T17:31:24.880 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-01-11T17:31:24.880 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-11T17:31:24.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T17:31:24.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-01-11T17:31:25.163 DEBUG:teuthology.task.kernel:sha1 for client.0 is c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b 2024-01-11T17:31:25.164 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b"... 2024-01-11T17:31:25.164 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-01-11T17:31:25.185 INFO:teuthology.orchestra.run.smithi007.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-11T17:31:25.185 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b 2024-01-11T17:31:25.185 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-11T17:31:25.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum upgrade -y linux-firmware 2024-01-11T17:31:26.781 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T17:31:26.782 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-11T17:31:27.087 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:31:27.088 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T17:31:27.088 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:31:28.843 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-01-11T17:31:30.431 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-01-11T17:31:34.444 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 466 kB/s | 24 kB 00:00 2024-01-11T17:31:35.328 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-11T17:31:35.329 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-01-11T17:31:35.330 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-11T17:31:35.504 INFO:teuthology.task.kernel:Downloading kernel c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b on client.0... 2024-01-11T17:31:35.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b 2024-01-11T17:31:35.652 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b/centos/8/flavors/default/x86_64/ 2024-01-11T17:31:35.652 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gc719ee96ba78-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b/centos/8/flavors/default/x86_64/ --input-file=- 2024-01-11T17:31:35.655 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi007.front.sepia.ceph.com to complete... 2024-01-11T17:31:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11 17:31:50 URL:https://3.chacra.ceph.com/r/kernel/testing/c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b/centos/8/flavors/default/x86_64/kernel-6.6.0_gc719ee96ba78-1.x86_64.rpm [383182948/383182948] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-11T17:31:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:FINISHED --2024-01-11 17:31:50-- 2024-01-11T17:31:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:Total wall clock time: 14s 2024-01-11T17:31:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:Downloaded: 1 files, 365M in 14s (25.9 MB/s) 2024-01-11T17:31:50.175 INFO:teuthology.task.kernel:Installing kernel c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b on client.0... 2024-01-11T17:31:50.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-11T17:31:52.094 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying... ######################################## 2024-01-11T17:31:52.194 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing... ######################################## 2024-01-11T17:31:52.380 INFO:teuthology.orchestra.run.smithi007.stdout:Updating / installing... 2024-01-11T17:32:34.594 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-6.6.0_gc719ee96ba78-1 ######################################## 2024-01-11T17:32:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-11T17:33:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-11T17:35:15.232 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-11T17:35:15.232 DEBUG:teuthology.orchestra.run.smithi007:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/System.map-6.6.0-gc719ee96ba78 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/config-6.6.0-gc719ee96ba78 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/vmlinuz-6.6.0-gc719ee96ba78 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-11T17:35:15.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-11T17:35:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kernel 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kernel/cpu 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kernel/cpu/mce 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kvm 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/arch/x86/kvm/kvm.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/ansi_cprng.ko 2024-01-11T17:35:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/anubis.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/arc4.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx/async_memcpy.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx/async_pq.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx/async_tx.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx/async_xor.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/async_tx/raid6test.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/authenc.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/authencesn.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/blake2b_generic.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/blowfish_common.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/blowfish_generic.ko 2024-01-11T17:35:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/camellia_generic.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/cast5_generic.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/cast6_generic.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/cast_common.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/ccm.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/cmac.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/crc32_generic.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/crypto_engine.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/crypto_user.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/cts.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/des_generic.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/dh_generic.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/echainiv.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/essiv.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/fcrypt.ko 2024-01-11T17:35:15.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/khazad.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/lz4.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/lz4hc.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/md4.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/michael_mic.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/pcbc.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/pcrypt.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/rmd160.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/seed.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/serpent_generic.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/tcrypt.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/tea.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/twofish_common.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/twofish_generic.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/vmac.ko 2024-01-11T17:35:15.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/wp512.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/xcbc.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/xor.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/crypto/xxhash_generic.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/acpi_pad.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/custom_method.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/ec_sys.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/platform_profile.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/sbs.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/sbshc.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/acpi/video.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/acard-ahci.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/ahci_platform.ko 2024-01-11T17:35:15.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/ata_generic.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/libahci_platform.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_acpi.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_ali.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_amd.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_artop.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_atiixp.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_atp867x.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_cmd640.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_cmd64x.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_cypress.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_efar.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_hpt366.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_hpt37x.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-11T17:35:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_it8213.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_it821x.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_jmicron.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_marvell.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_mpiix.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_netcell.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_ninja32.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_ns87410.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_ns87415.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_oldpiix.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_opti.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_optidma.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_pcmcia.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-11T17:35:15.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_piccolo.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_rdc.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_sch.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_serverworks.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_sil680.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_sis.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_sl82c105.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_triflex.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pata_via.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/pdc_adma.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_inic162x.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_mv.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_nv.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_promise.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_qstor.ko 2024-01-11T17:35:15.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_sil.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_sil24.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_sis.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_svw.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_sx4.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_uli.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_via.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ata/sata_vsc.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/auxdisplay 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/auxdisplay/ks0108.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/base 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/base/regmap 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/bcma 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/bcma/bcma.ko 2024-01-11T17:35:15.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/aoe 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/aoe/aoe.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/brd.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/drbd 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/drbd/drbd.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/floppy.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/loop.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/mtip32xx 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/nbd.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/null_blk 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/null_blk/null_blk.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/pktcdvd.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/rbd.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/virtio_blk.ko 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/xen-blkback 2024-01-11T17:35:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/xen-blkfront.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/zram 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/block/zram/zram.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hangcheck-timer.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hw_random 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hw_random/via-rng.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi 2024-01-11T17:35:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/lp.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/mwave 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/mwave/mwave.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/ppdev.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tlclk.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm/tpm.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-11T17:35:15.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/uv_mmtimer.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/char/virtio_console.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/ccp 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/ccp/ccp.ko 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/intel 2024-01-11T17:35:15.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/intel/qat 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/intel/qat/qat_common 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/padlock-aes.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/padlock-sha.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/virtio 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dca 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dca/dca.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/devfreq 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dma 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dma/dw 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dma/ioat 2024-01-11T17:35:15.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/amd64_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/e752x_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/edac_mce_amd.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i3000_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i3200_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i5100_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i5400_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i7300_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i7core_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/i82975x_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/ie31200_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/sb_edac.ko 2024-01-11T17:35:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/edac/x38_edac.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/firmware 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/firmware/edd.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/gpio 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/gpio/gpio-ich.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-appleir.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-aureal.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-axff.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-dr.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-elecom.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-elo.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-emsff.ko 2024-01-11T17:35:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-gaff.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-gt683r.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-gyration.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-holtekff.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-hyperv.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-icade.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-keytouch.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-kye.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-lcpower.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-led.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-lenovo.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-11T17:35:15.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-multitouch.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-ortek.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-penmount.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-petalynx.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-picolcd.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-pl.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-plantronics.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-primax.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-rmi.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-common.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-11T17:35:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-roccat.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-saitek.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-samsung.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-sjoy.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-sony.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-speedlink.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-steelseries.ko 2024-01-11T17:35:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-sunplus.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-tivo.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-tmff.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-topseed.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-twinhan.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-uclogic.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-waltop.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-wiimote.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-xinmo.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-zpff.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/hid-zydacron.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/uhid.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hid/wacom.ko 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hv 2024-01-11T17:35:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hv/hv_balloon.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hv/hv_utils.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hv/hv_vmbus.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/abituguru.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/abituguru3.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ad7414.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ad7418.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adc128d818.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adm1025.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adm1026.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adm1029.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adm1031.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adm9240.ko 2024-01-11T17:35:15.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ads7828.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adt7410.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adt7411.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adt7462.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adt7470.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adt7475.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/adt7x10.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/amc6821.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/applesmc.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/asb100.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/asc7621.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/atxp1.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/coretemp.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-11T17:35:15.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/dme1737.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ds1621.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ds620.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/emc1403.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/emc6w201.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/f71805f.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/f71882fg.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/f75375s.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/fam15h_power.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/fschmd.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/g760a.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/g762.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/gl518sm.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/gl520sm.ko 2024-01-11T17:35:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/i5500_temp.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/i5k_amb.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ibmaem.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ibmpex.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ina209.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ina2xx.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/it87.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/k10temp.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/k8temp.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lineage-pem.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm63.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm73.ko 2024-01-11T17:35:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm75.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm77.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm78.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm80.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm83.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm85.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm87.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm90.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm92.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm93.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm95234.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm95241.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/lm95245.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc2945.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc4151.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc4215.ko 2024-01-11T17:35:15.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc4222.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc4245.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc4260.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/ltc4261.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max16065.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max1619.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max1668.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max197.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max6639.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max6650.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/max6697.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/mcp3021.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/nct6683.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/nct6775-core.ko 2024-01-11T17:35:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/nct6775.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/nct7802.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pc87360.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pc87427.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pcf8591.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-11T17:35:15.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/powr1220.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/sch5627.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/sch5636.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/sht15.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/sht21.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/shtc1.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/sis5595.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/smsc47b397.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/smsc47m1.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/smsc47m192.ko 2024-01-11T17:35:15.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/thmc50.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/tmp102.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/tmp103.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/tmp401.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/tmp421.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/via-cputemp.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/via686a.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/vt1211.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/vt8231.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83627ehf.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83627hf.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83781d.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83791d.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83792d.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83793.ko 2024-01-11T17:35:15.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83795.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83l785ts.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/hwmon/w83l786ng.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/algos 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-11T17:35:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-11T17:35:15.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/i2c-dev.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/i2c-mux.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/i2c-smbus.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/i2c/i2c-stub.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/ib_core.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-11T17:35:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/cxgb4 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/mlx4 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/mlx5 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/mthca 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/ipoib 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/iser 2024-01-11T17:35:15.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/isert 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/srp 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/srpt 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/gameport 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/gameport/gameport.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/joydev.ko 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/keyboard 2024-01-11T17:35:15.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/appletouch.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/bcm5974.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/cyapatp.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/sermouse.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/rmi4 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/serio 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/serio/altera_ps2.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/serio/arc_ps2.ko 2024-01-11T17:35:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/serio/serio_raw.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/input/sparse-keymap.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/iommu 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/iommu/amd 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/iommu/amd/iommu_v2.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/leds-blinkm.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/leds-lm3530.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/leds-lp3944.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/leds-ss4200.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-11T17:35:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/bcache 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/bcache/bcache.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-bio-prison.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-cache-smq.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-cache.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-crypt.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-delay.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-flakey.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-log-userspace.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-multipath.ko 2024-01-11T17:35:15.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-queue-length.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-raid.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-round-robin.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-service-time.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-switch.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-thin-pool.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/dm-verity.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/faulty.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/linear.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/multipath.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/persistent-data 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/raid0.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/raid1.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/raid10.ko 2024-01-11T17:35:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/md/raid456.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptbase.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptctl.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptfc.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptlan.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptsas.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptscsih.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/message/fusion/mptspi.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/lpc_ich.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/lpc_sch.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/mfd-core.ko 2024-01-11T17:35:15.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/sm501.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/viperboard.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/vx855.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/mfd/wl1273-core.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/altera-stapl 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/apds9802als.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/apds990x.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/bh1770glc.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/cb710 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/cb710/cb710.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/eeprom 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/eeprom/at24.ko 2024-01-11T17:35:15.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/eeprom/eeprom.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/eeprom/max6875.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/enclosure.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/hpilo.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/isl29003.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/isl29020.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/lis3lv02d 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/mei 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/mei/mei-me.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/mei/mei-txe.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/mei/mei.ko 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/sgi-gru 2024-01-11T17:35:15.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/sgi-xp 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/tifm_7xx1.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/tifm_core.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/tsl2550.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/vmw_balloon.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/vmw_vmci 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/bonding 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/bonding/bonding.ko 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa/b53 2024-01-11T17:35:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa/mv88e6xxx 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/dummy.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/eql.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/3com 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/8390 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-11T17:35:15.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/adaptec 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/agere 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/alteon 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/altera 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/amd 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/alx 2024-01-11T17:35:15.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atlx 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/genet 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-11T17:35:15.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/brocade 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/brocade/bna 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/cisco 2024-01-11T17:35:15.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/cisco/enic 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dlink 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/dnet.ko 2024-01-11T17:35:15.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/emulex 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/emulex/benet 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/ethoc.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/fealnx.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/e1000 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/e1000e 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/fm10k 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/i40e 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/iavf 2024-01-11T17:35:15.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/igb 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/igbvf 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/jme.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/marvell 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox 2024-01-11T17:35:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/micrel 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/myricom 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/natsemi 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/neterion 2024-01-11T17:35:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/nvidia 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/packetengines 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/qlogic 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/rdc 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/realtek 2024-01-11T17:35:15.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/rocker 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sfc 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/silan 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sis 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/smsc 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/stmicro 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sun 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/tehuti 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/ti 2024-01-11T17:35:15.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/via 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/wiznet 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/xircom 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/geneve.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/hyperv 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ieee802154 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ifb.ko 2024-01-11T17:35:15.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ipvlan 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/macvlan.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/macvtap.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/mdio 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/mdio.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/mii.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/net_failover.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/netconsole.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/nlmon.ko 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/pcs 2024-01-11T17:35:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/amd.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/bcm87xx.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/broadcom.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/cicada.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/davicom.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/dp83640.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/et1011c.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/icplus.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/lxt.ko 2024-01-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/marvell.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/micrel.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/national.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/phylink.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/qsemi.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/realtek.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/smsc.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/ste10Xp.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/phy/vitesse.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/ppp_async.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-11T17:35:15.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/pppoe.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/pppox.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/ppp/pptp.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/slip 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/slip/slhc.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/slip/slip.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/sungem_phy.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/tap.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team/team.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team/team_mode_random.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-11T17:35:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/tun.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/veth.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/virtio_net.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/vmxnet3 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/vxlan 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/vxlan/vxlan.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/xen-netback 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/net/xen-netfront.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ntb 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ntb/ntb.ko 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/nvme 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/nvme/host 2024-01-11T17:35:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/nvme/host/nvme-core.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/nvme/host/nvme.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/parport 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/parport/parport.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/parport/parport_cs.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/parport/parport_pc.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/parport/parport_serial.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pci 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pci/hotplug 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pci/pcie 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pci/xen-pcifront.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pcmcia 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pcmcia/i82092.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pcmcia/pd6729.ko 2024-01-11T17:35:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pinctrl 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pinctrl/intel 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/acerhdf.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-11T17:35:15.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/hdaps.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/intel 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/intel_ips.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/platform/x86/wmi.ko 2024-01-11T17:35:15.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/powercap 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pps 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pps/clients 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pps/clients/pps_parport.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/pps/pps_core.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ptp 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ptp/ptp.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ptp/ptp_kvm.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-11T17:35:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-em3027.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-11T17:35:15.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-max6900.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/rtc/rtc-x1205.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/3w-9xxx.ko 2024-01-11T17:35:15.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/3w-sas.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/3w-xxxx.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/BusLogic.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/a100u2w.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/aacraid 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/advansys.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/aic7xxx 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/am53c974.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/arcmsr 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/atp870u.ko 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/be2iscsi 2024-01-11T17:35:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/bfa 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/bfa/bfa.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/bnx2fc 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/bnx2i 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/ch.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/csiostor 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/cxgbi 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-11T17:35:15.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/dc395x.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/device_handler 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/dmx3191d.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/esas2r 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/esp_scsi.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/fcoe 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/fnic 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/fnic/fnic.ko 2024-01-11T17:35:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/hpsa.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/hptiop.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/hv_storvsc.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/initio.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/ipr.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/ips.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/isci 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/isci/isci.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/libfc 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/libfc/libfc.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/libiscsi.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/libsas 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/libsas/libsas.ko 2024-01-11T17:35:15.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/lpfc 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/megaraid 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/megaraid.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/mpt3sas 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/mvsas 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/mvumi.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/pm8001 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/pmcraid.ko 2024-01-11T17:35:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/qla1280.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/qla2xxx 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/qla4xxx 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/raid_class.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/scsi_debug.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/ses.ko 2024-01-11T17:35:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/st.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/stex.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/sym53c8xx_2 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/virtio_scsi.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/scsi/wd719x.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ssb 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ssb/ssb.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/iscsi 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/loopback 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/target_core_file.ko 2024-01-11T17:35:15.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/target_core_iblock.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/target_core_mod.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/target_core_pscsi.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/target_core_user.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/tcm_fc 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-11T17:35:15.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/ipwireless 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/n_gsm.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/n_hdlc.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/nozomi.ko 2024-01-11T17:35:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/serial 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/serial/8250 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/serial/arc_uart.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/serial/jsm 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ufs 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ufs/core 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ufs/host 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/uio 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/uio/uio.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/uio/uio_aec.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/uio/uio_cif.ko 2024-01-11T17:35:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/uio/uio_pci_generic.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/uio/uio_sercos3.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/class 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/class/cdc-acm.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/class/usblp.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/class/usbtmc.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/host 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/image 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/image/mdc800.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/image/microtek.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/adutux.ko 2024-01-11T17:35:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/appledisplay.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/emi26.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/emi62.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/ezusb.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/idmouse.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/iowarrior.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/ldusb.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/legousbtower.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/sisusbvga 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/usb3503.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/usblcd.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/uss720.ko 2024-01-11T17:35:15.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/misc/yurex.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/phy 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/phy/phy-generic.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/aircable.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ark3116.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ch341.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/cp210x.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/cyberjack.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/empeg.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-11T17:35:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/io_ti.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ipaq.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ipw.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ir-usb.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/keyspan.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/mct_u232.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/mos7720.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/mos7840.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/navman.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/omninet.ko 2024-01-11T17:35:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/opticon.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/option.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/oti6858.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/pl2303.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/qcaux.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/qcserial.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/quatech2.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/safe_serial.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/sierra.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ssu100.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/symbolserial.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/usb_debug.ko 2024-01-11T17:35:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/visor.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/whiteheat.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/uas.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-karma.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-11T17:35:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/storage/usb-storage.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/usbip 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vfio 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vfio/pci 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vfio/vfio.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vhost 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vhost/vhost.ko 2024-01-11T17:35:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vhost/vhost_net.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/vhost/vhost_scsi.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/video 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/video/backlight 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/video/backlight/apple_bl.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/video/backlight/lcd.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio_balloon.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio_mmio.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio_pci.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-11T17:35:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/virtio/virtio_ring.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/masters 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/masters/ds2482.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/masters/ds2490.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-11T17:35:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/w1/wire.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/hpwdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/i6300esb.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/ib700wdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/ibmasr.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-11T17:35:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/it87_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/machzwd.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/nv_tco.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/softdog.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/via_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/wdt_pci.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/watchdog/xen_wdt.ko 2024-01-11T17:35:15.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-evtchn.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-gntalloc.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-gntdev.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-pciback 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xen-privcmd.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xenfs 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/9p 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/9p/9p.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/affs 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/affs/affs.ko 2024-01-11T17:35:15.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/befs 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/befs/befs.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/binfmt_misc.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/btrfs 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/btrfs/btrfs.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/cachefiles 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/cachefiles/cachefiles.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ceph 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ceph/ceph.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/coda 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/coda/coda.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/cramfs 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/cramfs/cramfs.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/dlm 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/dlm/dlm.ko 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ecryptfs 2024-01-11T17:35:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/f2fs 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/f2fs/f2fs.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fat 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fat/fat.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fat/msdos.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fat/vfat.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fscache 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fscache/fscache.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fuse 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fuse/cuse.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/fuse/fuse.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/gfs2 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/gfs2/gfs2.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/hfs 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/hfs/hfs.ko 2024-01-11T17:35:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/hfsplus 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/hfsplus/hfsplus.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/isofs 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/isofs/isofs.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/jfs 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/jfs/jfs.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/lockd 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/lockd/lockd.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/minix 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/minix/minix.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/netfs 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/netfs/netfs.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/blocklayout 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-11T17:35:15.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/filelayout 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/flexfilelayout 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/nfs.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/nfsv3.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs/nfsv4.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs_common 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs_common/grace.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfs_common/nfs_acl.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfsd 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nfsd/nfsd.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nilfs2 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nilfs2/nilfs2.ko 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls 2024-01-11T17:35:15.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-celtic.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-centeuro.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-croatian.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-cyrillic.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-gaelic.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-greek.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-iceland.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-inuit.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-roman.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-romanian.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/mac-turkish.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp1250.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp1251.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp1255.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp737.ko 2024-01-11T17:35:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp775.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp850.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp852.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp855.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp857.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp860.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp861.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp862.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp863.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp864.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp865.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp866.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp869.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp874.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp932.ko 2024-01-11T17:35:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp936.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp949.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_cp950.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_euc-jp.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-1.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-13.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-14.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-15.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-2.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-3.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-4.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-5.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-6.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-7.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_iso8859-9.ko 2024-01-11T17:35:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_koi8-r.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_koi8-ru.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_koi8-u.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/nls/nls_utf8.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/cluster 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/dlm 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/dlmfs 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/ocfs2.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-11T17:35:15.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/overlayfs 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/overlayfs/overlay.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/pstore 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/pstore/ramoops.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/reiserfs 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/reiserfs/reiserfs.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/romfs 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/romfs/romfs.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/smb 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/smb/client 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/smb/client/cifs.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/smb/common 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/smb/common/cifs_arc4.ko 2024-01-11T17:35:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/smb/common/cifs_md4.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/squashfs 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/squashfs/squashfs.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/sysv 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/sysv/sysv.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/udf 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/udf/udf.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ufs 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/ufs/ufs.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/xfs 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/fs/xfs/xfs.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/kernel 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/kernel/rcu 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/kernel/rcu/rcutorture.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/kernel/torture.ko 2024-01-11T17:35:15.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/kernel/trace 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/asn1_encoder.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/crc-itu-t.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/crc8.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/crypto 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/crypto/libarc4.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/crypto/libdes.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/libcrc32c.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/lru_cache.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/lz4 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/lz4/lz4_compress.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/lz4/lz4hc_compress.ko 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/math 2024-01-11T17:35:15.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/math/cordic.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/raid6 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/raid6/raid6_pq.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/reed_solomon 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/ts_bm.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/ts_fsm.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/ts_kmp.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/zstd 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/lib/zstd/zstd_compress.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/mm 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/mm/hwpoison-inject.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/6lowpan.ko 2024-01-11T17:35:15.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_dest.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_fragment.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_hop.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_mobility.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_routing.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/6lowpan/nhc_udp.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/802 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/802/garp.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/802/mrp.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/802/p8022.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/802/psnap.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/802/stp.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/8021q 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/8021q/8021q.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/9p 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/9p/9pnet.ko 2024-01-11T17:35:15.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/9p/9pnet_fd.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/9p/9pnet_rdma.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/9p/9pnet_virtio.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/batman-adv 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/batman-adv/batman-adv.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/br_netfilter.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/bridge.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-11T17:35:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/ebtables.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-11T17:35:15.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ceph 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ceph/libceph.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/core 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/core/failover.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/core/pktgen.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dccp 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dccp/dccp.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dccp/dccp_diag.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dccp/dccp_ipv4.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dccp/dccp_ipv6.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dns_resolver 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dns_resolver/dns_resolver.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dsa 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dsa/dsa_core.ko 2024-01-11T17:35:15.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dsa/tag_brcm.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dsa/tag_dsa.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dsa/tag_none.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/dsa/tag_trailer.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ieee802154 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ieee802154/6lowpan 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ieee802154/ieee802154.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/ah4.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/esp4.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/fou.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/gre.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/inet_diag.ko 2024-01-11T17:35:15.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/ip_gre.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/ip_tunnel.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/ip_vti.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/ipcomp.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/ipip.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-11T17:35:15.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_bic.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_dctcp.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_diag.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_highspeed.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_htcp.ko 2024-01-11T17:35:15.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_hybla.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_illinois.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_lp.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_scalable.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_vegas.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_veno.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_westwood.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tcp_yeah.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/tunnel4.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/udp_diag.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/udp_tunnel.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/ah6.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/esp6.ko 2024-01-11T17:35:15.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/fou6.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/ip6_tunnel.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/ip6_vti.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/ipcomp6.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-11T17:35:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/sit.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/tunnel6.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-11T17:35:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/key 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/key/af_key.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_core.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_eth.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_ip.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_ip6.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_netlink.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/l2tp/l2tp_ppp.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/llc 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/llc/llc.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/mac802154 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/mac802154/mac802154.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/mpls 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/mpls/mpls_gso.ko 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter 2024-01-11T17:35:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-11T17:35:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-11T17:35:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conncount.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-11T17:35:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_flow_table.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_log_syslog.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_nat.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_nat_irc.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_nat_sip.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nf_tables.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nfnetlink.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nfnetlink_log.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-11T17:35:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_chain_nat.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_compat.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_ct.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_fib.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_hash.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_limit.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_log.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_masq.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_nat.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_numgen.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_queue.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_quota.ko 2024-01-11T17:35:15.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_redir.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_reject.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_reject_inet.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_socket.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/nft_tunnel.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_AUDIT.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_CT.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_DSCP.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_HL.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_HMARK.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_LED.ko 2024-01-11T17:35:15.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_LOG.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_NETMAP.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_NFLOG.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_RATEEST.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_SECMARK.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_TEE.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_TPROXY.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_TRACE.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_addrtype.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_bpf.ko 2024-01-11T17:35:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_cgroup.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_cluster.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_comment.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_connbytes.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_connlabel.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_connlimit.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_connmark.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_conntrack.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_cpu.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_dccp.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_devgroup.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_dscp.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_ecn.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_esp.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_hashlimit.ko 2024-01-11T17:35:15.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_helper.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_hl.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_ipcomp.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_iprange.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_ipvs.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_l2tp.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_length.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_limit.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_mac.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_mark.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_multiport.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_nat.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_nfacct.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_osf.ko 2024-01-11T17:35:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_owner.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_physdev.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_pkttype.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_policy.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_quota.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_rateest.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_realm.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_recent.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_sctp.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_set.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_socket.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_state.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_statistic.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_string.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_tcpmss.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_time.ko 2024-01-11T17:35:15.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netfilter/xt_u32.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netlink 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/netlink/netlink_diag.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/nsh 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/nsh/nsh.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/openvswitch 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/openvswitch/openvswitch.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/openvswitch/vport-geneve.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/openvswitch/vport-gre.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/openvswitch/vport-vxlan.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/packet 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/packet/af_packet_diag.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/rds 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/rds/rds.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/rds/rds_rdma.ko 2024-01-11T17:35:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/rds/rds_tcp.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_csum.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_gact.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_ipt.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_mirred.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_nat.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_pedit.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_police.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_simple.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_skbedit.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/act_vlan.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/cls_basic.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/cls_bpf.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/cls_flow.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/cls_fw.ko 2024-01-11T17:35:15.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/cls_route.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/cls_u32.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/em_cmp.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/em_ipset.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/em_meta.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/em_nbyte.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/em_text.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/em_u32.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_choke.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_codel.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_drr.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_fq.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_gred.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_hfsc.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_hhf.ko 2024-01-11T17:35:15.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_htb.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_ingress.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_mqprio.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_mqprio_lib.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_multiq.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_netem.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_pie.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_plug.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_prio.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_qfq.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_red.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_sfb.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_sfq.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_tbf.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sched/sch_teql.ko 2024-01-11T17:35:15.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sctp 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sctp/sctp.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sctp/sctp_diag.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc/auth_gss 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc/sunrpc.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc/xprtrdma 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/tipc 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/tipc/diag.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/tipc/tipc.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/unix 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/unix/unix_diag.ko 2024-01-11T17:35:15.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/vmw_vsock 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/vmw_vsock/vsock.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/xfrm 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/security 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/security/keys 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/security/keys/encrypted-keys 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/security/keys/trusted-keys 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/security/keys/trusted-keys/trusted.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/virt 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/virt/lib 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/kernel/virt/lib/irqbypass.ko 2024-01-11T17:35:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.alias 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.alias.bin 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.builtin 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.builtin.bin 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.builtin.modinfo 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.dep 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.dep.bin 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.devname 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.order 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.softdep 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.symbols 2024-01-11T17:35:15.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.6.0-gc719ee96ba78/modules.symbols.bin 2024-01-11T17:35:15.421 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gc719ee96ba78 2024-01-11T17:35:15.421 DEBUG:teuthology.orchestra.run.smithi007:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:postinstall scriptlet (using /bin/sh): 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gc719ee96ba78 -a -r /boot/System.map-6.6.0-gc719ee96ba78 ]; then 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:cp /boot/vmlinuz-6.6.0-gc719ee96ba78 /boot/.vmlinuz-6.6.0-gc719ee96ba78-rpm 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:cp /boot/System.map-6.6.0-gc719ee96ba78 /boot/.System.map-6.6.0-gc719ee96ba78-rpm 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:rm -f /boot/vmlinuz-6.6.0-gc719ee96ba78 /boot/System.map-6.6.0-gc719ee96ba78 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/installkernel 6.6.0-gc719ee96ba78 /boot/.vmlinuz-6.6.0-gc719ee96ba78-rpm /boot/.System.map-6.6.0-gc719ee96ba78-rpm 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:rm -f /boot/.vmlinuz-6.6.0-gc719ee96ba78-rpm /boot/.System.map-6.6.0-gc719ee96ba78-rpm 2024-01-11T17:35:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:new-kernel-pkg --remove 6.6.0-gc719ee96ba78 --rminitrd --initrdfile=/boot/initramfs-6.6.0-gc719ee96ba78.img 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-install remove 6.6.0-gc719ee96ba78 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/update-bootloader --remove 6.6.0-gc719ee96ba78 2024-01-11T17:35:15.451 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-01-11T17:35:15.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-01-11T17:35:15.534 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-01-11T17:35:15.534 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-01-11T17:35:15.534 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.02 2024-01-11T17:35:15.534 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 123.el8 2024-01-11T17:35:15.534 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-01-11T17:35:15.534 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Group : System Environment/Base 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 9265450 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Relocations : (not relocatable) 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : Red Hat, Inc. 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : Red Hat, Inc. 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:35:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-11T17:35:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-11T17:35:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-11T17:35:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-01-11T17:35:15.536 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:35:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-01-11T17:35:15.537 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-11T17:35:15.537 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 6.6.0-gc719ee96ba78 2024-01-11T17:35:15.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-11T17:35:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-01-11T17:35:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-01-11T17:35:18.323 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-01-11T17:35:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.u7A5mpazLC 2024-01-11T17:35:18.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.u7A5mpazLC 2024-01-11T17:35:18.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.u7A5mpazLC 2024-01-11T17:35:18.601 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.u7A5mpazLC is 6KB 2024-01-11T17:35:18.627 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.u7A5mpazLC 2024-01-11T17:35:18.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-01-11T17:35:18.720 INFO:teuthology.orchestra.run.smithi007.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-01-11T17:35:18.720 INFO:teuthology.orchestra.run.smithi007.stdout:2db68f91c8f90721a6596efe657a5324-6.6.0-gc719ee96ba78.conf 2024-01-11T17:35:18.720 INFO:teuthology.orchestra.run.smithi007.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-01-11T17:35:18.720 INFO:teuthology.orchestra.run.smithi007.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-01-11T17:35:18.720 INFO:teuthology.orchestra.run.smithi007.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-01-11T17:35:18.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.6.0-gc719ee96ba78 2024-01-11T17:35:18.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-01-11T17:35:48.887 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-01-11T17:35:48.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:35:48.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:36:07.228 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-11T17:36:16.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:36:16.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:36:19.292 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-11T17:36:31.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:36:31.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:36:49.728 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-11T17:37:04.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:37:04.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:37:04.991 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-01-11T17:37:05.388 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:37:05.389 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-gc719ee96ba78"... 2024-01-11T17:37:05.389 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-01-11T17:37:05.431 INFO:teuthology.orchestra.run.smithi007.stdout:6.6.0-gc719ee96ba78 2024-01-11T17:37:05.431 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gc719ee96ba78 vs 6.6.0-gc719ee96ba78 2024-01-11T17:37:05.431 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-11T17:37:05.431 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-01-11T17:37:06.433 DEBUG:teuthology.parallel:result is None 2024-01-11T17:37:06.433 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-11T17:37:06.441 INFO:teuthology.task.internal:Creating test directory... 2024-01-11T17:37:06.441 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-11T17:37:06.446 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-11T17:37:06.448 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-11T17:37:06.488 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-11T17:37:06.495 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-11T17:37:06.503 INFO:teuthology.task.internal:Creating archive directory... 2024-01-11T17:37:06.503 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-11T17:37:06.533 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-11T17:37:06.535 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-11T17:37:06.594 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-11T17:37:06.601 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-11T17:37:06.601 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-11T17:37:06.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T17:37:06.628 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-11T17:37:06.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T17:37:06.648 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-11T17:37:06.653 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-11T17:37:06.671 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-11T17:37:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T17:37:06.757 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T17:37:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T17:37:06.842 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T17:37:06.859 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T17:37:06.882 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T17:37:06.884 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-11T17:37:06.891 INFO:teuthology.task.internal:Configuring sudo... 2024-01-11T17:37:06.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-11T17:37:06.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-11T17:37:06.899 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-11T17:37:06.962 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-11T17:37:06.971 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-11T17:37:06.971 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-11T17:37:06.975 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-11T17:37:06.977 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-11T17:37:07.013 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T17:37:07.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T17:37:07.194 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T17:37:07.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T17:37:07.348 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:37:07.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-11T17:37:07.417 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T17:37:07.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T17:37:07.534 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T17:37:07.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T17:37:07.632 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:37:07.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-11T17:37:07.706 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T17:37:07.755 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T17:37:07.831 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T17:37:07.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T17:37:07.929 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:37:07.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-11T17:37:08.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-01-11T17:37:08.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-01-11T17:37:08.011 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-01-11T17:37:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T17:37:08.105 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T17:37:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T17:37:08.538 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-11T17:37:08.546 INFO:teuthology.task.internal:Starting timer... 2024-01-11T17:37:08.546 INFO:teuthology.run_tasks:Running task pcp... 2024-01-11T17:37:08.556 INFO:teuthology.run_tasks:Running task selinux... 2024-01-11T17:37:08.564 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-01-11T17:37:08.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-01-11T17:37:08.713 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-01-11T17:37:08.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-01-11T17:37:08.792 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-01-11T17:37:08.795 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-01-11T17:37:08.909 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-01-11T17:37:08.912 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-11T17:37:08.912 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-01-11T17:37:08.967 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-01-11T17:37:08.968 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-01-11T17:37:08.999 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-01-11T17:37:08.999 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-01-11T17:37:09.038 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-01-11T17:37:09.039 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi038.front.sepia.ceph.com': 'permissive'} 2024-01-11T17:37:09.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T17:37:09.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T17:37:09.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T17:37:09.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T17:37:09.148 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T17:37:09.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T17:37:09.184 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-11T17:37:09.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-01-11T17:37:09.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-01-11T17:37:09.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-01-11T17:37:09.327 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-11T17:37:09.337 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-01-11T17:37:09.341 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-01-11T17:37:09.473 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-11T17:37:09.487 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-11T17:37:09.488 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi016.front.sepia.ceph.com,smithi038.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-11T17:44:55.196 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi038.front.sepia.ceph.com')] 2024-01-11T17:44:55.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:44:55.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:44:55.298 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-01-11T17:44:55.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T17:44:55.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-01-11T17:44:55.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:44:55.493 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-01-11T17:44:55.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-01-11T17:44:55.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-01-11T17:44:55.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T17:44:55.648 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-01-11T17:44:55.721 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-01-11T17:44:55.722 INFO:teuthology.run_tasks:Running task clock... 2024-01-11T17:44:55.735 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-11T17:44:55.735 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-11T17:44:55.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T17:44:55.740 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-11T17:44:55.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T17:44:55.742 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-11T17:44:55.743 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T17:44:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-11T17:44:55.798 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-11T17:44:55.802 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-11T17:44:55.814 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-11T17:44:55.819 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-11T17:44:55.850 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-11T17:44:55.883 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-01-11T17:44:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-01-11T17:44:55.897 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-01-11T17:44:55.909 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-01-11T17:44:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-11T17:44:55.927 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-11T17:44:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-11T17:44:55.944 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-11T17:44:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-01-11T17:44:55.977 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-01-11T17:44:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2024-01-11T17:44:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T17:44:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-01-11T17:44:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.009 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-11T17:44:56.030 INFO:teuthology.orchestra.run.smithi038.stderr:bash: ntpq: command not found 2024-01-11T17:44:56.036 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T17:44:56.036 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-01-11T17:44:56.036 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.036 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.036 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.036 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.047 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-11T17:44:56.153 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2024-01-11T17:44:56.160 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T17:44:56.161 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-01-11T17:44:56.161 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.161 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.161 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.161 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T17:44:56.162 INFO:teuthology.run_tasks:Running task pexec... 2024-01-11T17:44:56.172 INFO:teuthology.task.pexec:Executing custom commands... 2024-01-11T17:44:56.172 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:44:56.172 DEBUG:teuthology.orchestra.run.smithi007:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-01-11T17:44:56.173 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi016.front.sepia.ceph.com 2024-01-11T17:44:56.173 DEBUG:teuthology.orchestra.run.smithi016:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-01-11T17:44:56.173 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi038.front.sepia.ceph.com 2024-01-11T17:44:56.173 DEBUG:teuthology.orchestra.run.smithi038:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-01-11T17:44:56.607 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2024-01-11T17:44:56.608 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-01-11T17:44:56.694 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T17:44:57.451 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 143 kB/s | 4.5 kB 00:00 2024-01-11T17:44:57.460 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 131 kB/s | 4.1 kB 00:00 2024-01-11T17:44:57.605 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 129 kB/s | 4.1 kB 00:00 2024-01-11T17:44:57.803 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 141 kB/s | 4.5 kB 00:00 2024-01-11T17:44:57.905 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 143 kB/s | 4.5 kB 00:00 2024-01-11T17:44:58.008 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 126 kB/s | 4.1 kB 00:00 2024-01-11T17:45:01.220 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-01-11T17:45:01.221 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-01-11T17:45:01.221 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout:Resetting modules: 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout: container-tools 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-01-11T17:45:01.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:01.296 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout:Resetting modules: 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout: container-tools 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-01-11T17:45:01.298 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:Resetting modules: 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: container-tools 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-01-11T17:45:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-01-11T17:45:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:06.831 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-01-11T17:45:06.913 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-01-11T17:45:07.365 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:07.391 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-11T17:45:07.461 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:08.031 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:08.183 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 116 kB/s | 4.5 kB 00:00 2024-01-11T17:45:08.344 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 105 kB/s | 4.1 kB 00:00 2024-01-11T17:45:08.659 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 122 kB/s | 4.5 kB 00:00 2024-01-11T17:45:08.747 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 112 kB/s | 4.1 kB 00:00 2024-01-11T17:45:08.955 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 105 kB/s | 4.1 kB 00:00 2024-01-11T17:45:09.270 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 123 kB/s | 4.5 kB 00:00 2024-01-11T17:45:12.005 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-01-11T17:45:12.146 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-01-11T17:45:12.185 INFO:teuthology.orchestra.run.smithi038.stdout:========================================================================================================== 2024-01-11T17:45:12.185 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-01-11T17:45:12.185 INFO:teuthology.orchestra.run.smithi038.stdout:========================================================================================================== 2024-01-11T17:45:12.185 INFO:teuthology.orchestra.run.smithi038.stdout:Installing group/module packages: 2024-01-11T17:45:12.185 INFO:teuthology.orchestra.run.smithi038.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-01-11T17:45:12.186 INFO:teuthology.orchestra.run.smithi038.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout:Installing module profiles: 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout: container-tools/common 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling module streams: 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout: container-tools rhel8 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout:========================================================================================================== 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout:Install 13 Packages 2024-01-11T17:45:12.187 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:12.188 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 26 M 2024-01-11T17:45:12.188 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 87 M 2024-01-11T17:45:12.188 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-01-11T17:45:12.285 INFO:teuthology.orchestra.run.smithi038.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 258 kB/s | 25 kB 00:00 2024-01-11T17:45:12.310 INFO:teuthology.orchestra.run.smithi038.stdout:(2/13): yajl-2.1.0-12.el8.x86_64.rpm 338 kB/s | 41 kB 00:00 2024-01-11T17:45:12.313 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================================== 2024-01-11T17:45:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================================== 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout:Installing group/module packages: 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-01-11T17:45:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout:Installing module profiles: 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: container-tools/common 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling module streams: 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: container-tools rhel8 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================================== 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout:Install 13 Packages 2024-01-11T17:45:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:12.316 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 26 M 2024-01-11T17:45:12.316 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 87 M 2024-01-11T17:45:12.316 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-01-11T17:45:12.336 INFO:teuthology.orchestra.run.smithi038.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 641 kB/s | 94 kB 00:00 2024-01-11T17:45:12.378 INFO:teuthology.orchestra.run.smithi038.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 11 MB/s | 1.0 MB 00:00 2024-01-11T17:45:12.388 INFO:teuthology.orchestra.run.smithi016.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2024-01-11T17:45:12.413 INFO:teuthology.orchestra.run.smithi016.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 258 kB/s | 25 kB 00:00 2024-01-11T17:45:12.428 INFO:teuthology.orchestra.run.smithi038.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 7.8 MB/s | 739 kB 00:00 2024-01-11T17:45:12.439 INFO:teuthology.orchestra.run.smithi016.stdout:(3/13): yajl-2.1.0-12.el8.x86_64.rpm 338 kB/s | 41 kB 00:00 2024-01-11T17:45:12.453 INFO:teuthology.orchestra.run.smithi038.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 524 kB/s | 39 kB 00:00 2024-01-11T17:45:12.487 INFO:teuthology.orchestra.run.smithi038.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 4.0 MB/s | 239 kB 00:00 2024-01-11T17:45:12.489 INFO:teuthology.orchestra.run.smithi016.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 10 MB/s | 1.0 MB 00:00 2024-01-11T17:45:12.531 INFO:teuthology.orchestra.run.smithi016.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 7.9 MB/s | 739 kB 00:00 2024-01-11T17:45:12.556 INFO:teuthology.orchestra.run.smithi016.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 588 kB/s | 39 kB 00:00 2024-01-11T17:45:12.570 INFO:teuthology.orchestra.run.smithi038.stdout:(8/13): python3-podman-4.6.0-1.module+el8.9.0+2 1.8 MB/s | 153 kB 00:00 2024-01-11T17:45:12.581 INFO:teuthology.orchestra.run.smithi016.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 4.7 MB/s | 239 kB 00:00 2024-01-11T17:45:12.658 INFO:teuthology.orchestra.run.smithi016.stdout:(8/13): python3-podman-4.6.0-1.module+el8.9.0+2 2.0 MB/s | 153 kB 00:00 2024-01-11T17:45:12.662 INFO:teuthology.orchestra.run.smithi038.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 18 MB/s | 3.7 MB 00:00 2024-01-11T17:45:12.766 INFO:teuthology.orchestra.run.smithi016.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 18 MB/s | 3.7 MB 00:00 2024-01-11T17:45:12.821 INFO:teuthology.orchestra.run.smithi038.stdout:(10/13): buildah-1.31.3-1.module+el8.9.0+20326+ 17 MB/s | 8.8 MB 00:00 2024-01-11T17:45:12.888 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-11T17:45:12.911 INFO:teuthology.orchestra.run.smithi038.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 10 MB/s | 2.5 MB 00:00 2024-01-11T17:45:12.933 INFO:teuthology.orchestra.run.smithi016.stdout:(10/13): buildah-1.31.3-1.module+el8.9.0+20326+ 17 MB/s | 8.8 MB 00:00 2024-01-11T17:45:12.939 INFO:teuthology.orchestra.run.smithi038.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 423 kB/s | 49 kB 00:00 2024-01-11T17:45:13.008 INFO:teuthology.orchestra.run.smithi016.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 10 MB/s | 2.5 MB 00:00 2024-01-11T17:45:13.033 INFO:teuthology.orchestra.run.smithi016.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 494 kB/s | 49 kB 00:00 2024-01-11T17:45:13.064 INFO:teuthology.orchestra.run.smithi007.stdout:========================================================================================================== 2024-01-11T17:45:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-01-11T17:45:13.064 INFO:teuthology.orchestra.run.smithi007.stdout:========================================================================================================== 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout:Installing group/module packages: 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-01-11T17:45:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout:Installing module profiles: 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: container-tools/common 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling module streams: 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: container-tools rhel8 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:13.066 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-01-11T17:45:13.067 INFO:teuthology.orchestra.run.smithi007.stdout:========================================================================================================== 2024-01-11T17:45:13.067 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-01-11T17:45:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:13.068 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 26 M 2024-01-11T17:45:13.068 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 87 M 2024-01-11T17:45:13.068 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-01-11T17:45:13.072 INFO:teuthology.orchestra.run.smithi038.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 16 MB/s | 8.1 MB 00:00 2024-01-11T17:45:13.072 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:45:13.072 INFO:teuthology.orchestra.run.smithi038.stdout:Total 29 MB/s | 26 MB 00:00 2024-01-11T17:45:13.133 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 384 kB/s | 25 kB 00:00 2024-01-11T17:45:13.158 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): yajl-2.1.0-12.el8.x86_64.rpm 456 kB/s | 41 kB 00:00 2024-01-11T17:45:13.159 INFO:teuthology.orchestra.run.smithi016.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 16 MB/s | 8.1 MB 00:00 2024-01-11T17:45:13.159 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:45:13.159 INFO:teuthology.orchestra.run.smithi016.stdout:Total 30 MB/s | 26 MB 00:00 2024-01-11T17:45:13.175 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 877 kB/s | 94 kB 00:00 2024-01-11T17:45:13.234 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 10 MB/s | 1.0 MB 00:00 2024-01-11T17:45:13.276 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 7.2 MB/s | 739 kB 00:00 2024-01-11T17:45:13.301 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 587 kB/s | 39 kB 00:00 2024-01-11T17:45:13.343 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 3.5 MB/s | 239 kB 00:00 2024-01-11T17:45:13.375 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-01-11T17:45:13.385 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-podman-4.6.0-1.module+el8.9.0+2 3.6 MB/s | 153 kB 00:00 2024-01-11T17:45:13.400 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-01-11T17:45:13.400 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-01-11T17:45:13.464 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-01-11T17:45:13.488 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-01-11T17:45:13.489 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-01-11T17:45:13.527 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 16 MB/s | 3.7 MB 00:00 2024-01-11T17:45:13.655 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-01-11T17:45:13.665 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-01-11T17:45:13.671 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 18 MB/s | 2.5 MB 00:00 2024-01-11T17:45:13.745 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-01-11T17:45:13.754 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-01-11T17:45:13.794 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): buildah-1.31.3-1.module+el8.9.0+20326+ 14 MB/s | 8.8 MB 00:00 2024-01-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 329 kB/s | 49 kB 00:00 2024-01-11T17:45:13.945 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 15 MB/s | 8.1 MB 00:00 2024-01-11T17:45:13.945 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:45:13.945 INFO:teuthology.orchestra.run.smithi007.stdout:Total 29 MB/s | 26 MB 00:00 2024-01-11T17:45:14.340 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-01-11T17:45:14.400 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-01-11T17:45:14.420 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-01-11T17:45:14.448 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-01-11T17:45:14.448 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-01-11T17:45:14.606 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2024-01-11T17:45:14.691 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2024-01-11T17:45:15.056 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-01-11T17:45:15.066 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-01-11T17:45:15.176 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2024-01-11T17:45:15.302 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2024-01-11T17:45:15.365 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2024-01-11T17:45:15.476 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2024-01-11T17:45:15.595 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2024-01-11T17:45:15.688 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2024-01-11T17:45:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-01-11T17:45:15.830 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-01-11T17:45:15.915 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-01-11T17:45:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2024-01-11T17:45:16.072 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-01-11T17:45:16.164 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-01-11T17:45:16.333 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2024-01-11T17:45:16.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2024-01-11T17:45:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2024-01-11T17:45:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2024-01-11T17:45:16.990 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2024-01-11T17:45:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-01-11T17:45:17.330 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2024-01-11T17:45:17.377 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2024-01-11T17:45:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-01-11T17:45:17.563 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2024-01-11T17:45:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2024-01-11T17:45:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2024-01-11T17:45:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2024-01-11T17:45:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2024-01-11T17:45:18.748 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2024-01-11T17:45:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-01-11T17:45:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2024-01-11T17:45:19.021 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-01-11T17:45:19.230 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2024-01-11T17:45:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2024-01-11T17:45:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2024-01-11T17:45:19.453 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-01-11T17:45:19.468 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2024-01-11T17:45:19.915 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2024-01-11T17:45:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2024-01-11T17:45:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2024-01-11T17:45:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2024-01-11T17:45:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2024-01-11T17:45:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2024-01-11T17:45:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2024-01-11T17:45:19.935 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-01-11T17:45:19.935 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-01-11T17:45:19.935 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2024-01-11T17:45:19.936 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2024-01-11T17:45:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-01-11T17:45:20.656 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2024-01-11T17:45:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-01-11T17:45:21.595 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2024-01-11T17:45:21.596 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2024-01-11T17:45:26.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2024-01-11T17:45:26.255 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2024-01-11T17:45:26.286 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2024-01-11T17:45:26.286 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2024-01-11T17:45:26.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:26.852 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-01-11T17:45:26.852 INFO:teuthology.orchestra.run.smithi016.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: yajl-2.1.0-12.el8.x86_64 2024-01-11T17:45:26.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:45:26.854 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-01-11T17:45:26.898 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:26.898 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-01-11T17:45:26.898 INFO:teuthology.orchestra.run.smithi038.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.898 INFO:teuthology.orchestra.run.smithi038.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:26.899 INFO:teuthology.orchestra.run.smithi038.stdout: yajl-2.1.0-12.el8.x86_64 2024-01-11T17:45:26.900 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:45:26.900 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-01-11T17:45:27.041 DEBUG:teuthology.parallel:result is None 2024-01-11T17:45:27.125 DEBUG:teuthology.parallel:result is None 2024-01-11T17:45:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2024-01-11T17:45:27.470 INFO:teuthology.orchestra.run.smithi007.stdout:Installed products updated. 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-01-11T17:45:28.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-01-11T17:45:28.120 INFO:teuthology.orchestra.run.smithi007.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.120 INFO:teuthology.orchestra.run.smithi007.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2024-01-11T17:45:28.120 INFO:teuthology.orchestra.run.smithi007.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2024-01-11T17:45:28.120 INFO:teuthology.orchestra.run.smithi007.stdout: yajl-2.1.0-12.el8.x86_64 2024-01-11T17:45:28.120 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:45:28.120 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-11T17:45:28.375 DEBUG:teuthology.parallel:result is None 2024-01-11T17:45:28.375 INFO:teuthology.run_tasks:Running task install... 2024-01-11T17:45:28.384 DEBUG:teuthology.task.install:project ceph 2024-01-11T17:45:28.384 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117'}} 2024-01-11T17:45:28.384 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117'} 2024-01-11T17:45:28.384 INFO:teuthology.task.install:Using flavor: default 2024-01-11T17:45:28.390 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-11T17:45:28.390 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-11T17:45:28.390 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117', 'tag': None, 'wait_for_package': False} 2024-01-11T17:45:28.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:45:28.393 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117', 'tag': None, 'wait_for_package': False} 2024-01-11T17:45:28.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:45:28.394 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117', 'tag': None, 'wait_for_package': False} 2024-01-11T17:45:28.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:45:28.545 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/ 2024-01-11T17:45:28.545 INFO:teuthology.task.install.rpm:Package version is 19.0.0-446.gc27531ad 2024-01-11T17:45:28.549 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/ 2024-01-11T17:45:28.549 INFO:teuthology.task.install.rpm:Package version is 19.0.0-446.gc27531ad 2024-01-11T17:45:28.612 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/ 2024-01-11T17:45:28.613 INFO:teuthology.task.install.rpm:Package version is 19.0.0-446.gc27531ad 2024-01-11T17:45:28.660 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-11T17:45:28.661 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:45:28.661 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-11T17:45:28.697 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-11T17:45:28.697 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c27531add9155519f390b228e498f0a2c5eae117/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-11T17:45:28.722 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-11T17:45:28.722 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:45:28.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-11T17:45:28.726 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-11T17:45:28.726 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:45:28.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-11T17:45:28.762 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-11T17:45:28.762 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c27531add9155519f390b228e498f0a2c5eae117/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-11T17:45:28.773 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-11T17:45:28.792 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-11T17:45:28.792 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c27531add9155519f390b228e498f0a2c5eae117/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-11T17:45:28.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-11T17:45:28.862 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-11T17:45:28.894 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-01-11T17:45:28.896 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-01-11T17:45:28.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-11T17:45:28.926 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-11T17:45:28.957 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-01-11T17:45:28.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-01-11T17:45:28.999 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-11T17:45:29.068 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-01-11T17:45:29.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-01-11T17:45:29.331 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:29.393 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:29.557 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:29.852 INFO:teuthology.orchestra.run.smithi038.stdout:52 files removed 2024-01-11T17:45:29.871 INFO:teuthology.orchestra.run.smithi016.stdout:52 files removed 2024-01-11T17:45:29.914 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-11T17:45:29.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-11T17:45:30.111 INFO:teuthology.orchestra.run.smithi007.stdout:52 files removed 2024-01-11T17:45:30.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-11T17:45:30.298 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:30.316 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:30.653 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T17:45:31.292 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 296 kB/s | 83 kB 00:00 2024-01-11T17:45:31.324 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 224 kB/s | 83 kB 00:00 2024-01-11T17:45:31.595 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 282 kB/s | 83 kB 00:00 2024-01-11T17:45:31.693 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2024-01-11T17:45:31.787 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2024-01-11T17:45:31.921 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 33 kB/s | 9.5 kB 00:00 2024-01-11T17:45:32.094 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2024-01-11T17:45:32.230 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.3 kB/s | 1.5 kB 00:00 2024-01-11T17:45:32.279 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 3.2 kB/s | 1.5 kB 00:00 2024-01-11T17:45:32.285 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-01-11T17:45:32.406 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2024-01-11T17:45:32.505 INFO:teuthology.orchestra.run.smithi038.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-01-11T17:45:33.766 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-01-11T17:45:33.982 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-01-11T17:45:36.647 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 16 MB 00:04 2024-01-11T17:45:37.772 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 516 kB/s | 24 kB 00:00 2024-01-11T17:45:38.182 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 27 MB/s | 8.8 MB 00:00 2024-01-11T17:45:38.407 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 585 kB/s | 24 kB 00:00 2024-01-11T17:45:39.950 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 64 MB 00:01 2024-01-11T17:45:40.693 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 307 kB/s | 24 kB 00:00 2024-01-11T17:45:40.942 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 58 MB 00:01 2024-01-11T17:45:41.864 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 64 MB 00:01 2024-01-11T17:45:51.476 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 58 MB 00:01 2024-01-11T17:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 22 MB/s | 8.8 MB 00:00 2024-01-11T17:45:52.979 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 64 MB 00:01 2024-01-11T17:45:55.533 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 58 MB 00:01 2024-01-11T17:46:03.361 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 12 MB/s | 8.8 MB 00:00 2024-01-11T17:46:09.464 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T17:46:09.466 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T17:46:09.486 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-11T17:46:09.488 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-11T17:46:09.490 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-11T17:46:09.491 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-11T17:46:09.494 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-11T17:46:09.499 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-11T17:46:09.500 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-11T17:46:09.735 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout:=================================================================================================================================== 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout:=================================================================================================================================== 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.0.0-446.gc27531ad.el8 ceph 6.4 k 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.0.0-446.gc27531ad.el8 ceph 5.5 M 2024-01-11T17:46:09.752 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 945 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-446.gc27531ad.el8 ceph 175 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.0.0-446.gc27531ad.el8 ceph 1.6 M 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 133 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 1.8 M 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 7.4 M 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 50 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.0.0-446.gc27531ad.el8 ceph 14 M 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.0.0-446.gc27531ad.el8 ceph 53 M 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 262 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 418 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.0.0-446.gc27531ad.el8 ceph 263 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 18 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-11T17:46:09.753 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.0.0-446.gc27531ad.el8 ceph 31 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 863 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.0.0-446.gc27531ad.el8 ceph 139 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.0.0-446.gc27531ad.el8 ceph 197 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.0.0-446.gc27531ad.el8 ceph 370 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 365 k 2024-01-11T17:46:09.754 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.0.0-446.gc27531ad.el8 ceph 107 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 93 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.6 M 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 181 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.8 M 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.8 M 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-11T17:46:09.755 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.0.0-446.gc27531ad.el8 ceph 23 M 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 28 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.0.0-446.gc27531ad.el8 ceph 2.3 M 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 243 k 2024-01-11T17:46:09.756 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.0.0-446.gc27531ad.el8 ceph 4.9 M 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 19 M 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 15 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.0.0-446.gc27531ad.el8 ceph 24 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.0.0-446.gc27531ad.el8 ceph 183 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-11T17:46:09.757 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 545 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 6.9 M 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-11T17:46:09.758 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.3.4-12.el8 codeready-builder-for-rhel-8-x86_64-rpms 30 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-11T17:46:09.759 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 45 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.0.0-446.gc27531ad.el8 ceph 130 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-11T17:46:09.760 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-11T17:46:09.761 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-11T17:46:09.763 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-11T17:46:09.764 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling module streams: 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql 10 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout:=================================================================================================================================== 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout:Install 131 Packages 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 16 Packages 2024-01-11T17:46:09.765 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:46:09.766 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 223 M 2024-01-11T17:46:09.766 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-01-11T17:46:10.035 INFO:teuthology.orchestra.run.smithi038.stdout:(1/147): ceph-19.0.0-446.gc27531ad.el8.x86_64.r 33 kB/s | 6.4 kB 00:00 2024-01-11T17:46:10.502 INFO:teuthology.orchestra.run.smithi038.stdout:(2/147): ceph-fuse-19.0.0-446.gc27531ad.el8.x86 2.0 MB/s | 945 kB 00:00 2024-01-11T17:46:10.627 INFO:teuthology.orchestra.run.smithi038.stdout:(3/147): ceph-base-19.0.0-446.gc27531ad.el8.x86 7.0 MB/s | 5.5 MB 00:00 2024-01-11T17:46:10.676 INFO:teuthology.orchestra.run.smithi038.stdout:(4/147): ceph-immutable-object-cache-19.0.0-446 1.0 MB/s | 175 kB 00:00 2024-01-11T17:46:10.911 INFO:teuthology.orchestra.run.smithi038.stdout:(5/147): ceph-mds-19.0.0-446.gc27531ad.el8.x86_ 8.2 MB/s | 2.3 MB 00:00 2024-01-11T17:46:11.155 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T17:46:11.157 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T17:46:11.170 INFO:teuthology.orchestra.run.smithi038.stdout:(6/147): ceph-mgr-19.0.0-446.gc27531ad.el8.x86_ 3.3 MB/s | 1.6 MB 00:00 2024-01-11T17:46:11.177 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-11T17:46:11.178 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-11T17:46:11.180 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-11T17:46:11.181 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-11T17:46:11.184 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-11T17:46:11.191 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-11T17:46:11.191 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-11T17:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:=================================================================================================================================== 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:=================================================================================================================================== 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-446.gc27531ad.el8 ceph 6.4 k 2024-01-11T17:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-446.gc27531ad.el8 ceph 5.5 M 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 945 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-446.gc27531ad.el8 ceph 175 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-446.gc27531ad.el8 ceph 1.6 M 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 133 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 1.8 M 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 7.4 M 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 50 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-446.gc27531ad.el8 ceph 14 M 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-446.gc27531ad.el8 ceph 53 M 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 262 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 418 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-446.gc27531ad.el8 ceph 263 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 18 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-11T17:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-446.gc27531ad.el8 ceph 31 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 863 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-446.gc27531ad.el8 ceph 139 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-446.gc27531ad.el8 ceph 197 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-446.gc27531ad.el8 ceph 370 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 365 k 2024-01-11T17:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-446.gc27531ad.el8 ceph 107 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 93 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.6 M 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 181 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.8 M 2024-01-11T17:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.8 M 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-446.gc27531ad.el8 ceph 23 M 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 28 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-446.gc27531ad.el8 ceph 2.3 M 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 243 k 2024-01-11T17:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-446.gc27531ad.el8 ceph 4.9 M 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 19 M 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 15 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-446.gc27531ad.el8 ceph 24 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-446.gc27531ad.el8 ceph 183 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-01-11T17:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 545 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 6.9 M 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.3.4-12.el8 codeready-builder-for-rhel-8-x86_64-rpms 30 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-11T17:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 45 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-446.gc27531ad.el8 ceph 130 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-11T17:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T17:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-11T17:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-11T17:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling module streams: 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql 10 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout:=================================================================================================================================== 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout:Install 131 Packages 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 16 Packages 2024-01-11T17:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:46:11.457 INFO:teuthology.orchestra.run.smithi038.stdout:(7/147): ceph-mon-19.0.0-446.gc27531ad.el8.x86_ 9.1 MB/s | 4.9 MB 00:00 2024-01-11T17:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 223 M 2024-01-11T17:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-01-11T17:46:11.878 INFO:teuthology.orchestra.run.smithi007.stdout:(1/147): ceph-19.0.0-446.gc27531ad.el8.x86_64.r 17 kB/s | 6.4 kB 00:00 2024-01-11T17:46:12.220 INFO:teuthology.orchestra.run.smithi038.stdout:(8/147): ceph-common-19.0.0-446.gc27531ad.el8.x 9.6 MB/s | 23 MB 00:02 2024-01-11T17:46:12.354 INFO:teuthology.orchestra.run.smithi038.stdout:(9/147): ceph-selinux-19.0.0-446.gc27531ad.el8. 182 kB/s | 24 kB 00:00 2024-01-11T17:46:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T17:46:12.594 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T17:46:12.614 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-11T17:46:12.616 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-11T17:46:12.617 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-11T17:46:12.619 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-11T17:46:12.622 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-11T17:46:12.626 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-11T17:46:12.628 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-11T17:46:12.653 INFO:teuthology.orchestra.run.smithi007.stdout:(2/147): ceph-fuse-19.0.0-446.gc27531ad.el8.x86 1.2 MB/s | 945 kB 00:00 2024-01-11T17:46:12.862 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-01-11T17:46:12.878 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================================================== 2024-01-11T17:46:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-01-11T17:46:12.878 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================================================== 2024-01-11T17:46:12.878 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-01-11T17:46:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-446.gc27531ad.el8 ceph 6.4 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-446.gc27531ad.el8 ceph 5.5 M 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 945 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-446.gc27531ad.el8 ceph 175 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-446.gc27531ad.el8 ceph 1.6 M 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 133 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 1.8 M 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 7.4 M 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 50 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-446.gc27531ad.el8 ceph 14 M 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-446.gc27531ad.el8 ceph 53 M 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 262 k 2024-01-11T17:46:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 418 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-446.gc27531ad.el8 ceph 263 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 18 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-446.gc27531ad.el8 ceph 31 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 863 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-446.gc27531ad.el8 ceph 139 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-01-11T17:46:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-446.gc27531ad.el8 ceph 197 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-446.gc27531ad.el8 ceph 370 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 365 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-446.gc27531ad.el8 ceph 107 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 93 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.6 M 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 181 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-11T17:46:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.8 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 3.8 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-446.gc27531ad.el8 ceph 23 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 28 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-446.gc27531ad.el8 ceph 2.3 M 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 243 k 2024-01-11T17:46:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-446.gc27531ad.el8 ceph 4.9 M 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-446.gc27531ad.el8 ceph 19 M 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-446.gc27531ad.el8 ceph-noarch 15 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-446.gc27531ad.el8 ceph 24 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-446.gc27531ad.el8 ceph 183 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-11T17:46:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 545 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-446.gc27531ad.el8 ceph 6.9 M 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.3.4-12.el8 codeready-builder-for-rhel-8-x86_64-rpms 30 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-11T17:46:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-446.gc27531ad.el8 ceph 45 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-446.gc27531ad.el8 ceph 130 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-11T17:46:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-11T17:46:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-11T17:46:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling module streams: 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql 10 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:46:12.888 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-01-11T17:46:12.889 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================================================== 2024-01-11T17:46:12.889 INFO:teuthology.orchestra.run.smithi016.stdout:Install 131 Packages 2024-01-11T17:46:12.889 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 16 Packages 2024-01-11T17:46:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:46:12.889 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 223 M 2024-01-11T17:46:12.889 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-01-11T17:46:13.103 INFO:teuthology.orchestra.run.smithi007.stdout:(3/147): ceph-base-19.0.0-446.gc27531ad.el8.x86 3.4 MB/s | 5.5 MB 00:01 2024-01-11T17:46:13.137 INFO:teuthology.orchestra.run.smithi038.stdout:(10/147): ceph-radosgw-19.0.0-446.gc27531ad.el8 8.2 MB/s | 14 MB 00:01 2024-01-11T17:46:13.200 INFO:teuthology.orchestra.run.smithi016.stdout:(1/147): ceph-19.0.0-446.gc27531ad.el8.x86_64.r 24 kB/s | 6.4 kB 00:00 2024-01-11T17:46:13.252 INFO:teuthology.orchestra.run.smithi007.stdout:(4/147): ceph-immutable-object-cache-19.0.0-446 292 kB/s | 175 kB 00:00 2024-01-11T17:46:13.254 INFO:teuthology.orchestra.run.smithi038.stdout:(11/147): cephfs-mirror-19.0.0-446.gc27531ad.el 2.2 MB/s | 263 kB 00:00 2024-01-11T17:46:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:(12/147): libcephfs-devel-19.0.0-446.gc27531ad. 287 kB/s | 31 kB 00:00 2024-01-11T17:46:13.738 INFO:teuthology.orchestra.run.smithi007.stdout:(5/147): ceph-mgr-19.0.0-446.gc27531ad.el8.x86_ 3.4 MB/s | 1.6 MB 00:00 2024-01-11T17:46:13.813 INFO:teuthology.orchestra.run.smithi038.stdout:(13/147): libcephfs2-19.0.0-446.gc27531ad.el8.x 1.9 MB/s | 863 kB 00:00 2024-01-11T17:46:14.022 INFO:teuthology.orchestra.run.smithi038.stdout:(14/147): libcephsqlite-19.0.0-446.gc27531ad.el 878 kB/s | 183 kB 00:00 2024-01-11T17:46:14.156 INFO:teuthology.orchestra.run.smithi038.stdout:(15/147): librados-devel-19.0.0-446.gc27531ad.e 1.0 MB/s | 139 kB 00:00 2024-01-11T17:46:14.382 INFO:teuthology.orchestra.run.smithi016.stdout:(2/147): ceph-fuse-19.0.0-446.gc27531ad.el8.x86 800 kB/s | 945 kB 00:01 2024-01-11T17:46:14.481 INFO:teuthology.orchestra.run.smithi038.stdout:(16/147): libradosstriper1-19.0.0-446.gc27531ad 1.6 MB/s | 545 kB 00:00 2024-01-11T17:46:14.609 INFO:teuthology.orchestra.run.smithi016.stdout:(3/147): ceph-immutable-object-cache-19.0.0-446 768 kB/s | 175 kB 00:00 2024-01-11T17:46:14.687 INFO:teuthology.orchestra.run.smithi007.stdout:(6/147): ceph-mds-19.0.0-446.gc27531ad.el8.x86_ 1.5 MB/s | 2.3 MB 00:01 2024-01-11T17:46:14.981 INFO:teuthology.orchestra.run.smithi038.stdout:(17/147): ceph-osd-19.0.0-446.gc27531ad.el8.x86 4.9 MB/s | 19 MB 00:03 2024-01-11T17:46:14.996 INFO:teuthology.orchestra.run.smithi007.stdout:(7/147): ceph-common-19.0.0-446.gc27531ad.el8.x 6.6 MB/s | 23 MB 00:03 2024-01-11T17:46:15.065 INFO:teuthology.orchestra.run.smithi038.stdout:(18/147): python3-ceph-argparse-19.0.0-446.gc27 538 kB/s | 45 kB 00:00 2024-01-11T17:46:15.071 INFO:teuthology.orchestra.run.smithi007.stdout:(8/147): ceph-mon-19.0.0-446.gc27531ad.el8.x86_ 3.6 MB/s | 4.9 MB 00:01 2024-01-11T17:46:15.157 INFO:teuthology.orchestra.run.smithi038.stdout:(19/147): python3-ceph-common-19.0.0-446.gc2753 1.4 MB/s | 130 kB 00:00 2024-01-11T17:46:15.180 INFO:teuthology.orchestra.run.smithi007.stdout:(9/147): ceph-selinux-19.0.0-446.gc27531ad.el8. 224 kB/s | 24 kB 00:00 2024-01-11T17:46:15.265 INFO:teuthology.orchestra.run.smithi038.stdout:(20/147): python3-cephfs-19.0.0-446.gc27531ad.e 1.8 MB/s | 197 kB 00:00 2024-01-11T17:46:15.676 INFO:teuthology.orchestra.run.smithi016.stdout:(4/147): ceph-mds-19.0.0-446.gc27531ad.el8.x86_ 2.2 MB/s | 2.3 MB 00:01 2024-01-11T17:46:15.752 INFO:teuthology.orchestra.run.smithi016.stdout:(5/147): ceph-base-19.0.0-446.gc27531ad.el8.x86 1.9 MB/s | 5.5 MB 00:02 2024-01-11T17:46:16.149 INFO:teuthology.orchestra.run.smithi038.stdout:(21/147): python3-rados-19.0.0-446.gc27531ad.el 419 kB/s | 370 kB 00:00 2024-01-11T17:46:16.316 INFO:teuthology.orchestra.run.smithi038.stdout:(22/147): python3-rbd-19.0.0-446.gc27531ad.el8. 2.1 MB/s | 365 kB 00:00 2024-01-11T17:46:16.389 INFO:teuthology.orchestra.run.smithi007.stdout:(10/147): ceph-radosgw-19.0.0-446.gc27531ad.el8 10 MB/s | 14 MB 00:01 2024-01-11T17:46:16.399 INFO:teuthology.orchestra.run.smithi038.stdout:(23/147): python3-rgw-19.0.0-446.gc27531ad.el8. 1.3 MB/s | 107 kB 00:00 2024-01-11T17:46:16.464 INFO:teuthology.orchestra.run.smithi007.stdout:(11/147): cephfs-mirror-19.0.0-446.gc27531ad.el 3.4 MB/s | 263 kB 00:00 2024-01-11T17:46:16.477 INFO:teuthology.orchestra.run.smithi016.stdout:(6/147): ceph-mgr-19.0.0-446.gc27531ad.el8.x86_ 2.1 MB/s | 1.6 MB 00:00 2024-01-11T17:46:16.491 INFO:teuthology.orchestra.run.smithi038.stdout:(24/147): rbd-fuse-19.0.0-446.gc27531ad.el8.x86 1.0 MB/s | 93 kB 00:00 2024-01-11T17:46:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:(12/147): libcephfs-devel-19.0.0-446.gc27531ad. 465 kB/s | 31 kB 00:00 2024-01-11T17:46:16.665 INFO:teuthology.orchestra.run.smithi007.stdout:(13/147): libcephfs2-19.0.0-446.gc27531ad.el8.x 6.3 MB/s | 863 kB 00:00 2024-01-11T17:46:16.782 INFO:teuthology.orchestra.run.smithi007.stdout:(14/147): libcephsqlite-19.0.0-446.gc27531ad.el 1.5 MB/s | 183 kB 00:00 2024-01-11T17:46:16.873 INFO:teuthology.orchestra.run.smithi007.stdout:(15/147): librados-devel-19.0.0-446.gc27531ad.e 1.5 MB/s | 139 kB 00:00 2024-01-11T17:46:16.967 INFO:teuthology.orchestra.run.smithi038.stdout:(25/147): librgw2-19.0.0-446.gc27531ad.el8.x86_ 2.8 MB/s | 6.9 MB 00:02 2024-01-11T17:46:16.999 INFO:teuthology.orchestra.run.smithi007.stdout:(16/147): libradosstriper1-19.0.0-446.gc27531ad 4.3 MB/s | 545 kB 00:00 2024-01-11T17:46:17.208 INFO:teuthology.orchestra.run.smithi038.stdout:(26/147): rbd-nbd-19.0.0-446.gc27531ad.el8.x86_ 750 kB/s | 181 kB 00:00 2024-01-11T17:46:17.309 INFO:teuthology.orchestra.run.smithi038.stdout:(27/147): ceph-grafana-dashboards-19.0.0-446.gc 280 kB/s | 28 kB 00:00 2024-01-11T17:46:17.385 INFO:teuthology.orchestra.run.smithi016.stdout:(7/147): ceph-mon-19.0.0-446.gc27531ad.el8.x86_ 3.0 MB/s | 4.9 MB 00:01 2024-01-11T17:46:17.509 INFO:teuthology.orchestra.run.smithi038.stdout:(28/147): ceph-mgr-cephadm-19.0.0-446.gc27531ad 663 kB/s | 133 kB 00:00 2024-01-11T17:46:17.668 INFO:teuthology.orchestra.run.smithi038.stdout:(29/147): rbd-mirror-19.0.0-446.gc27531ad.el8.x 3.1 MB/s | 3.6 MB 00:01 2024-01-11T17:46:17.932 INFO:teuthology.orchestra.run.smithi007.stdout:(17/147): librgw2-19.0.0-446.gc27531ad.el8.x86_ 7.4 MB/s | 6.9 MB 00:00 2024-01-11T17:46:18.016 INFO:teuthology.orchestra.run.smithi007.stdout:(18/147): python3-ceph-argparse-19.0.0-446.gc27 539 kB/s | 45 kB 00:00 2024-01-11T17:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout:(19/147): python3-ceph-common-19.0.0-446.gc2753 1.4 MB/s | 130 kB 00:00 2024-01-11T17:46:18.185 INFO:teuthology.orchestra.run.smithi038.stdout:(30/147): ceph-mgr-dashboard-19.0.0-446.gc27531 2.6 MB/s | 1.8 MB 00:00 2024-01-11T17:46:18.200 INFO:teuthology.orchestra.run.smithi007.stdout:(20/147): python3-cephfs-19.0.0-446.gc27531ad.e 2.1 MB/s | 197 kB 00:00 2024-01-11T17:46:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:(21/147): python3-rados-19.0.0-446.gc27531ad.el 2.6 MB/s | 370 kB 00:00 2024-01-11T17:46:18.351 INFO:teuthology.orchestra.run.smithi038.stdout:(31/147): ceph-mgr-modules-core-19.0.0-446.gc27 1.4 MB/s | 243 kB 00:00 2024-01-11T17:46:18.435 INFO:teuthology.orchestra.run.smithi038.stdout:(32/147): ceph-mgr-rook-19.0.0-446.gc27531ad.el 602 kB/s | 50 kB 00:00 2024-01-11T17:46:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:(22/147): python3-rbd-19.0.0-446.gc27531ad.el8. 3.6 MB/s | 365 kB 00:00 2024-01-11T17:46:18.494 INFO:teuthology.orchestra.run.smithi038.stdout:(33/147): ceph-prometheus-alerts-19.0.0-446.gc2 257 kB/s | 15 kB 00:00 2024-01-11T17:46:18.526 INFO:teuthology.orchestra.run.smithi007.stdout:(23/147): python3-rgw-19.0.0-446.gc27531ad.el8. 1.3 MB/s | 107 kB 00:00 2024-01-11T17:46:18.609 INFO:teuthology.orchestra.run.smithi007.stdout:(24/147): rbd-fuse-19.0.0-446.gc27531ad.el8.x86 1.1 MB/s | 93 kB 00:00 2024-01-11T17:46:18.661 INFO:teuthology.orchestra.run.smithi038.stdout:(34/147): ceph-volume-19.0.0-446.gc27531ad.el8. 1.5 MB/s | 262 kB 00:00 2024-01-11T17:46:18.886 INFO:teuthology.orchestra.run.smithi038.stdout:(35/147): cephadm-19.0.0-446.gc27531ad.el8.noar 1.8 MB/s | 418 kB 00:00 2024-01-11T17:46:19.003 INFO:teuthology.orchestra.run.smithi038.stdout:(36/147): cephfs-top-19.0.0-446.gc27531ad.el8.n 155 kB/s | 18 kB 00:00 2024-01-11T17:46:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:(25/147): rbd-mirror-19.0.0-446.gc27531ad.el8.x 6.0 MB/s | 3.6 MB 00:00 2024-01-11T17:46:19.253 INFO:teuthology.orchestra.run.smithi038.stdout:(37/147): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-01-11T17:46:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:(26/147): rbd-nbd-19.0.0-446.gc27531ad.el8.x86_ 2.4 MB/s | 181 kB 00:00 2024-01-11T17:46:19.369 INFO:teuthology.orchestra.run.smithi007.stdout:(27/147): ceph-grafana-dashboards-19.0.0-446.gc 373 kB/s | 28 kB 00:00 2024-01-11T17:46:19.453 INFO:teuthology.orchestra.run.smithi007.stdout:(28/147): ceph-mgr-cephadm-19.0.0-446.gc27531ad 1.6 MB/s | 133 kB 00:00 2024-01-11T17:46:19.745 INFO:teuthology.orchestra.run.smithi038.stdout:(38/147): liboath-2.6.2-3.el8.x86_64.rpm 120 kB/s | 59 kB 00:00 2024-01-11T17:46:19.769 INFO:teuthology.orchestra.run.smithi007.stdout:(29/147): ceph-mgr-dashboard-19.0.0-446.gc27531 5.6 MB/s | 1.8 MB 00:00 2024-01-11T17:46:19.770 INFO:teuthology.orchestra.run.smithi038.stdout:(39/147): gperftools-libs-2.7-9.el8.x86_64.rpm 399 kB/s | 306 kB 00:00 2024-01-11T17:46:19.920 INFO:teuthology.orchestra.run.smithi038.stdout:(40/147): luarocks-3.5.0-1.el8.noarch.rpm 1.1 MB/s | 167 kB 00:00 2024-01-11T17:46:19.963 INFO:teuthology.orchestra.run.smithi038.stdout:(41/147): libunwind-1.3.1-3.el8.x86_64.rpm 345 kB/s | 75 kB 00:00 2024-01-11T17:46:20.105 INFO:teuthology.orchestra.run.smithi038.stdout:(42/147): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 308 kB/s | 44 kB 00:00 2024-01-11T17:46:20.147 INFO:teuthology.orchestra.run.smithi038.stdout:(43/147): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2024-01-11T17:46:20.264 INFO:teuthology.orchestra.run.smithi038.stdout:(44/147): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2024-01-11T17:46:20.389 INFO:teuthology.orchestra.run.smithi038.stdout:(45/147): python3-beautifulsoup4-4.6.3-2.el8.1. 653 kB/s | 185 kB 00:00 2024-01-11T17:46:20.460 INFO:teuthology.orchestra.run.smithi016.stdout:(8/147): ceph-radosgw-19.0.0-446.gc27531ad.el8. 4.5 MB/s | 14 MB 00:03 2024-01-11T17:46:20.510 INFO:teuthology.orchestra.run.smithi016.stdout:(9/147): ceph-selinux-19.0.0-446.gc27531ad.el8. 490 kB/s | 24 kB 00:00 2024-01-11T17:46:20.673 INFO:teuthology.orchestra.run.smithi038.stdout:(46/147): python3-certifi-2018.10.15-7.el8.noar 40 kB/s | 16 kB 00:00 2024-01-11T17:46:20.695 INFO:teuthology.orchestra.run.smithi007.stdout:(30/147): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-01-11T17:46:20.706 INFO:teuthology.orchestra.run.smithi038.stdout:(47/147): python3-cheroot-8.5.2-1.el8.noarch.rp 544 kB/s | 173 kB 00:00 2024-01-11T17:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout:(31/147): ceph-mgr-modules-core-19.0.0-446.gc27 2.9 MB/s | 243 kB 00:00 2024-01-11T17:46:20.806 INFO:teuthology.orchestra.run.smithi038.stdout:(48/147): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2024-01-11T17:46:20.837 INFO:teuthology.orchestra.run.smithi007.stdout:(32/147): ceph-mgr-rook-19.0.0-446.gc27531ad.el 861 kB/s | 50 kB 00:00 2024-01-11T17:46:20.856 INFO:teuthology.orchestra.run.smithi038.stdout:(49/147): python3-google-auth-1.1.1-10.el8.noar 541 kB/s | 81 kB 00:00 2024-01-11T17:46:20.887 INFO:teuthology.orchestra.run.smithi007.stdout:(33/147): ceph-prometheus-alerts-19.0.0-446.gc2 300 kB/s | 15 kB 00:00 2024-01-11T17:46:20.907 INFO:teuthology.orchestra.run.smithi038.stdout:(50/147): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-01-11T17:46:20.966 INFO:teuthology.orchestra.run.smithi038.stdout:(51/147): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-01-11T17:46:20.971 INFO:teuthology.orchestra.run.smithi007.stdout:(34/147): ceph-volume-19.0.0-446.gc27531ad.el8. 3.1 MB/s | 262 kB 00:00 2024-01-11T17:46:21.071 INFO:teuthology.orchestra.run.smithi007.stdout:(35/147): cephadm-19.0.0-446.gc27531ad.el8.noar 4.1 MB/s | 418 kB 00:00 2024-01-11T17:46:21.091 INFO:teuthology.orchestra.run.smithi038.stdout:(52/147): python3-logutils-0.3.5-11.el8.noarch. 390 kB/s | 49 kB 00:00 2024-01-11T17:46:21.133 INFO:teuthology.orchestra.run.smithi038.stdout:(53/147): python3-kubernetes-11.0.0-6.el8.noarc 5.2 MB/s | 1.2 MB 00:00 2024-01-11T17:46:21.147 INFO:teuthology.orchestra.run.smithi007.stdout:(36/147): cephfs-top-19.0.0-446.gc27531ad.el8.n 241 kB/s | 18 kB 00:00 2024-01-11T17:46:21.225 INFO:teuthology.orchestra.run.smithi038.stdout:(54/147): python3-more-itertools-7.2.0-3.el8.no 439 kB/s | 59 kB 00:00 2024-01-11T17:46:21.250 INFO:teuthology.orchestra.run.smithi038.stdout:(55/147): python3-natsort-7.1.1-2.el8.noarch.rp 514 kB/s | 60 kB 00:00 2024-01-11T17:46:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:(56/147): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2024-01-11T17:46:21.376 INFO:teuthology.orchestra.run.smithi038.stdout:(57/147): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2024-01-11T17:46:21.451 INFO:teuthology.orchestra.run.smithi038.stdout:(58/147): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2024-01-11T17:46:21.518 INFO:teuthology.orchestra.run.smithi038.stdout:(59/147): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2024-01-11T17:46:21.585 INFO:teuthology.orchestra.run.smithi038.stdout:(60/147): python3-rsa-4.9-2.el8.noarch.rpm 481 kB/s | 64 kB 00:00 2024-01-11T17:46:21.618 INFO:teuthology.orchestra.run.smithi038.stdout:(61/147): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2024-01-11T17:46:21.685 INFO:teuthology.orchestra.run.smithi038.stdout:(62/147): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2024-01-11T17:46:21.705 INFO:teuthology.orchestra.run.smithi007.stdout:(37/147): gperftools-libs-2.7-9.el8.x86_64.rpm 549 kB/s | 306 kB 00:00 2024-01-11T17:46:21.727 INFO:teuthology.orchestra.run.smithi038.stdout:(63/147): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2024-01-11T17:46:21.786 INFO:teuthology.orchestra.run.smithi038.stdout:(64/147): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2024-01-11T17:46:21.831 INFO:teuthology.orchestra.run.smithi007.stdout:(38/147): liboath-2.6.2-3.el8.x86_64.rpm 471 kB/s | 59 kB 00:00 2024-01-11T17:46:21.869 INFO:teuthology.orchestra.run.smithi038.stdout:(65/147): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2024-01-11T17:46:21.931 INFO:teuthology.orchestra.run.smithi038.stdout:(66/147): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2024-01-11T17:46:21.939 INFO:teuthology.orchestra.run.smithi007.stdout:(39/147): libunwind-1.3.1-3.el8.x86_64.rpm 696 kB/s | 75 kB 00:00 2024-01-11T17:46:21.987 INFO:teuthology.orchestra.run.smithi038.stdout:(67/147): python3-websocket-client-0.56.0-5.el8 523 kB/s | 61 kB 00:00 2024-01-11T17:46:22.029 INFO:teuthology.orchestra.run.smithi038.stdout:(68/147): python3-webtest-2.0.33-1.el8.noarch.r 871 kB/s | 85 kB 00:00 2024-01-11T17:46:22.088 INFO:teuthology.orchestra.run.smithi038.stdout:(69/147): python3-zc-lockfile-2.0-2.el8.noarch. 228 kB/s | 23 kB 00:00 2024-01-11T17:46:22.105 INFO:teuthology.orchestra.run.smithi038.stdout:(70/147): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-01-11T17:46:22.106 INFO:teuthology.orchestra.run.smithi007.stdout:(40/147): luarocks-3.5.0-1.el8.noarch.rpm 1.0 MB/s | 167 kB 00:00 2024-01-11T17:46:22.205 INFO:teuthology.orchestra.run.smithi038.stdout:(71/147): thrift-0.13.0-2.el8.x86_64.rpm 9.9 MB/s | 1.7 MB 00:00 2024-01-11T17:46:22.315 INFO:teuthology.orchestra.run.smithi007.stdout:(41/147): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2024-01-11T17:46:22.347 INFO:teuthology.orchestra.run.smithi038.stdout:(72/147): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2024-01-11T17:46:22.372 INFO:teuthology.orchestra.run.smithi038.stdout:(73/147): lua-devel-5.3.4-12.el8.x86_64.rpm 112 kB/s | 30 kB 00:00 2024-01-11T17:46:22.415 INFO:teuthology.orchestra.run.smithi007.stdout:(42/147): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 437 kB/s | 44 kB 00:00 2024-01-11T17:46:22.447 INFO:teuthology.orchestra.run.smithi038.stdout:(74/147): python3-prettytable-0.7.2-14.el8.noar 439 kB/s | 44 kB 00:00 2024-01-11T17:46:22.473 INFO:teuthology.orchestra.run.smithi038.stdout:(75/147): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2024-01-11T17:46:22.541 INFO:teuthology.orchestra.run.smithi007.stdout:(43/147): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2024-01-11T17:46:22.598 INFO:teuthology.orchestra.run.smithi038.stdout:(76/147): python3-jsonpatch-1.21-2.el8.noarch.r 176 kB/s | 27 kB 00:00 2024-01-11T17:46:22.641 INFO:teuthology.orchestra.run.smithi007.stdout:(44/147): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2024-01-11T17:46:22.682 INFO:teuthology.orchestra.run.smithi038.stdout:(77/147): python3-pyasn1-0.3.7-6.el8.noarch.rpm 605 kB/s | 126 kB 00:00 2024-01-11T17:46:22.707 INFO:teuthology.orchestra.run.smithi038.stdout:(78/147): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2024-01-11T17:46:22.741 INFO:teuthology.orchestra.run.smithi007.stdout:(45/147): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2024-01-11T17:46:22.757 INFO:teuthology.orchestra.run.smithi038.stdout:(79/147): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2024-01-11T17:46:22.791 INFO:teuthology.orchestra.run.smithi038.stdout:(80/147): flex-2.6.1-9.el8.x86_64.rpm 3.7 MB/s | 320 kB 00:00 2024-01-11T17:46:22.833 INFO:teuthology.orchestra.run.smithi038.stdout:(81/147): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2024-01-11T17:46:22.850 INFO:teuthology.orchestra.run.smithi007.stdout:(46/147): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2024-01-11T17:46:22.952 INFO:teuthology.orchestra.run.smithi016.stdout:(10/147): ceph-osd-19.0.0-446.gc27531ad.el8.x86 2.9 MB/s | 19 MB 00:06 2024-01-11T17:46:22.975 INFO:teuthology.orchestra.run.smithi007.stdout:(47/147): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2024-01-11T17:46:23.069 INFO:teuthology.orchestra.run.smithi016.stdout:(11/147): cephfs-mirror-19.0.0-446.gc27531ad.el 2.2 MB/s | 263 kB 00:00 2024-01-11T17:46:23.101 INFO:teuthology.orchestra.run.smithi007.stdout:(48/147): python3-google-auth-1.1.1-10.el8.noar 652 kB/s | 81 kB 00:00 2024-01-11T17:46:23.153 INFO:teuthology.orchestra.run.smithi016.stdout:(12/147): libcephfs-devel-19.0.0-446.gc27531ad. 374 kB/s | 31 kB 00:00 2024-01-11T17:46:23.201 INFO:teuthology.orchestra.run.smithi007.stdout:(49/147): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-01-11T17:46:23.365 INFO:teuthology.orchestra.run.smithi038.stdout:(82/147): ceph-test-19.0.0-446.gc27531ad.el8.x8 4.8 MB/s | 53 MB 00:11 2024-01-11T17:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout:(13/147): libcephfs2-19.0.0-446.gc27531ad.el8.x 3.7 MB/s | 863 kB 00:00 2024-01-11T17:46:23.400 INFO:teuthology.orchestra.run.smithi038.stdout:(83/147): python3-markupsafe-0.23-19.el8.x86_64 64 kB/s | 39 kB 00:00 2024-01-11T17:46:23.433 INFO:teuthology.orchestra.run.smithi038.stdout:(84/147): lttng-ust-2.8.1-11.el8.x86_64.rpm 432 kB/s | 259 kB 00:00 2024-01-11T17:46:23.470 INFO:teuthology.orchestra.run.smithi016.stdout:(14/147): libcephsqlite-19.0.0-446.gc27531ad.el 2.0 MB/s | 183 kB 00:00 2024-01-11T17:46:23.501 INFO:teuthology.orchestra.run.smithi038.stdout:(85/147): uuid-1.6.2-43.el8.x86_64.rpm 699 kB/s | 64 kB 00:00 2024-01-11T17:46:23.517 INFO:teuthology.orchestra.run.smithi038.stdout:(86/147): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.2 MB/s | 103 kB 00:00 2024-01-11T17:46:23.543 INFO:teuthology.orchestra.run.smithi038.stdout:(87/147): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-01-11T17:46:23.562 INFO:teuthology.orchestra.run.smithi016.stdout:(15/147): librados-devel-19.0.0-446.gc27531ad.e 1.5 MB/s | 139 kB 00:00 2024-01-11T17:46:23.593 INFO:teuthology.orchestra.run.smithi038.stdout:(88/147): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2024-01-11T17:46:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:(89/147): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2024-01-11T17:46:23.719 INFO:teuthology.orchestra.run.smithi007.stdout:(50/147): ceph-osd-19.0.0-446.gc27531ad.el8.x86 2.1 MB/s | 19 MB 00:09 2024-01-11T17:46:23.751 INFO:teuthology.orchestra.run.smithi007.stdout:(51/147): python3-jaraco-functools-2.0-4.el8.no 32 kB/s | 18 kB 00:00 2024-01-11T17:46:23.761 INFO:teuthology.orchestra.run.smithi016.stdout:(16/147): libradosstriper1-19.0.0-446.gc27531ad 2.7 MB/s | 545 kB 00:00 2024-01-11T17:46:23.768 INFO:teuthology.orchestra.run.smithi038.stdout:(90/147): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-01-11T17:46:23.794 INFO:teuthology.orchestra.run.smithi038.stdout:(91/147): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2024-01-11T17:46:23.853 INFO:teuthology.orchestra.run.smithi038.stdout:(92/147): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-01-11T17:46:23.860 INFO:teuthology.orchestra.run.smithi007.stdout:(52/147): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2024-01-11T17:46:23.886 INFO:teuthology.orchestra.run.smithi038.stdout:(93/147): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2024-01-11T17:46:23.911 INFO:teuthology.orchestra.run.smithi038.stdout:(94/147): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2024-01-11T17:46:23.960 INFO:teuthology.orchestra.run.smithi007.stdout:(53/147): python3-more-itertools-7.2.0-3.el8.no 589 kB/s | 59 kB 00:00 2024-01-11T17:46:24.012 INFO:teuthology.orchestra.run.smithi038.stdout:(95/147): automake-1.16.1-8.el8.noarch.rpm 7.0 MB/s | 714 kB 00:00 2024-01-11T17:46:24.021 INFO:teuthology.orchestra.run.smithi016.stdout:(17/147): ceph-common-19.0.0-446.gc27531ad.el8. 2.1 MB/s | 23 MB 00:11 2024-01-11T17:46:24.077 INFO:teuthology.orchestra.run.smithi007.stdout:(54/147): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2024-01-11T17:46:24.105 INFO:teuthology.orchestra.run.smithi016.stdout:(18/147): python3-ceph-argparse-19.0.0-446.gc27 538 kB/s | 45 kB 00:00 2024-01-11T17:46:24.107 INFO:teuthology.orchestra.run.smithi038.stdout:(96/147): openblas-threads-0.3.15-6.el8.x86_64. 17 MB/s | 5.2 MB 00:00 2024-01-11T17:46:24.137 INFO:teuthology.orchestra.run.smithi038.stdout:(97/147): libpq-13.11-1.el8.x86_64.rpm 1.5 MB/s | 198 kB 00:00 2024-01-11T17:46:24.171 INFO:teuthology.orchestra.run.smithi038.stdout:(98/147): jq-1.6-7.el8.x86_64.rpm 3.1 MB/s | 202 kB 00:00 2024-01-11T17:46:24.194 INFO:teuthology.orchestra.run.smithi007.stdout:(55/147): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2024-01-11T17:46:24.196 INFO:teuthology.orchestra.run.smithi038.stdout:(99/147): python3-pytz-2017.2-11.el8.noarch.rpm 922 kB/s | 54 kB 00:00 2024-01-11T17:46:24.205 INFO:teuthology.orchestra.run.smithi016.stdout:(19/147): python3-ceph-common-19.0.0-446.gc2753 1.3 MB/s | 130 kB 00:00 2024-01-11T17:46:24.272 INFO:teuthology.orchestra.run.smithi038.stdout:(100/147): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-01-11T17:46:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:(56/147): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2024-01-11T17:46:24.313 INFO:teuthology.orchestra.run.smithi038.stdout:(101/147): postgresql-contrib-10.23-3.module+el 6.8 MB/s | 811 kB 00:00 2024-01-11T17:46:24.322 INFO:teuthology.orchestra.run.smithi016.stdout:(20/147): python3-cephfs-19.0.0-446.gc27531ad.e 1.6 MB/s | 197 kB 00:00 2024-01-11T17:46:24.355 INFO:teuthology.orchestra.run.smithi038.stdout:(102/147): postgresql-10.23-3.module+el8.9.0+20 8.2 MB/s | 1.5 MB 00:00 2024-01-11T17:46:24.395 INFO:teuthology.orchestra.run.smithi007.stdout:(57/147): python3-repoze-lru-0.7-6.el8.noarch.r 337 kB/s | 34 kB 00:00 2024-01-11T17:46:24.428 INFO:teuthology.orchestra.run.smithi007.stdout:(58/147): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2024-01-11T17:46:24.431 INFO:teuthology.orchestra.run.smithi038.stdout:(103/147): libverto-devel-0.3.0-5.el8.x86_64.rp 239 kB/s | 18 kB 00:00 2024-01-11T17:46:24.472 INFO:teuthology.orchestra.run.smithi016.stdout:(21/147): python3-rados-19.0.0-446.gc27531ad.el 2.4 MB/s | 370 kB 00:00 2024-01-11T17:46:24.504 INFO:teuthology.orchestra.run.smithi007.stdout:(59/147): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-01-11T17:46:24.529 INFO:teuthology.orchestra.run.smithi007.stdout:(60/147): python3-rsa-4.9-2.el8.noarch.rpm 640 kB/s | 64 kB 00:00 2024-01-11T17:46:24.531 INFO:teuthology.orchestra.run.smithi038.stdout:(104/147): postgresql-server-10.23-3.module+el8 20 MB/s | 5.1 MB 00:00 2024-01-11T17:46:24.548 INFO:teuthology.orchestra.run.smithi038.stdout:(105/147): python3-jwt-1.6.1-2.el8.noarch.rpm 369 kB/s | 43 kB 00:00 2024-01-11T17:46:24.596 INFO:teuthology.orchestra.run.smithi007.stdout:(61/147): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2024-01-11T17:46:24.630 INFO:teuthology.orchestra.run.smithi007.stdout:(62/147): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2024-01-11T17:46:24.705 INFO:teuthology.orchestra.run.smithi007.stdout:(63/147): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2024-01-11T17:46:24.731 INFO:teuthology.orchestra.run.smithi007.stdout:(64/147): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2024-01-11T17:46:24.790 INFO:teuthology.orchestra.run.smithi038.stdout:(106/147): python3-scipy-1.0.0-21.module+el8.9. 29 MB/s | 14 MB 00:00 2024-01-11T17:46:24.814 INFO:teuthology.orchestra.run.smithi007.stdout:(65/147): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2024-01-11T17:46:24.840 INFO:teuthology.orchestra.run.smithi007.stdout:(66/147): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2024-01-11T17:46:24.840 INFO:teuthology.orchestra.run.smithi038.stdout:(107/147): python3-pycparser-2.14-14.el8.noarch 373 kB/s | 109 kB 00:00 2024-01-11T17:46:24.856 INFO:teuthology.orchestra.run.smithi016.stdout:(22/147): python3-rbd-19.0.0-446.gc27531ad.el8. 953 kB/s | 365 kB 00:00 2024-01-11T17:46:24.874 INFO:teuthology.orchestra.run.smithi038.stdout:(108/147): python3-oauthlib-2.1.0-1.el8.noarch. 452 kB/s | 155 kB 00:00 2024-01-11T17:46:24.915 INFO:teuthology.orchestra.run.smithi007.stdout:(67/147): python3-websocket-client-0.56.0-5.el8 611 kB/s | 61 kB 00:00 2024-01-11T17:46:24.940 INFO:teuthology.orchestra.run.smithi007.stdout:(68/147): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2024-01-11T17:46:24.964 INFO:teuthology.orchestra.run.smithi016.stdout:(23/147): python3-rgw-19.0.0-446.gc27531ad.el8. 988 kB/s | 107 kB 00:00 2024-01-11T17:46:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:(109/147): python3-requests-oauthlib-1.0.0-1.el 197 kB/s | 43 kB 00:00 2024-01-11T17:46:25.016 INFO:teuthology.orchestra.run.smithi007.stdout:(69/147): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2024-01-11T17:46:25.023 INFO:teuthology.orchestra.run.smithi016.stdout:(24/147): rbd-fuse-19.0.0-446.gc27531ad.el8.x86 1.6 MB/s | 93 kB 00:00 2024-01-11T17:46:25.041 INFO:teuthology.orchestra.run.smithi007.stdout:(70/147): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2024-01-11T17:46:25.166 INFO:teuthology.orchestra.run.smithi007.stdout:(71/147): thrift-0.13.0-2.el8.x86_64.rpm 7.7 MB/s | 1.7 MB 00:00 2024-01-11T17:46:25.241 INFO:teuthology.orchestra.run.smithi038.stdout:(110/147): pcre2-devel-10.32-2.el8.x86_64.rpm 2.5 MB/s | 605 kB 00:00 2024-01-11T17:46:25.266 INFO:teuthology.orchestra.run.smithi038.stdout:(111/147): libaio-devel-0.3.112-1.el8.x86_64.rp 44 kB/s | 19 kB 00:00 2024-01-11T17:46:25.325 INFO:teuthology.orchestra.run.smithi038.stdout:(112/147): libzstd-devel-1.4.4-1.el8.x86_64.rpm 97 kB/s | 44 kB 00:00 2024-01-11T17:46:25.325 INFO:teuthology.orchestra.run.smithi007.stdout:(72/147): libverto-devel-0.3.0-5.el8.x86_64.rpm 63 kB/s | 18 kB 00:00 2024-01-11T17:46:25.340 INFO:teuthology.orchestra.run.smithi016.stdout:(25/147): librgw2-19.0.0-446.gc27531ad.el8.x86_ 4.4 MB/s | 6.9 MB 00:01 2024-01-11T17:46:25.350 INFO:teuthology.orchestra.run.smithi007.stdout:(73/147): python3-jwt-1.6.1-2.el8.noarch.rpm 232 kB/s | 43 kB 00:00 2024-01-11T17:46:25.417 INFO:teuthology.orchestra.run.smithi007.stdout:(74/147): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2024-01-11T17:46:25.424 INFO:teuthology.orchestra.run.smithi016.stdout:(26/147): rbd-nbd-19.0.0-446.gc27531ad.el8.x86_ 2.1 MB/s | 181 kB 00:00 2024-01-11T17:46:25.425 INFO:teuthology.orchestra.run.smithi038.stdout:(113/147): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-01-11T17:46:25.442 INFO:teuthology.orchestra.run.smithi007.stdout:(75/147): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2024-01-11T17:46:25.450 INFO:teuthology.orchestra.run.smithi038.stdout:(114/147): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2024-01-11T17:46:25.476 INFO:teuthology.orchestra.run.smithi038.stdout:(115/147): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2024-01-11T17:46:25.499 INFO:teuthology.orchestra.run.smithi016.stdout:(27/147): ceph-grafana-dashboards-19.0.0-446.gc 375 kB/s | 28 kB 00:00 2024-01-11T17:46:25.551 INFO:teuthology.orchestra.run.smithi007.stdout:(76/147): python3-requests-oauthlib-1.0.0-1.el8 320 kB/s | 43 kB 00:00 2024-01-11T17:46:25.566 INFO:teuthology.orchestra.run.smithi016.stdout:(28/147): ceph-mgr-cephadm-19.0.0-446.gc27531ad 1.9 MB/s | 133 kB 00:00 2024-01-11T17:46:25.576 INFO:teuthology.orchestra.run.smithi007.stdout:(77/147): libaio-devel-0.3.112-1.el8.x86_64.rpm 140 kB/s | 19 kB 00:00 2024-01-11T17:46:25.617 INFO:teuthology.orchestra.run.smithi038.stdout:(116/147): keyutils-libs-devel-1.5.10-9.el8.x86 250 kB/s | 48 kB 00:00 2024-01-11T17:46:25.642 INFO:teuthology.orchestra.run.smithi038.stdout:(117/147): libsepol-devel-2.9-3.el8.x86_64.rpm 453 kB/s | 87 kB 00:00 2024-01-11T17:46:25.735 INFO:teuthology.orchestra.run.smithi038.stdout:(118/147): zlib-devel-1.2.11-18.el8_5.x86_64.rp 223 kB/s | 58 kB 00:00 2024-01-11T17:46:25.868 INFO:teuthology.orchestra.run.smithi038.stdout:(119/147): libcom_err-devel-1.45.6-4.el8.x86_64 154 kB/s | 39 kB 00:00 2024-01-11T17:46:25.892 INFO:teuthology.orchestra.run.smithi016.stdout:(29/147): rbd-mirror-19.0.0-446.gc27531ad.el8.x 4.2 MB/s | 3.6 MB 00:00 2024-01-11T17:46:25.919 INFO:teuthology.orchestra.run.smithi038.stdout:(120/147): fuse-2.9.7-17.el8.x86_64.rpm 303 kB/s | 83 kB 00:00 2024-01-11T17:46:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:(30/147): ceph-mgr-dashboard-19.0.0-446.gc27531 4.8 MB/s | 1.8 MB 00:00 2024-01-11T17:46:26.036 INFO:teuthology.orchestra.run.smithi038.stdout:(121/147): libgfortran-8.5.0-20.el8.x86_64.rpm 2.1 MB/s | 645 kB 00:00 2024-01-11T17:46:26.042 INFO:teuthology.orchestra.run.smithi016.stdout:(31/147): ceph-mgr-modules-core-19.0.0-446.gc27 2.2 MB/s | 243 kB 00:00 2024-01-11T17:46:26.069 INFO:teuthology.orchestra.run.smithi038.stdout:(122/147): libquadmath-8.5.0-20.el8.x86_64.rpm 859 kB/s | 172 kB 00:00 2024-01-11T17:46:26.077 INFO:teuthology.orchestra.run.smithi007.stdout:(78/147): ceph-test-19.0.0-446.gc27531ad.el8.x8 4.9 MB/s | 53 MB 00:10 2024-01-11T17:46:26.102 INFO:teuthology.orchestra.run.smithi007.stdout:(79/147): libzstd-devel-1.4.4-1.el8.x86_64.rpm 80 kB/s | 44 kB 00:00 2024-01-11T17:46:26.109 INFO:teuthology.orchestra.run.smithi016.stdout:(32/147): ceph-mgr-rook-19.0.0-446.gc27531ad.el 753 kB/s | 50 kB 00:00 2024-01-11T17:46:26.144 INFO:teuthology.orchestra.run.smithi007.stdout:(80/147): pcre2-devel-10.32-2.el8.x86_64.rpm 1.0 MB/s | 605 kB 00:00 2024-01-11T17:46:26.176 INFO:teuthology.orchestra.run.smithi016.stdout:(33/147): ceph-prometheus-alerts-19.0.0-446.gc2 225 kB/s | 15 kB 00:00 2024-01-11T17:46:26.245 INFO:teuthology.orchestra.run.smithi007.stdout:(81/147): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-01-11T17:46:26.270 INFO:teuthology.orchestra.run.smithi007.stdout:(82/147): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-01-11T17:46:26.276 INFO:teuthology.orchestra.run.smithi016.stdout:(34/147): ceph-volume-19.0.0-446.gc27531ad.el8. 2.6 MB/s | 262 kB 00:00 2024-01-11T17:46:26.303 INFO:teuthology.orchestra.run.smithi007.stdout:(83/147): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2024-01-11T17:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout:(84/147): keyutils-libs-devel-1.5.10-9.el8.x86_ 523 kB/s | 48 kB 00:00 2024-01-11T17:46:26.337 INFO:teuthology.orchestra.run.smithi038.stdout:(123/147): libuuid-devel-2.32.1-43.el8.x86_64.r 237 kB/s | 100 kB 00:00 2024-01-11T17:46:26.370 INFO:teuthology.orchestra.run.smithi038.stdout:(124/147): python3-cffi-1.11.5-6.el8.x86_64.rpm 714 kB/s | 238 kB 00:00 2024-01-11T17:46:26.401 INFO:teuthology.orchestra.run.smithi016.stdout:(35/147): cephadm-19.0.0-446.gc27531ad.el8.noar 3.3 MB/s | 418 kB 00:00 2024-01-11T17:46:26.403 INFO:teuthology.orchestra.run.smithi038.stdout:(125/147): python3-cryptography-3.2.1-6.el8.x86 1.6 MB/s | 559 kB 00:00 2024-01-11T17:46:26.429 INFO:teuthology.orchestra.run.smithi007.stdout:(85/147): libsepol-devel-2.9-3.el8.x86_64.rpm 547 kB/s | 87 kB 00:00 2024-01-11T17:46:26.468 INFO:teuthology.orchestra.run.smithi016.stdout:(36/147): cephfs-top-19.0.0-446.gc27531ad.el8.n 272 kB/s | 18 kB 00:00 2024-01-11T17:46:26.471 INFO:teuthology.orchestra.run.smithi007.stdout:(86/147): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 344 kB/s | 58 kB 00:00 2024-01-11T17:46:26.488 INFO:teuthology.orchestra.run.smithi007.stdout:(87/147): libcom_err-devel-1.45.6-4.el8.x86_64. 256 kB/s | 39 kB 00:00 2024-01-11T17:46:26.597 INFO:teuthology.orchestra.run.smithi007.stdout:(88/147): libgfortran-8.5.0-20.el8.x86_64.rpm 5.0 MB/s | 645 kB 00:00 2024-01-11T17:46:26.622 INFO:teuthology.orchestra.run.smithi007.stdout:(89/147): fuse-2.9.7-17.el8.x86_64.rpm 431 kB/s | 83 kB 00:00 2024-01-11T17:46:26.681 INFO:teuthology.orchestra.run.smithi007.stdout:(90/147): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 100 kB 00:00 2024-01-11T17:46:26.737 INFO:teuthology.orchestra.run.smithi038.stdout:(126/147): xfsprogs-devel-5.0.0-12.el8.x86_64.r 182 kB/s | 73 kB 00:00 2024-01-11T17:46:26.739 INFO:teuthology.orchestra.run.smithi007.stdout:(91/147): libquadmath-8.5.0-20.el8.x86_64.rpm 686 kB/s | 172 kB 00:00 2024-01-11T17:46:26.814 INFO:teuthology.orchestra.run.smithi007.stdout:(92/147): python3-cryptography-3.2.1-6.el8.x86_ 4.1 MB/s | 559 kB 00:00 2024-01-11T17:46:26.915 INFO:teuthology.orchestra.run.smithi007.stdout:(93/147): python3-cffi-1.11.5-6.el8.x86_64.rpm 813 kB/s | 238 kB 00:00 2024-01-11T17:46:26.929 INFO:teuthology.orchestra.run.smithi038.stdout:(127/147): NetworkManager-initscripts-updown-1. 273 kB/s | 143 kB 00:00 2024-01-11T17:46:26.940 INFO:teuthology.orchestra.run.smithi007.stdout:(94/147): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 362 kB/s | 73 kB 00:00 2024-01-11T17:46:26.971 INFO:teuthology.orchestra.run.smithi038.stdout:(128/147): elfutils-libelf-devel-0.189-3.el8.x8 109 kB/s | 62 kB 00:00 2024-01-11T17:46:27.090 INFO:teuthology.orchestra.run.smithi007.stdout:(95/147): NetworkManager-initscripts-updown-1.4 521 kB/s | 143 kB 00:00 2024-01-11T17:46:27.177 INFO:teuthology.orchestra.run.smithi016.stdout:(37/147): ceph-test-19.0.0-446.gc27531ad.el8.x8 7.9 MB/s | 53 MB 00:06 2024-01-11T17:46:27.202 INFO:teuthology.orchestra.run.smithi016.stdout:(38/147): gperftools-libs-2.7-9.el8.x86_64.rpm 417 kB/s | 306 kB 00:00 2024-01-11T17:46:27.299 INFO:teuthology.orchestra.run.smithi016.stdout:(39/147): libunwind-1.3.1-3.el8.x86_64.rpm 778 kB/s | 75 kB 00:00 2024-01-11T17:46:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:(96/147): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.2 MB/s | 562 kB 00:00 2024-01-11T17:46:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:(40/147): liboath-2.6.2-3.el8.x86_64.rpm 352 kB/s | 59 kB 00:00 2024-01-11T17:46:27.370 INFO:teuthology.orchestra.run.smithi016.stdout:(41/147): luarocks-3.5.0-1.el8.noarch.rpm 2.3 MB/s | 167 kB 00:00 2024-01-11T17:46:27.374 INFO:teuthology.orchestra.run.smithi007.stdout:(97/147): elfutils-libelf-devel-0.189-3.el8.x86 134 kB/s | 62 kB 00:00 2024-01-11T17:46:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:(42/147): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 870 kB/s | 44 kB 00:00 2024-01-11T17:46:27.446 INFO:teuthology.orchestra.run.smithi038.stdout:(129/147): krb5-devel-1.18.2-26.el8_9.x86_64.rp 1.1 MB/s | 562 kB 00:00 2024-01-11T17:46:27.496 INFO:teuthology.orchestra.run.smithi016.stdout:(43/147): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2024-01-11T17:46:27.513 INFO:teuthology.orchestra.run.smithi038.stdout:(130/147): librabbitmq-0.9.0-4.el8.x86_64.rpm 61 kB/s | 47 kB 00:00 2024-01-11T17:46:27.516 INFO:teuthology.orchestra.run.smithi007.stdout:(98/147): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2024-01-11T17:46:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:(44/147): python3-asyncssh-2.7.0-2.el8.noarch.r 2.6 MB/s | 476 kB 00:00 2024-01-11T17:46:27.542 INFO:teuthology.orchestra.run.smithi007.stdout:(99/147): librabbitmq-0.9.0-4.el8.x86_64.rpm 78 kB/s | 47 kB 00:00 2024-01-11T17:46:27.546 INFO:teuthology.orchestra.run.smithi016.stdout:(45/147): python3-cachetools-3.1.1-4.el8.noarch 658 kB/s | 33 kB 00:00 2024-01-11T17:46:27.630 INFO:teuthology.orchestra.run.smithi038.stdout:(131/147): openssl-devel-1.1.1k-12.el8_9.x86_64 3.5 MB/s | 2.3 MB 00:00 2024-01-11T17:46:27.634 INFO:teuthology.orchestra.run.smithi007.stdout:(100/147): python3-pyasn1-modules-0.3.7-6.el8.n 1.2 MB/s | 110 kB 00:00 2024-01-11T17:46:27.655 INFO:teuthology.orchestra.run.smithi016.stdout:(46/147): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-01-11T17:46:27.659 INFO:teuthology.orchestra.run.smithi007.stdout:(101/147): python3-prettytable-0.7.2-14.el8.noa 309 kB/s | 44 kB 00:00 2024-01-11T17:46:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:(47/147): python3-certifi-2018.10.15-7.el8.noar 108 kB/s | 16 kB 00:00 2024-01-11T17:46:27.697 INFO:teuthology.orchestra.run.smithi016.stdout:(48/147): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2024-01-11T17:46:27.701 INFO:teuthology.orchestra.run.smithi007.stdout:(102/147): python3-jsonpatch-1.21-2.el8.noarch. 398 kB/s | 27 kB 00:00 2024-01-11T17:46:27.722 INFO:teuthology.orchestra.run.smithi038.stdout:(132/147): elfutils-libelf-0.189-3.el8.x86_64.r 2.5 MB/s | 232 kB 00:00 2024-01-11T17:46:27.731 INFO:teuthology.orchestra.run.smithi016.stdout:(49/147): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2024-01-11T17:46:27.756 INFO:teuthology.orchestra.run.smithi016.stdout:(50/147): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2024-01-11T17:46:27.776 INFO:teuthology.orchestra.run.smithi007.stdout:(103/147): openssl-devel-1.1.1k-12.el8_9.x86_64 5.3 MB/s | 2.3 MB 00:00 2024-01-11T17:46:27.790 INFO:teuthology.orchestra.run.smithi016.stdout:(51/147): python3-jaraco-functools-2.0-4.el8.no 302 kB/s | 18 kB 00:00 2024-01-11T17:46:27.802 INFO:teuthology.orchestra.run.smithi007.stdout:(104/147): python3-pyasn1-0.3.7-6.el8.noarch.rp 887 kB/s | 126 kB 00:00 2024-01-11T17:46:27.835 INFO:teuthology.orchestra.run.smithi007.stdout:(105/147): python3-werkzeug-0.12.2-4.el8.noarch 3.3 MB/s | 457 kB 00:00 2024-01-11T17:46:27.848 INFO:teuthology.orchestra.run.smithi016.stdout:(52/147): python3-logutils-0.3.5-11.el8.noarch. 834 kB/s | 49 kB 00:00 2024-01-11T17:46:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:(106/147): bison-3.0.4-10.el8.x86_64.rpm 6.7 MB/s | 688 kB 00:00 2024-01-11T17:46:27.891 INFO:teuthology.orchestra.run.smithi016.stdout:(53/147): python3-kubernetes-11.0.0-6.el8.noarc 8.7 MB/s | 1.2 MB 00:00 2024-01-11T17:46:27.916 INFO:teuthology.orchestra.run.smithi016.stdout:(54/147): python3-more-itertools-7.2.0-3.el8.no 880 kB/s | 59 kB 00:00 2024-01-11T17:46:27.952 INFO:teuthology.orchestra.run.smithi007.stdout:(107/147): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-01-11T17:46:27.972 INFO:teuthology.orchestra.run.smithi038.stdout:(133/147): elfutils-libs-0.189-3.el8.x86_64.rpm 1.2 MB/s | 303 kB 00:00 2024-01-11T17:46:27.974 INFO:teuthology.orchestra.run.smithi016.stdout:(55/147): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2024-01-11T17:46:28.008 INFO:teuthology.orchestra.run.smithi016.stdout:(56/147): python3-natsort-7.1.1-2.el8.noarch.rp 513 kB/s | 60 kB 00:00 2024-01-11T17:46:28.011 INFO:teuthology.orchestra.run.smithi007.stdout:(108/147): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-01-11T17:46:28.034 INFO:teuthology.orchestra.run.smithi016.stdout:(57/147): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2024-01-11T17:46:28.036 INFO:teuthology.orchestra.run.smithi007.stdout:(109/147): python3-markupsafe-0.23-19.el8.x86_6 244 kB/s | 39 kB 00:00 2024-01-11T17:46:28.059 INFO:teuthology.orchestra.run.smithi016.stdout:(58/147): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2024-01-11T17:46:28.084 INFO:teuthology.orchestra.run.smithi016.stdout:(59/147): python3-repoze-lru-0.7-6.el8.noarch.r 445 kB/s | 34 kB 00:00 2024-01-11T17:46:28.087 INFO:teuthology.orchestra.run.smithi007.stdout:(110/147): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2024-01-11T17:46:28.109 INFO:teuthology.orchestra.run.smithi016.stdout:(60/147): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2024-01-11T17:46:28.134 INFO:teuthology.orchestra.run.smithi016.stdout:(61/147): python3-rsa-4.9-2.el8.noarch.rpm 849 kB/s | 64 kB 00:00 2024-01-11T17:46:28.160 INFO:teuthology.orchestra.run.smithi016.stdout:(62/147): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2024-01-11T17:46:28.167 INFO:teuthology.orchestra.run.smithi038.stdout:(134/147): librados2-19.0.0-446.gc27531ad.el8.x 5.3 MB/s | 3.8 MB 00:00 2024-01-11T17:46:28.178 INFO:teuthology.orchestra.run.smithi007.stdout:(111/147): uuid-1.6.2-43.el8.x86_64.rpm 451 kB/s | 64 kB 00:00 2024-01-11T17:46:28.185 INFO:teuthology.orchestra.run.smithi016.stdout:(63/147): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2024-01-11T17:46:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:(64/147): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2024-01-11T17:46:28.236 INFO:teuthology.orchestra.run.smithi016.stdout:(65/147): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2024-01-11T17:46:28.254 INFO:teuthology.orchestra.run.smithi007.stdout:(112/147): oniguruma-6.8.2-2.el8.x86_64.rpm 773 kB/s | 187 kB 00:00 2024-01-11T17:46:28.281 INFO:teuthology.orchestra.run.smithi038.stdout:(135/147): libmount-2.32.1-43.el8.x86_64.rpm 767 kB/s | 237 kB 00:00 2024-01-11T17:46:28.287 INFO:teuthology.orchestra.run.smithi007.stdout:(113/147): python3-pyOpenSSL-19.0.0-1.el8.noarc 516 kB/s | 103 kB 00:00 2024-01-11T17:46:28.303 INFO:teuthology.orchestra.run.smithi016.stdout:(66/147): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2024-01-11T17:46:28.328 INFO:teuthology.orchestra.run.smithi016.stdout:(67/147): python3-websocket-client-0.56.0-5.el8 666 kB/s | 61 kB 00:00 2024-01-11T17:46:28.361 INFO:teuthology.orchestra.run.smithi016.stdout:(68/147): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2024-01-11T17:46:28.387 INFO:teuthology.orchestra.run.smithi016.stdout:(69/147): python3-zc-lockfile-2.0-2.el8.noarch. 393 kB/s | 23 kB 00:00 2024-01-11T17:46:28.405 INFO:teuthology.orchestra.run.smithi007.stdout:(114/147): python3-jinja2-2.10.1-3.el8.noarch.r 3.5 MB/s | 538 kB 00:00 2024-01-11T17:46:28.412 INFO:teuthology.orchestra.run.smithi016.stdout:(70/147): python3-webtest-2.0.33-1.el8.noarch.r 783 kB/s | 85 kB 00:00 2024-01-11T17:46:28.431 INFO:teuthology.orchestra.run.smithi038.stdout:(136/147): libsmartcols-2.32.1-43.el8.x86_64.rp 673 kB/s | 179 kB 00:00 2024-01-11T17:46:28.437 INFO:teuthology.orchestra.run.smithi016.stdout:(71/147): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-01-11T17:46:28.529 INFO:teuthology.orchestra.run.smithi016.stdout:(72/147): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2024-01-11T17:46:28.532 INFO:teuthology.orchestra.run.smithi038.stdout:(137/147): libuuid-2.32.1-43.el8.x86_64.rpm 395 kB/s | 99 kB 00:00 2024-01-11T17:46:28.547 INFO:teuthology.orchestra.run.smithi007.stdout:(115/147): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2024-01-11T17:46:28.590 INFO:teuthology.orchestra.run.smithi038.stdout:(138/147): librbd1-19.0.0-446.gc27531ad.el8.x86 3.5 MB/s | 3.8 MB 00:01 2024-01-11T17:46:28.622 INFO:teuthology.orchestra.run.smithi007.stdout:(116/147): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-01-11T17:46:28.646 INFO:teuthology.orchestra.run.smithi016.stdout:(73/147): libverto-devel-0.3.0-5.el8.x86_64.rpm 76 kB/s | 18 kB 00:00 2024-01-11T17:46:28.648 INFO:teuthology.orchestra.run.smithi007.stdout:(117/147): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-01-11T17:46:28.689 INFO:teuthology.orchestra.run.smithi007.stdout:(118/147): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2024-01-11T17:46:28.707 INFO:teuthology.orchestra.run.smithi038.stdout:(139/147): util-linux-2.32.1-43.el8.x86_64.rpm 9.0 MB/s | 2.5 MB 00:00 2024-01-11T17:46:28.721 INFO:teuthology.orchestra.run.smithi016.stdout:(74/147): python3-oauthlib-2.1.0-1.el8.noarch.r 807 kB/s | 155 kB 00:00 2024-01-11T17:46:28.738 INFO:teuthology.orchestra.run.smithi016.stdout:(75/147): python3-jwt-1.6.1-2.el8.noarch.rpm 146 kB/s | 43 kB 00:00 2024-01-11T17:46:28.798 INFO:teuthology.orchestra.run.smithi007.stdout:(119/147): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2024-01-11T17:46:28.814 INFO:teuthology.orchestra.run.smithi016.stdout:(76/147): python3-pycparser-2.14-14.el8.noarch. 653 kB/s | 109 kB 00:00 2024-01-11T17:46:28.824 INFO:teuthology.orchestra.run.smithi038.stdout:(140/147): xfsdump-3.1.8-6.el8.x86_64.rpm 1.4 MB/s | 333 kB 00:00 2024-01-11T17:46:28.831 INFO:teuthology.orchestra.run.smithi007.stdout:(120/147): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-01-11T17:46:28.848 INFO:teuthology.orchestra.run.smithi016.stdout:(77/147): python3-requests-oauthlib-1.0.0-1.el8 340 kB/s | 43 kB 00:00 2024-01-11T17:46:28.864 INFO:teuthology.orchestra.run.smithi016.stdout:(78/147): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2024-01-11T17:46:28.891 INFO:teuthology.orchestra.run.smithi038.stdout:(141/147): util-linux-user-2.32.1-43.el8.x86_64 286 kB/s | 103 kB 00:00 2024-01-11T17:46:28.923 INFO:teuthology.orchestra.run.smithi016.stdout:(79/147): libzstd-devel-1.4.4-1.el8.x86_64.rpm 402 kB/s | 44 kB 00:00 2024-01-11T17:46:28.933 INFO:teuthology.orchestra.run.smithi038.stdout:(142/147): iproute-6.2.0-5.el8_9.x86_64.rpm 3.7 MB/s | 854 kB 00:00 2024-01-11T17:46:28.948 INFO:teuthology.orchestra.run.smithi016.stdout:(80/147): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2024-01-11T17:46:29.015 INFO:teuthology.orchestra.run.smithi007.stdout:(121/147): openblas-threads-0.3.15-6.el8.x86_64 14 MB/s | 5.2 MB 00:00 2024-01-11T17:46:29.066 INFO:teuthology.orchestra.run.smithi007.stdout:(122/147): automake-1.16.1-8.el8.noarch.rpm 3.0 MB/s | 714 kB 00:00 2024-01-11T17:46:29.082 INFO:teuthology.orchestra.run.smithi016.stdout:(81/147): pcre2-devel-10.32-2.el8.x86_64.rpm 2.5 MB/s | 605 kB 00:00 2024-01-11T17:46:29.083 INFO:teuthology.orchestra.run.smithi038.stdout:(143/147): libfdisk-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 254 kB 00:00 2024-01-11T17:46:29.107 INFO:teuthology.orchestra.run.smithi016.stdout:(82/147): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-01-11T17:46:29.132 INFO:teuthology.orchestra.run.smithi016.stdout:(83/147): libselinux-devel-2.9-5.el8.x86_64.rpm 1.1 MB/s | 200 kB 00:00 2024-01-11T17:46:29.149 INFO:teuthology.orchestra.run.smithi007.stdout:(123/147): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-01-11T17:46:29.150 INFO:teuthology.orchestra.run.smithi038.stdout:(144/147): libblkid-2.32.1-43.el8.x86_64.rpm 677 kB/s | 220 kB 00:00 2024-01-11T17:46:29.191 INFO:teuthology.orchestra.run.smithi007.stdout:(124/147): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-01-11T17:46:29.216 INFO:teuthology.orchestra.run.smithi007.stdout:(125/147): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2024-01-11T17:46:29.224 INFO:teuthology.orchestra.run.smithi016.stdout:(84/147): keyutils-libs-devel-1.5.10-9.el8.x86_ 339 kB/s | 48 kB 00:00 2024-01-11T17:46:29.240 INFO:teuthology.orchestra.run.smithi038.stdout:(145/147): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.5 MB/s | 1.1 MB 00:00 2024-01-11T17:46:29.259 INFO:teuthology.orchestra.run.smithi016.stdout:(85/147): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 459 kB/s | 58 kB 00:00 2024-01-11T17:46:29.284 INFO:teuthology.orchestra.run.smithi016.stdout:(86/147): libsepol-devel-2.9-3.el8.x86_64.rpm 494 kB/s | 87 kB 00:00 2024-01-11T17:46:29.309 INFO:teuthology.orchestra.run.smithi038.stdout:(146/147): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.1 MB/s | 711 kB 00:00 2024-01-11T17:46:29.359 INFO:teuthology.orchestra.run.smithi016.stdout:(87/147): libcom_err-devel-1.45.6-4.el8.x86_64. 286 kB/s | 39 kB 00:00 2024-01-11T17:46:29.375 INFO:teuthology.orchestra.run.smithi007.stdout:(126/147): postgresql-10.23-3.module+el8.9.0+20 8.2 MB/s | 1.5 MB 00:00 2024-01-11T17:46:29.400 INFO:teuthology.orchestra.run.smithi007.stdout:(127/147): python3-pytz-2017.2-11.el8.noarch.rp 216 kB/s | 54 kB 00:00 2024-01-11T17:46:29.417 INFO:teuthology.orchestra.run.smithi016.stdout:(88/147): fuse-2.9.7-17.el8.x86_64.rpm 523 kB/s | 83 kB 00:00 2024-01-11T17:46:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:(128/147): postgresql-contrib-10.23-3.module+el 3.5 MB/s | 811 kB 00:00 2024-01-11T17:46:29.459 INFO:teuthology.orchestra.run.smithi016.stdout:(89/147): libgfortran-8.5.0-20.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2024-01-11T17:46:29.476 INFO:teuthology.orchestra.run.smithi038.stdout:(147/147): openssl-libs-1.1.1k-12.el8_9.x86_64. 4.5 MB/s | 1.5 MB 00:00 2024-01-11T17:46:29.479 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:46:29.479 INFO:teuthology.orchestra.run.smithi038.stdout:Total 11 MB/s | 223 MB 00:19 2024-01-11T17:46:29.493 INFO:teuthology.orchestra.run.smithi016.stdout:(90/147): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-01-11T17:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout:(129/147): lua-devel-5.3.4-12.el8.x86_64.rpm 200 kB/s | 30 kB 00:00 2024-01-11T17:46:29.627 INFO:teuthology.orchestra.run.smithi016.stdout:(91/147): libuuid-devel-2.32.1-43.el8.x86_64.rp 477 kB/s | 100 kB 00:00 2024-01-11T17:46:29.652 INFO:teuthology.orchestra.run.smithi016.stdout:(92/147): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2024-01-11T17:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout:(130/147): postgresql-server-10.23-3.module+el8 17 MB/s | 5.1 MB 00:00 2024-01-11T17:46:29.694 INFO:teuthology.orchestra.run.smithi016.stdout:(93/147): python3-cryptography-3.2.1-6.el8.x86_ 2.7 MB/s | 559 kB 00:00 2024-01-11T17:46:29.728 INFO:teuthology.orchestra.run.smithi016.stdout:(94/147): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 722 kB/s | 73 kB 00:00 2024-01-11T17:46:29.753 INFO:teuthology.orchestra.run.smithi016.stdout:(95/147): NetworkManager-initscripts-updown-1.4 1.4 MB/s | 143 kB 00:00 2024-01-11T17:46:29.778 INFO:teuthology.orchestra.run.smithi016.stdout:(96/147): elfutils-libelf-devel-0.189-3.el8.x86 737 kB/s | 62 kB 00:00 2024-01-11T17:46:29.878 INFO:teuthology.orchestra.run.smithi016.stdout:(97/147): librabbitmq-0.9.0-4.el8.x86_64.rpm 312 kB/s | 47 kB 00:00 2024-01-11T17:46:29.912 INFO:teuthology.orchestra.run.smithi016.stdout:(98/147): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.5 MB/s | 562 kB 00:00 2024-01-11T17:46:29.979 INFO:teuthology.orchestra.run.smithi016.stdout:(99/147): openssl-devel-1.1.1k-12.el8_9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-01-11T17:46:30.007 INFO:teuthology.orchestra.run.smithi016.stdout:(100/147): lua-devel-5.3.4-12.el8.x86_64.rpm 239 kB/s | 30 kB 00:00 2024-01-11T17:46:30.038 INFO:teuthology.orchestra.run.smithi016.stdout:(101/147): python3-jsonpointer-1.10-11.el8.noar 159 kB/s | 20 kB 00:00 2024-01-11T17:46:30.105 INFO:teuthology.orchestra.run.smithi016.stdout:(102/147): python3-prettytable-0.7.2-14.el8.noa 350 kB/s | 44 kB 00:00 2024-01-11T17:46:30.118 INFO:teuthology.orchestra.run.smithi007.stdout:(131/147): python3-scipy-1.0.0-21.module+el8.9. 19 MB/s | 14 MB 00:00 2024-01-11T17:46:30.239 INFO:teuthology.orchestra.run.smithi016.stdout:(103/147): python3-pyasn1-modules-0.3.7-6.el8.n 471 kB/s | 110 kB 00:00 2024-01-11T17:46:30.252 INFO:teuthology.orchestra.run.smithi007.stdout:(132/147): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-01-11T17:46:30.264 INFO:teuthology.orchestra.run.smithi016.stdout:(104/147): python3-jsonpatch-1.21-2.el8.noarch. 118 kB/s | 27 kB 00:00 2024-01-11T17:46:30.289 INFO:teuthology.orchestra.run.smithi016.stdout:(105/147): python3-pyasn1-0.3.7-6.el8.noarch.rp 688 kB/s | 126 kB 00:00 2024-01-11T17:46:30.352 INFO:teuthology.orchestra.run.smithi007.stdout:(133/147): librbd1-19.0.0-446.gc27531ad.el8.x86 5.6 MB/s | 3.8 MB 00:00 2024-01-11T17:46:30.381 INFO:teuthology.orchestra.run.smithi016.stdout:(106/147): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2024-01-11T17:46:30.406 INFO:teuthology.orchestra.run.smithi016.stdout:(107/147): python3-werkzeug-0.12.2-4.el8.noarch 2.7 MB/s | 457 kB 00:00 2024-01-11T17:46:30.411 INFO:teuthology.orchestra.run.smithi007.stdout:(134/147): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-01-11T17:46:30.431 INFO:teuthology.orchestra.run.smithi016.stdout:(108/147): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-01-11T17:46:30.457 INFO:teuthology.orchestra.run.smithi016.stdout:(109/147): python3-markupsafe-0.23-19.el8.x86_6 773 kB/s | 39 kB 00:00 2024-01-11T17:46:30.478 INFO:teuthology.orchestra.run.smithi007.stdout:(135/147): libmount-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 237 kB 00:00 2024-01-11T17:46:30.503 INFO:teuthology.orchestra.run.smithi007.stdout:(136/147): libsmartcols-2.32.1-43.el8.x86_64.rp 1.9 MB/s | 179 kB 00:00 2024-01-11T17:46:30.541 INFO:teuthology.orchestra.run.smithi016.stdout:(110/147): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2024-01-11T17:46:30.566 INFO:teuthology.orchestra.run.smithi016.stdout:(111/147): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-01-11T17:46:30.591 INFO:teuthology.orchestra.run.smithi016.stdout:(112/147): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2024-01-11T17:46:30.620 INFO:teuthology.orchestra.run.smithi007.stdout:(137/147): libuuid-2.32.1-43.el8.x86_64.rpm 698 kB/s | 99 kB 00:00 2024-01-11T17:46:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:(113/147): uuid-1.6.2-43.el8.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-01-11T17:46:30.656 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-01-11T17:46:30.678 INFO:teuthology.orchestra.run.smithi007.stdout:(138/147): librados2-19.0.0-446.gc27531ad.el8.x 3.5 MB/s | 3.8 MB 00:01 2024-01-11T17:46:30.692 INFO:teuthology.orchestra.run.smithi016.stdout:(114/147): python3-pyOpenSSL-19.0.0-1.el8.noarc 824 kB/s | 103 kB 00:00 2024-01-11T17:46:30.729 INFO:teuthology.orchestra.run.smithi007.stdout:(139/147): util-linux-user-2.32.1-43.el8.x86_64 944 kB/s | 103 kB 00:00 2024-01-11T17:46:30.776 INFO:teuthology.orchestra.run.smithi016.stdout:(115/147): python3-jinja2-2.10.1-3.el8.noarch.r 3.9 MB/s | 538 kB 00:00 2024-01-11T17:46:30.779 INFO:teuthology.orchestra.run.smithi007.stdout:(140/147): util-linux-2.32.1-43.el8.x86_64.rpm 9.0 MB/s | 2.5 MB 00:00 2024-01-11T17:46:30.838 INFO:teuthology.orchestra.run.smithi007.stdout:(141/147): xfsdump-3.1.8-6.el8.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-01-11T17:46:30.884 INFO:teuthology.orchestra.run.smithi016.stdout:(116/147): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-01-11T17:46:30.922 INFO:teuthology.orchestra.run.smithi007.stdout:(142/147): iproute-6.2.0-5.el8_9.x86_64.rpm 4.3 MB/s | 854 kB 00:00 2024-01-11T17:46:30.926 INFO:teuthology.orchestra.run.smithi016.stdout:(117/147): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-01-11T17:46:30.972 INFO:teuthology.orchestra.run.smithi007.stdout:(143/147): libblkid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-01-11T17:46:30.997 INFO:teuthology.orchestra.run.smithi007.stdout:(144/147): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2024-01-11T17:46:31.026 INFO:teuthology.orchestra.run.smithi016.stdout:(118/147): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-01-11T17:46:31.052 INFO:teuthology.orchestra.run.smithi016.stdout:(119/147): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-01-11T17:46:31.085 INFO:teuthology.orchestra.run.smithi016.stdout:(120/147): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2024-01-11T17:46:31.098 INFO:teuthology.orchestra.run.smithi007.stdout:(145/147): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-01-11T17:46:31.107 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-01-11T17:46:31.107 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-01-11T17:46:31.165 INFO:teuthology.orchestra.run.smithi007.stdout:(146/147): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-01-11T17:46:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:(147/147): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.2 MB/s | 711 kB 00:00 2024-01-11T17:46:31.192 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:46:31.192 INFO:teuthology.orchestra.run.smithi007.stdout:Total 11 MB/s | 223 MB 00:19 2024-01-11T17:46:31.210 INFO:teuthology.orchestra.run.smithi016.stdout:(121/147): python3-mako-1.0.6-14.el8.noarch.rpm 990 kB/s | 157 kB 00:00 2024-01-11T17:46:31.353 INFO:teuthology.orchestra.run.smithi016.stdout:(122/147): openblas-threads-0.3.15-6.el8.x86_64 16 MB/s | 5.2 MB 00:00 2024-01-11T17:46:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:(123/147): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-01-11T17:46:31.503 INFO:teuthology.orchestra.run.smithi016.stdout:(124/147): automake-1.16.1-8.el8.noarch.rpm 2.4 MB/s | 714 kB 00:00 2024-01-11T17:46:31.528 INFO:teuthology.orchestra.run.smithi016.stdout:(125/147): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-01-11T17:46:31.579 INFO:teuthology.orchestra.run.smithi016.stdout:(126/147): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2024-01-11T17:46:31.637 INFO:teuthology.orchestra.run.smithi016.stdout:(127/147): python3-pytz-2017.2-11.el8.noarch.rp 406 kB/s | 54 kB 00:00 2024-01-11T17:46:31.688 INFO:teuthology.orchestra.run.smithi016.stdout:(128/147): postgresql-10.23-3.module+el8.9.0+20 9.5 MB/s | 1.5 MB 00:00 2024-01-11T17:46:31.730 INFO:teuthology.orchestra.run.smithi016.stdout:(129/147): postgresql-contrib-10.23-3.module+el 5.3 MB/s | 811 kB 00:00 2024-01-11T17:46:31.872 INFO:teuthology.orchestra.run.smithi016.stdout:(130/147): postgresql-server-10.23-3.module+el8 22 MB/s | 5.1 MB 00:00 2024-01-11T17:46:32.155 INFO:teuthology.orchestra.run.smithi016.stdout:(131/147): python3-scipy-1.0.0-21.module+el8.9. 29 MB/s | 14 MB 00:00 2024-01-11T17:46:32.289 INFO:teuthology.orchestra.run.smithi016.stdout:(132/147): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-01-11T17:46:32.389 INFO:teuthology.orchestra.run.smithi016.stdout:(133/147): librbd1-19.0.0-446.gc27531ad.el8.x86 7.3 MB/s | 3.8 MB 00:00 2024-01-11T17:46:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:(134/147): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-01-11T17:46:32.462 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-01-11T17:46:32.474 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-01-11T17:46:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:(135/147): libmount-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 237 kB 00:00 2024-01-11T17:46:32.548 INFO:teuthology.orchestra.run.smithi016.stdout:(136/147): libsmartcols-2.32.1-43.el8.x86_64.rp 1.8 MB/s | 179 kB 00:00 2024-01-11T17:46:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:(137/147): libuuid-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-01-11T17:46:32.615 INFO:teuthology.orchestra.run.smithi016.stdout:(138/147): util-linux-user-2.32.1-43.el8.x86_64 2.0 MB/s | 103 kB 00:00 2024-01-11T17:46:32.666 INFO:teuthology.orchestra.run.smithi016.stdout:(139/147): util-linux-2.32.1-43.el8.x86_64.rpm 21 MB/s | 2.5 MB 00:00 2024-01-11T17:46:32.725 INFO:teuthology.orchestra.run.smithi016.stdout:(140/147): librados2-19.0.0-446.gc27531ad.el8.x 3.8 MB/s | 3.8 MB 00:00 2024-01-11T17:46:32.800 INFO:teuthology.orchestra.run.smithi016.stdout:(141/147): xfsdump-3.1.8-6.el8.x86_64.rpm 1.8 MB/s | 333 kB 00:00 2024-01-11T17:46:32.825 INFO:teuthology.orchestra.run.smithi016.stdout:(142/147): libblkid-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2024-01-11T17:46:32.859 INFO:teuthology.orchestra.run.smithi016.stdout:(143/147): libfdisk-2.32.1-43.el8.x86_64.rpm 4.2 MB/s | 254 kB 00:00 2024-01-11T17:46:32.892 INFO:teuthology.orchestra.run.smithi016.stdout:(144/147): iproute-6.2.0-5.el8_9.x86_64.rpm 3.7 MB/s | 854 kB 00:00 2024-01-11T17:46:32.934 INFO:teuthology.orchestra.run.smithi016.stdout:(145/147): xfsprogs-5.0.0-12.el8.x86_64.rpm 9.8 MB/s | 1.1 MB 00:00 2024-01-11T17:46:32.976 INFO:teuthology.orchestra.run.smithi016.stdout:(146/147): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.9 MB/s | 711 kB 00:00 2024-01-11T17:46:33.018 INFO:teuthology.orchestra.run.smithi016.stdout:(147/147): openssl-libs-1.1.1k-12.el8_9.x86_64. 12 MB/s | 1.5 MB 00:00 2024-01-11T17:46:33.021 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:46:33.021 INFO:teuthology.orchestra.run.smithi016.stdout:Total 11 MB/s | 223 MB 00:20 2024-01-11T17:46:33.398 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-01-11T17:46:33.876 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-01-11T17:46:33.876 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-01-11T17:46:34.123 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-01-11T17:46:34.573 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-01-11T17:46:34.574 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-01-11T17:46:35.720 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-01-11T17:46:35.733 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-01-11T17:46:35.931 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-01-11T17:46:35.943 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-01-11T17:46:38.556 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-01-11T17:46:38.723 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-11T17:46:39.012 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-11T17:46:39.259 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-11T17:46:39.486 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/163 2024-01-11T17:46:39.674 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-11T17:46:39.780 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-11T17:46:39.974 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-11T17:46:40.049 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-11T17:46:40.301 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/163 2024-01-11T17:46:40.516 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/163 2024-01-11T17:46:40.591 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/163 2024-01-11T17:46:40.790 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/163 2024-01-11T17:46:41.008 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-11T17:46:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-01-11T17:46:41.463 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-11T17:46:41.633 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-11T17:46:41.699 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:46:41.860 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-01-11T17:46:41.887 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-11T17:46:42.023 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-11T17:46:42.153 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:46:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-11T17:46:42.290 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-11T17:46:42.383 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:46:42.392 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/163 2024-01-11T17:46:42.529 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-11T17:46:42.530 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:46:42.589 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-11T17:46:42.688 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-11T17:46:42.727 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/163 2024-01-11T17:46:42.741 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:46:42.848 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:46:42.881 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-11T17:46:42.934 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-11T17:46:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-11T17:46:43.006 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-11T17:46:43.187 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/163 2024-01-11T17:46:43.209 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-11T17:46:43.284 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-11T17:46:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/163 2024-01-11T17:46:43.515 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/163 2024-01-11T17:46:43.631 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-11T17:46:43.721 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/163 2024-01-11T17:46:43.750 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 12/163 2024-01-11T17:46:43.768 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-11T17:46:43.915 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:46:43.951 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-11T17:46:44.037 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-11T17:46:44.127 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:46:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-11T17:46:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-11T17:46:44.457 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-11T17:46:44.549 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-11T17:46:44.707 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-11T17:46:44.719 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-11T17:46:44.759 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpq-13.11-1.el8.x86_64 15/163 2024-01-11T17:46:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:46:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:46:45.031 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/163 2024-01-11T17:46:45.263 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/163 2024-01-11T17:46:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:46:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:46:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:46:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:46:45.658 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/163 2024-01-11T17:46:45.783 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:46:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:46:45.933 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/163 2024-01-11T17:46:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:46:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:46:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:46:46.082 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 20/163 2024-01-11T17:46:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:46:46.191 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 20/163 2024-01-11T17:46:46.373 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 21/163 2024-01-11T17:46:46.453 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 21/163 2024-01-11T17:46:46.648 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/163 2024-01-11T17:46:46.798 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/163 2024-01-11T17:46:46.948 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/163 2024-01-11T17:46:46.999 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 12/163 2024-01-11T17:46:47.058 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 12/163 2024-01-11T17:46:47.217 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:46:47.222 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:46:47.433 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:46:47.480 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:46:47.674 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/163 2024-01-11T17:46:47.725 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-11T17:46:47.738 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-11T17:46:47.818 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-11T17:46:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-11T17:46:47.857 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 24/163 2024-01-11T17:46:48.061 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpq-13.11-1.el8.x86_64 15/163 2024-01-11T17:46:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpq-13.11-1.el8.x86_64 15/163 2024-01-11T17:46:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/163 2024-01-11T17:46:48.374 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/163 2024-01-11T17:46:48.527 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 24/163 2024-01-11T17:46:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/163 2024-01-11T17:46:48.556 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/163 2024-01-11T17:46:48.673 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.15-6.el8.x86_64 25/163 2024-01-11T17:46:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/163 2024-01-11T17:46:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/163 2024-01-11T17:46:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/163 2024-01-11T17:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/163 2024-01-11T17:46:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 20/163 2024-01-11T17:46:49.439 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 25/163 2024-01-11T17:46:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 20/163 2024-01-11T17:46:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 20/163 2024-01-11T17:46:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 20/163 2024-01-11T17:46:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 21/163 2024-01-11T17:46:49.714 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/163 2024-01-11T17:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 21/163 2024-01-11T17:46:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 21/163 2024-01-11T17:46:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 21/163 2024-01-11T17:46:49.865 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 27/163 2024-01-11T17:46:49.982 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 22/163 2024-01-11T17:46:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 22/163 2024-01-11T17:46:50.055 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 28/163 2024-01-11T17:46:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/163 2024-01-11T17:46:50.300 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/163 2024-01-11T17:46:50.372 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/163 2024-01-11T17:46:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/163 2024-01-11T17:46:50.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/163 2024-01-11T17:46:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-11T17:46:50.673 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 30/163 2024-01-11T17:46:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-11T17:46:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-11T17:46:50.918 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/163 2024-01-11T17:46:50.923 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-11T17:46:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-11T17:46:51.050 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-11T17:46:51.155 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x 32/163 2024-01-11T17:46:51.287 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8 33/163 2024-01-11T17:46:51.545 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 34/163 2024-01-11T17:46:51.704 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-11T17:46:51.774 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-11T17:46:51.871 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-11T17:46:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-11T17:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-11T17:46:52.589 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-11T17:46:52.632 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/163 2024-01-11T17:46:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.15-6.el8.x86_64 28/163 2024-01-11T17:46:52.792 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.15-6.el8.x86_64 28/163 2024-01-11T17:46:53.014 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/163 2024-01-11T17:46:53.314 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/163 2024-01-11T17:46:53.504 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/163 2024-01-11T17:46:53.567 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 28/163 2024-01-11T17:46:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 28/163 2024-01-11T17:46:53.855 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 29/163 2024-01-11T17:46:53.857 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 29/163 2024-01-11T17:46:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 30/163 2024-01-11T17:46:54.093 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 30/163 2024-01-11T17:46:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/163 2024-01-11T17:46:54.387 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/163 2024-01-11T17:46:54.545 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x 32/163 2024-01-11T17:46:54.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x 32/163 2024-01-11T17:46:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8 33/163 2024-01-11T17:46:54.722 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8 33/163 2024-01-11T17:46:54.944 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 34/163 2024-01-11T17:46:55.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 34/163 2024-01-11T17:46:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-11T17:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-11T17:46:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/163 2024-01-11T17:46:55.789 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 39/163 2024-01-11T17:46:56.001 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/163 2024-01-11T17:46:56.252 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/163 2024-01-11T17:46:56.489 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 42/163 2024-01-11T17:46:56.683 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 43/163 2024-01-11T17:46:56.897 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 44/163 2024-01-11T17:46:57.110 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/163 2024-01-11T17:46:57.231 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/163 2024-01-11T17:46:57.234 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-11T17:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-11T17:46:57.447 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-11T17:46:57.556 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-11T17:46:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 36/163 2024-01-11T17:46:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 36/163 2024-01-11T17:46:57.782 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/163 2024-01-11T17:46:58.017 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/163 2024-01-11T17:46:58.189 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-11T17:46:58.685 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-11T17:46:58.714 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 37/163 2024-01-11T17:46:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 37/163 2024-01-11T17:46:59.014 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-11T17:46:59.036 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-11T17:46:59.049 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 38/163 2024-01-11T17:46:59.205 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 38/163 2024-01-11T17:46:59.252 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.7-17.el8.x86_64 51/163 2024-01-11T17:46:59.446 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 39/163 2024-01-11T17:46:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 39/163 2024-01-11T17:46:59.482 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 52/163 2024-01-11T17:46:59.645 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/163 2024-01-11T17:46:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/163 2024-01-11T17:46:59.728 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/163 2024-01-11T17:46:59.881 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/163 2024-01-11T17:46:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/163 2024-01-11T17:46:59.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/163 2024-01-11T17:47:00.069 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/163 2024-01-11T17:47:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 42/163 2024-01-11T17:47:00.173 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 42/163 2024-01-11T17:47:00.326 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 43/163 2024-01-11T17:47:00.399 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 56/163 2024-01-11T17:47:00.413 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 43/163 2024-01-11T17:47:00.512 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 44/163 2024-01-11T17:47:00.591 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 44/163 2024-01-11T17:47:00.701 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 57/163 2024-01-11T17:47:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/163 2024-01-11T17:47:00.823 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/163 2024-01-11T17:47:00.862 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/163 2024-01-11T17:47:00.903 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/163 2024-01-11T17:47:00.936 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/163 2024-01-11T17:47:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-11T17:47:01.099 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 59/163 2024-01-11T17:47:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-11T17:47:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-11T17:47:01.278 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-11T17:47:01.358 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 60/163 2024-01-11T17:47:01.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/163 2024-01-11T17:47:01.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/163 2024-01-11T17:47:01.702 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/163 2024-01-11T17:47:01.715 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 61/163 2024-01-11T17:47:01.750 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/163 2024-01-11T17:47:01.882 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-11T17:47:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-11T17:47:02.029 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 62/163 2024-01-11T17:47:02.316 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 63/163 2024-01-11T17:47:02.384 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-11T17:47:02.533 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 64/163 2024-01-11T17:47:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-11T17:47:02.699 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-11T17:47:02.785 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 65/163 2024-01-11T17:47:02.811 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-11T17:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-11T17:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-11T17:47:03.045 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 66/163 2024-01-11T17:47:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.1-8.el8.noarch 51/163 2024-01-11T17:47:03.273 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 67/163 2024-01-11T17:47:03.277 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.3.4-12.el8.x86_64 51/163 2024-01-11T17:47:03.290 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/163 2024-01-11T17:47:03.363 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/163 2024-01-11T17:47:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.1-8.el8.noarch 52/163 2024-01-11T17:47:03.508 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 68/163 2024-01-11T17:47:03.565 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 53/163 2024-01-11T17:47:03.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-43.el8.x86_64 53/163 2024-01-11T17:47:03.645 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 53/163 2024-01-11T17:47:03.699 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/163 2024-01-11T17:47:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 53/163 2024-01-11T17:47:03.817 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-7.el8.x86_64 54/163 2024-01-11T17:47:03.902 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 54/163 2024-01-11T17:47:03.928 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 70/163 2024-01-11T17:47:04.009 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 54/163 2024-01-11T17:47:04.015 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 55/163 2024-01-11T17:47:04.179 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-7.el8.x86_64 55/163 2024-01-11T17:47:04.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 56/163 2024-01-11T17:47:04.291 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 71/163 2024-01-11T17:47:04.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 57/163 2024-01-11T17:47:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 56/163 2024-01-11T17:47:04.519 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.1-8.el8.noarch 72/163 2024-01-11T17:47:04.552 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.3.4-12.el8.x86_64 58/163 2024-01-11T17:47:04.562 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 57/163 2024-01-11T17:47:04.688 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/163 2024-01-11T17:47:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-17.el8.x86_64 59/163 2024-01-11T17:47:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 58/163 2024-01-11T17:47:04.777 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/163 2024-01-11T17:47:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.7-17.el8.x86_64 59/163 2024-01-11T17:47:04.971 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 74/163 2024-01-11T17:47:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 60/163 2024-01-11T17:47:05.059 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 74/163 2024-01-11T17:47:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/163 2024-01-11T17:47:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 60/163 2024-01-11T17:47:05.281 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : jq-1.6-7.el8.x86_64 75/163 2024-01-11T17:47:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/163 2024-01-11T17:47:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/163 2024-01-11T17:47:05.454 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 76/163 2024-01-11T17:47:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/163 2024-01-11T17:47:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/163 2024-01-11T17:47:05.663 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 77/163 2024-01-11T17:47:05.843 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 78/163 2024-01-11T17:47:05.851 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/163 2024-01-11T17:47:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 64/163 2024-01-11T17:47:06.024 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.3.4-12.el8.x86_64 79/163 2024-01-11T17:47:06.175 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 64/163 2024-01-11T17:47:06.231 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 80/163 2024-01-11T17:47:06.321 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 65/163 2024-01-11T17:47:06.437 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 81/163 2024-01-11T17:47:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 65/163 2024-01-11T17:47:06.688 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 66/163 2024-01-11T17:47:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 66/163 2024-01-11T17:47:06.725 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 82/163 2024-01-11T17:47:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 67/163 2024-01-11T17:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 67/163 2024-01-11T17:47:06.954 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 83/163 2024-01-11T17:47:07.137 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 68/163 2024-01-11T17:47:07.157 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 68/163 2024-01-11T17:47:07.210 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 84/163 2024-01-11T17:47:07.375 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 85/163 2024-01-11T17:47:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 69/163 2024-01-11T17:47:07.601 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 86/163 2024-01-11T17:47:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 69/163 2024-01-11T17:47:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 70/163 2024-01-11T17:47:07.874 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 87/163 2024-01-11T17:47:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 70/163 2024-01-11T17:47:08.059 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 88/163 2024-01-11T17:47:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 71/163 2024-01-11T17:47:08.266 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 89/163 2024-01-11T17:47:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 71/163 2024-01-11T17:47:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/163 2024-01-11T17:47:08.427 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 90/163 2024-01-11T17:47:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/163 2024-01-11T17:47:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/163 2024-01-11T17:47:08.673 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 91/163 2024-01-11T17:47:08.692 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/163 2024-01-11T17:47:08.802 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/163 2024-01-11T17:47:08.909 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 92/163 2024-01-11T17:47:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/163 2024-01-11T17:47:09.083 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/163 2024-01-11T17:47:09.085 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 93/163 2024-01-11T17:47:09.313 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/163 2024-01-11T17:47:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/163 2024-01-11T17:47:09.366 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-2.6-1.el8.noarch 94/163 2024-01-11T17:47:09.551 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 77/163 2024-01-11T17:47:09.555 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/163 2024-01-11T17:47:09.613 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 95/163 2024-01-11T17:47:09.771 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 77/163 2024-01-11T17:47:09.797 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 96/163 2024-01-11T17:47:09.797 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 78/163 2024-01-11T17:47:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 78/163 2024-01-11T17:47:10.051 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 79/163 2024-01-11T17:47:10.066 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 97/163 2024-01-11T17:47:10.224 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 80/163 2024-01-11T17:47:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 79/163 2024-01-11T17:47:10.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 81/163 2024-01-11T17:47:10.463 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 80/163 2024-01-11T17:47:10.655 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 81/163 2024-01-11T17:47:10.736 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 82/163 2024-01-11T17:47:10.929 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 82/163 2024-01-11T17:47:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 83/163 2024-01-11T17:47:11.203 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 83/163 2024-01-11T17:47:11.236 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 84/163 2024-01-11T17:47:11.249 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 98/163 2024-01-11T17:47:11.395 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 85/163 2024-01-11T17:47:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 84/163 2024-01-11T17:47:11.582 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 99/163 2024-01-11T17:47:11.592 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 85/163 2024-01-11T17:47:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 86/163 2024-01-11T17:47:11.804 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 86/163 2024-01-11T17:47:11.859 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 100/163 2024-01-11T17:47:11.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 87/163 2024-01-11T17:47:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 88/163 2024-01-11T17:47:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 87/163 2024-01-11T17:47:12.161 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 101/163 2024-01-11T17:47:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 89/163 2024-01-11T17:47:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 88/163 2024-01-11T17:47:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 90/163 2024-01-11T17:47:12.485 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 102/163 2024-01-11T17:47:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 89/163 2024-01-11T17:47:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 90/163 2024-01-11T17:47:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 91/163 2024-01-11T17:47:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 91/163 2024-01-11T17:47:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 92/163 2024-01-11T17:47:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 92/163 2024-01-11T17:47:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 93/163 2024-01-11T17:47:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 93/163 2024-01-11T17:47:13.402 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 94/163 2024-01-11T17:47:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-2.6-1.el8.noarch 94/163 2024-01-11T17:47:13.631 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.5.0-1.el8.noarch 103/163 2024-01-11T17:47:13.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 95/163 2024-01-11T17:47:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 95/163 2024-01-11T17:47:13.848 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 96/163 2024-01-11T17:47:13.983 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 104/163 2024-01-11T17:47:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 96/163 2024-01-11T17:47:14.118 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 97/163 2024-01-11T17:47:14.221 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 105/163 2024-01-11T17:47:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 97/163 2024-01-11T17:47:15.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 98/163 2024-01-11T17:47:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 98/163 2024-01-11T17:47:15.718 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 99/163 2024-01-11T17:47:15.964 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 99/163 2024-01-11T17:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 100/163 2024-01-11T17:47:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 100/163 2024-01-11T17:47:16.313 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 101/163 2024-01-11T17:47:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 101/163 2024-01-11T17:47:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 102/163 2024-01-11T17:47:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 102/163 2024-01-11T17:47:17.185 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.5.0-1.el8.noarch 103/163 2024-01-11T17:47:17.401 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 104/163 2024-01-11T17:47:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 105/163 2024-01-11T17:47:17.814 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.5.0-1.el8.noarch 103/163 2024-01-11T17:47:18.227 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 104/163 2024-01-11T17:47:18.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 105/163 2024-01-11T17:47:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:19.349 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:20.574 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 107/163 2024-01-11T17:47:20.669 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 107/163 2024-01-11T17:47:20.844 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 108/163 2024-01-11T17:47:22.668 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:22.889 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:23.550 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:23.575 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 106/163 2024-01-11T17:47:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 107/163 2024-01-11T17:47:24.710 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 107/163 2024-01-11T17:47:24.736 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 107/163 2024-01-11T17:47:24.806 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 107/163 2024-01-11T17:47:24.907 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 108/163 2024-01-11T17:47:24.980 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 108/163 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 108/163 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /run 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-01-11T17:47:42.768 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-01-11T17:47:42.769 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2024-01-11T17:47:42.769 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:47:43.159 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 109/163 2024-01-11T17:47:44.986 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 109/163 2024-01-11T17:47:45.162 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 110/163 2024-01-11T17:47:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 110/163 2024-01-11T17:47:45.566 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 111/163 2024-01-11T17:47:46.098 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 111/163 2024-01-11T17:47:46.250 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 112/163 2024-01-11T17:47:46.272 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 112/163 2024-01-11T17:47:46.906 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 108/163 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2024-01-11T17:47:46.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 108/163 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /run 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-01-11T17:47:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2024-01-11T17:47:46.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:47:47.311 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 109/163 2024-01-11T17:47:47.354 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 109/163 2024-01-11T17:47:48.442 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.0.0-446.gc27531ad.el8.noarch 113/163 2024-01-11T17:47:48.624 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.0.0-446.gc27531ad.el8.noarch 113/163 2024-01-11T17:47:48.750 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el 114/163 2024-01-11T17:47:48.977 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.e 115/163 2024-01-11T17:47:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 109/163 2024-01-11T17:47:49.207 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 116/163 2024-01-11T17:47:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 109/163 2024-01-11T17:47:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 110/163 2024-01-11T17:47:49.454 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 116/163 2024-01-11T17:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 110/163 2024-01-11T17:47:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 110/163 2024-01-11T17:47:49.736 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 117/163 2024-01-11T17:47:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 110/163 2024-01-11T17:47:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 111/163 2024-01-11T17:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 111/163 2024-01-11T17:47:50.323 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 111/163 2024-01-11T17:47:50.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 112/163 2024-01-11T17:47:50.549 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 112/163 2024-01-11T17:47:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 111/163 2024-01-11T17:47:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 112/163 2024-01-11T17:47:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 112/163 2024-01-11T17:47:50.810 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 117/163 2024-01-11T17:47:50.957 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 118/163 2024-01-11T17:47:51.066 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 118/163 2024-01-11T17:47:51.431 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8 119/163 2024-01-11T17:47:51.603 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 120/163 2024-01-11T17:47:51.650 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 120/163 2024-01-11T17:47:51.815 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 121/163 2024-01-11T17:47:51.831 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 121/163 2024-01-11T17:47:52.499 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-446.gc27531ad.el8.noarch 113/163 2024-01-11T17:47:52.668 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-446.gc27531ad.el8.noarch 113/163 2024-01-11T17:47:52.794 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el 114/163 2024-01-11T17:47:53.038 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.e 115/163 2024-01-11T17:47:53.286 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.0.0-446.gc27531ad.el8.x86_64 122/163 2024-01-11T17:47:53.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 116/163 2024-01-11T17:47:53.503 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 123/163 2024-01-11T17:47:53.546 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 123/163 2024-01-11T17:47:53.559 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 116/163 2024-01-11T17:47:53.740 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-446.gc27531 124/163 2024-01-11T17:47:53.755 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-446.gc27531ad.el8.noarch 113/163 2024-01-11T17:47:53.772 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 117/163 2024-01-11T17:47:53.791 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-446.gc27531 124/163 2024-01-11T17:47:53.911 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-446.gc27531ad.el8.noarch 113/163 2024-01-11T17:47:53.973 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 125/163 2024-01-11T17:47:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el 114/163 2024-01-11T17:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.e 115/163 2024-01-11T17:47:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 125/163 2024-01-11T17:47:54.511 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 116/163 2024-01-11T17:47:54.581 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 126/163 2024-01-11T17:47:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 116/163 2024-01-11T17:47:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 117/163 2024-01-11T17:47:54.990 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 117/163 2024-01-11T17:47:55.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 118/163 2024-01-11T17:47:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 118/163 2024-01-11T17:47:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8 119/163 2024-01-11T17:47:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 120/163 2024-01-11T17:47:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 120/163 2024-01-11T17:47:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 121/163 2024-01-11T17:47:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 121/163 2024-01-11T17:47:56.144 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 117/163 2024-01-11T17:47:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 118/163 2024-01-11T17:47:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 118/163 2024-01-11T17:47:56.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8 119/163 2024-01-11T17:47:57.040 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 120/163 2024-01-11T17:47:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 120/163 2024-01-11T17:47:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 121/163 2024-01-11T17:47:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 121/163 2024-01-11T17:47:57.416 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-446.gc27531ad.el8.x86_64 122/163 2024-01-11T17:47:57.664 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 123/163 2024-01-11T17:47:57.706 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 123/163 2024-01-11T17:47:57.884 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-446.gc27531 124/163 2024-01-11T17:47:57.934 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-446.gc27531 124/163 2024-01-11T17:47:58.159 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 125/163 2024-01-11T17:47:58.575 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 125/163 2024-01-11T17:47:58.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 126/163 2024-01-11T17:47:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-446.gc27531ad.el8.x86_64 122/163 2024-01-11T17:47:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 123/163 2024-01-11T17:47:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 123/163 2024-01-11T17:47:59.302 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-446.gc27531 124/163 2024-01-11T17:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-446.gc27531 124/163 2024-01-11T17:47:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 125/163 2024-01-11T17:48:00.000 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 125/163 2024-01-11T17:48:00.252 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 126/163 2024-01-11T17:48:00.288 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 126/163 2024-01-11T17:48:00.807 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 127/163 2024-01-11T17:48:01.117 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 128/163 2024-01-11T17:48:01.339 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-25.el8.x86_64 129/163 2024-01-11T17:48:02.162 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 129/163 2024-01-11T17:48:02.470 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 130/163 2024-01-11T17:48:02.813 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 131/163 2024-01-11T17:48:03.039 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 132/163 2024-01-11T17:48:03.261 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 133/163 2024-01-11T17:48:03.502 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 134/163 2024-01-11T17:48:03.731 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_6 135/163 2024-01-11T17:48:04.527 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 126/163 2024-01-11T17:48:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 127/163 2024-01-11T17:48:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/163 2024-01-11T17:48:06.221 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 126/163 2024-01-11T17:48:06.291 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/163 2024-01-11T17:48:06.313 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:06.630 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:06.670 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:06.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 130/163 2024-01-11T17:48:07.127 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 137/163 2024-01-11T17:48:07.152 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 131/163 2024-01-11T17:48:07.357 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-25.el8.x86_64 132/163 2024-01-11T17:48:07.374 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 132/163 2024-01-11T17:48:07.467 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 138/163 2024-01-11T17:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 127/163 2024-01-11T17:48:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 133/163 2024-01-11T17:48:07.698 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 139/163 2024-01-11T17:48:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 134/163 2024-01-11T17:48:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/163 2024-01-11T17:48:07.922 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 140/163 2024-01-11T17:48:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_6 135/163 2024-01-11T17:48:08.151 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/163 2024-01-11T17:48:08.156 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 141/163 2024-01-11T17:48:08.455 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 142/163 2024-01-11T17:48:08.610 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 130/163 2024-01-11T17:48:08.698 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 143/163 2024-01-11T17:48:08.913 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 144/163 2024-01-11T17:48:08.925 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 144/163 2024-01-11T17:48:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 131/163 2024-01-11T17:48:09.105 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/163 2024-01-11T17:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-25.el8.x86_64 132/163 2024-01-11T17:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 132/163 2024-01-11T17:48:09.312 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.1-9.el8.x86_64 146/163 2024-01-11T17:48:09.401 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 146/163 2024-01-11T17:48:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 133/163 2024-01-11T17:48:09.597 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 134/163 2024-01-11T17:48:09.629 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.0.4-10.el8.x86_64 147/163 2024-01-11T17:48:09.646 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 147/163 2024-01-11T17:48:09.802 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_6 135/163 2024-01-11T17:48:09.814 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 148/163 2024-01-11T17:48:10.023 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 149/163 2024-01-11T17:48:10.192 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 150/163 2024-01-11T17:48:10.192 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-11T17:48:10.483 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-11T17:48:10.483 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-11T17:48:10.663 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:10.725 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-11T17:48:10.725 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-11T17:48:10.970 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-11T17:48:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:11.040 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:11.181 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 154/163 2024-01-11T17:48:11.376 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 155/163 2024-01-11T17:48:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 137/163 2024-01-11T17:48:11.631 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 156/163 2024-01-11T17:48:11.631 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 158/163 2024-01-11T17:48:11.927 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 138/163 2024-01-11T17:48:12.099 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 158/163 2024-01-11T17:48:12.099 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-11T17:48:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 139/163 2024-01-11T17:48:12.357 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-11T17:48:12.357 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-11T17:48:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 140/163 2024-01-11T17:48:12.615 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-11T17:48:12.615 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-11T17:48:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 141/163 2024-01-11T17:48:12.832 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-11T17:48:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 142/163 2024-01-11T17:48:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 143/163 2024-01-11T17:48:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 162/163 2024-01-11T17:48:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:13.506 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/163 2024-01-11T17:48:13.594 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/163 2024-01-11T17:48:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-11T17:48:13.807 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/163 2024-01-11T17:48:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/163 2024-01-11T17:48:13.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 146/163 2024-01-11T17:48:13.983 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 146/163 2024-01-11T17:48:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 137/163 2024-01-11T17:48:14.159 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/163 2024-01-11T17:48:14.317 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 148/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.0.0-446.gc27531ad.el8.x86_64 1/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 2/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 3/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 4/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-446.gc27531 5/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 6/163 2024-01-11T17:48:14.330 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 7/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 8/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 12/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_6 14/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 15/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 16/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 17/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 18/163 2024-01-11T17:48:14.331 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 19/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8 20/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x 21/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 22/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 23/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 24/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 25/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 26/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 27/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 28/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.e 29/163 2024-01-11T17:48:14.332 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 30/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 31/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 32/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8 33/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 34/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el 35/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 36/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.0.0-446.gc27531ad.el8.noarch 37/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 38/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/163 2024-01-11T17:48:14.345 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/163 2024-01-11T17:48:14.346 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/163 2024-01-11T17:48:14.347 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.3.4-12.el8.x86_64 73/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 74/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 75/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 76/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 77/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 78/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 79/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.1-9.el8.x86_64 81/163 2024-01-11T17:48:14.348 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 83/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 84/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 86/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 88/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 89/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 90/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 91/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 92/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 93/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 94/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 95/163 2024-01-11T17:48:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 96/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.1-8.el8.noarch 97/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpq-13.11-1.el8.x86_64 98/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : jq-1.6-7.el8.x86_64 99/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 100/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 101/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 102/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 103/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 104/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 106/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 107/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 108/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/163 2024-01-11T17:48:14.350 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 110/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 111/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 112/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 113/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 114/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 115/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 116/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 117/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 118/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 119/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 120/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 121/163 2024-01-11T17:48:14.351 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 122/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 123/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 124/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 125/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 126/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 127/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 128/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 129/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 130/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 131/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.0.0-446.gc27531ad.el8.x86_64 132/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 133/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 134/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 135/163 2024-01-11T17:48:14.352 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 136/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 137/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 138/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 139/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 140/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 141/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 142/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 143/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 144/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 145/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 146/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 147/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 148/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 149/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 150/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 151/163 2024-01-11T17:48:14.353 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 152/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 153/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 154/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 155/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 156/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 158/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 159/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 160/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 161/163 2024-01-11T17:48:14.354 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 162/163 2024-01-11T17:48:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 138/163 2024-01-11T17:48:14.533 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 149/163 2024-01-11T17:48:14.678 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 150/163 2024-01-11T17:48:14.678 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-11T17:48:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 139/163 2024-01-11T17:48:14.943 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 140/163 2024-01-11T17:48:14.943 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-11T17:48:14.943 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-11T17:48:15.194 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-11T17:48:15.195 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-11T17:48:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 141/163 2024-01-11T17:48:15.464 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-11T17:48:15.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 142/163 2024-01-11T17:48:15.633 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 154/163 2024-01-11T17:48:15.820 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 155/163 2024-01-11T17:48:15.863 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 143/163 2024-01-11T17:48:16.075 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 156/163 2024-01-11T17:48:16.075 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:16.090 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/163 2024-01-11T17:48:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/163 2024-01-11T17:48:16.293 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:16.293 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 158/163 2024-01-11T17:48:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/163 2024-01-11T17:48:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/163 2024-01-11T17:48:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 158/163 2024-01-11T17:48:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-11T17:48:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 146/163 2024-01-11T17:48:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 146/163 2024-01-11T17:48:16.816 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/163 2024-01-11T17:48:16.959 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-11T17:48:16.959 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-11T17:48:16.992 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 148/163 2024-01-11T17:48:17.180 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 149/163 2024-01-11T17:48:17.209 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-11T17:48:17.209 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-11T17:48:17.329 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 150/163 2024-01-11T17:48:17.330 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-11T17:48:17.459 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-11T17:48:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 162/163 2024-01-11T17:48:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-11T17:48:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-11T17:48:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-11T17:48:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-11T17:48:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-11T17:48:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 154/163 2024-01-11T17:48:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 155/163 2024-01-11T17:48:18.578 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:18.578 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-446.gc27531ad.el8.x86_64 1/163 2024-01-11T17:48:18.578 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 2/163 2024-01-11T17:48:18.578 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 3/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 4/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-446.gc27531 5/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 6/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 7/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 8/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:48:18.579 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 12/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_6 14/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 15/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 16/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 17/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 18/163 2024-01-11T17:48:18.581 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 19/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8 20/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x 21/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 22/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 23/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 24/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 25/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 26/163 2024-01-11T17:48:18.582 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 27/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 28/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.e 29/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 30/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 31/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 32/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8 33/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 34/163 2024-01-11T17:48:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el 35/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 36/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-446.gc27531ad.el8.noarch 37/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 38/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/163 2024-01-11T17:48:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/163 2024-01-11T17:48:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/163 2024-01-11T17:48:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 93/163 2024-01-11T17:48:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 94/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 95/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 96/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 98/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 99/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.3.4-12.el8.x86_64 100/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 101/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 102/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 103/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 104/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 106/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.0.4-10.el8.x86_64 107/163 2024-01-11T17:48:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.1-9.el8.x86_64 108/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 109/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 110/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 111/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 112/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 113/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 114/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 115/163 2024-01-11T17:48:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 116/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 117/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 118/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 119/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 120/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 121/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 122/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 123/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.1-8.el8.noarch 124/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpq-13.11-1.el8.x86_64 125/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-7.el8.x86_64 126/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 127/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 128/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 129/163 2024-01-11T17:48:18.602 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 130/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 131/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-446.gc27531ad.el8.x86_64 132/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 133/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 134/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 135/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 136/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 137/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 138/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 139/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 140/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 141/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 142/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 143/163 2024-01-11T17:48:18.603 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 144/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 145/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 146/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 147/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 148/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 149/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 150/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 151/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 152/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 153/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 154/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 155/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 156/163 2024-01-11T17:48:18.604 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:18.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 158/163 2024-01-11T17:48:18.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 159/163 2024-01-11T17:48:18.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 160/163 2024-01-11T17:48:18.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 161/163 2024-01-11T17:48:18.605 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 162/163 2024-01-11T17:48:18.768 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 156/163 2024-01-11T17:48:18.768 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 158/163 2024-01-11T17:48:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 158/163 2024-01-11T17:48:19.327 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-11T17:48:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-11T17:48:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-11T17:48:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-11T17:48:19.852 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-11T17:48:20.111 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-11T17:48:21.728 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 162/163 2024-01-11T17:48:21.728 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:22.327 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:22.328 INFO:teuthology.orchestra.run.smithi007.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-11T17:48:22.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-446.gc27531ad.el8.x86_64 1/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 2/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 3/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 4/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-446.gc27531 5/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 6/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 7/163 2024-01-11T17:48:23.061 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 8/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 9/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 10/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 11/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 12/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 13/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_6 14/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 15/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 16/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 17/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_ 18/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 19/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8 20/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x 21/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 22/163 2024-01-11T17:48:23.062 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 23/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 24/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 25/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 26/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 27/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 28/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.e 29/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noar 30/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.no 31/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-446.gc275 32/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8 33/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 34/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el 35/163 2024-01-11T17:48:23.063 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 36/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-446.gc27531ad.el8.noarch 37/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 38/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/163 2024-01-11T17:48:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/163 2024-01-11T17:48:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/163 2024-01-11T17:48:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 93/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 94/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 95/163 2024-01-11T17:48:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 96/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 98/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 99/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/163 2024-01-11T17:48:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 111/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 112/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 114/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 115/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 116/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 117/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 118/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 119/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 120/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 121/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 122/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.1-8.el8.noarch 123/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpq-13.11-1.el8.x86_64 124/163 2024-01-11T17:48:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-7.el8.x86_64 125/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 126/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 127/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 128/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 129/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 130/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.3.4-12.el8.x86_64 131/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-446.gc27531ad.el8.x86_64 132/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 133/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 134/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 135/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 136/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 137/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 138/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 139/163 2024-01-11T17:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 140/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 141/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 142/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 143/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 144/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 145/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 146/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 147/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 148/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 149/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 150/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 151/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 152/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 153/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 154/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 155/163 2024-01-11T17:48:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 156/163 2024-01-11T17:48:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-11T17:48:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 158/163 2024-01-11T17:48:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 159/163 2024-01-11T17:48:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 160/163 2024-01-11T17:48:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 161/163 2024-01-11T17:48:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 162/163 2024-01-11T17:48:26.194 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:26.194 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.274 INFO:teuthology.orchestra.run.smithi038.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: xfsdump-3.1.8-6.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.1-8.el8.noarch 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.0.4-10.el8.x86_64 2024-01-11T17:48:27.275 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.276 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.1-9.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: jq-1.6-7.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-11T17:48:27.277 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libpq-13.11-1.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-11T17:48:27.278 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.3.4-12.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.5.0-1.el8.noarch 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:27.279 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-11T17:48:27.280 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-2.6-1.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-11T17:48:27.281 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-11T17:48:27.282 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-43.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-11T17:48:27.283 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-11T17:48:27.284 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-11T17:48:27.284 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:48:27.284 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-01-11T17:48:29.048 DEBUG:teuthology.parallel:result is None 2024-01-11T17:48:34.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:34.615 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.275 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.8-6.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.1-8.el8.noarch 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.0.4-10.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.276 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.277 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.1-9.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-7.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.278 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libpq-13.11-1.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-11T17:48:35.279 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.3.4-12.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.5.0-1.el8.noarch 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-11T17:48:35.280 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-11T17:48:35.281 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-11T17:48:35.282 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-11T17:48:35.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-11T17:48:35.284 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-43.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-01-11T17:48:35.285 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-01-11T17:48:35.515 DEBUG:teuthology.parallel:result is None 2024-01-11T17:48:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-11T17:48:36.532 INFO:teuthology.orchestra.run.smithi007.stdout:Installed products updated. 2024-01-11T17:48:37.184 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:48:37.184 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-01-11T17:48:37.184 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-11T17:48:37.184 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-11T17:48:37.184 INFO:teuthology.orchestra.run.smithi007.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.8-6.el8.x86_64 2024-01-11T17:48:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.1-8.el8.noarch 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.0.4-10.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.186 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-446.gc27531ad.el8.noarch 2024-01-11T17:48:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.1-9.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-7.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libpq-13.11-1.el8.x86_64 2024-01-11T17:48:37.188 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.3.4-12.el8.x86_64 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.5.0-1.el8.noarch 2024-01-11T17:48:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-11T17:48:37.190 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-11T17:48:37.191 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-2.6-1.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-11T17:48:37.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-11T17:48:37.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-446.gc27531ad.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-43.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-11T17:48:37.194 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-11T17:48:37.195 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-11T17:48:37.195 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:48:37.195 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-11T17:48:37.593 DEBUG:teuthology.parallel:result is None 2024-01-11T17:48:37.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:48:37.732 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-11T17:48:37.787 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-446.gc27531ad.el8 2024-01-11T17:48:37.787 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-446.gc27531ad.el8 2024-01-11T17:48:37.787 INFO:teuthology.task.install:The correct ceph version 19.0.0-446.gc27531ad is installed. 2024-01-11T17:48:37.789 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:48:38.095 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-11T17:48:38.126 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-446.gc27531ad.el8 2024-01-11T17:48:38.126 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-446.gc27531ad.el8 2024-01-11T17:48:38.126 INFO:teuthology.task.install:The correct ceph version 19.0.0-446.gc27531ad is installed. 2024-01-11T17:48:38.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:48:38.468 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-11T17:48:38.499 INFO:teuthology.orchestra.run.smithi038.stdout:19.0.0-446.gc27531ad.el8 2024-01-11T17:48:38.499 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-446.gc27531ad.el8 2024-01-11T17:48:38.500 INFO:teuthology.task.install:The correct ceph version 19.0.0-446.gc27531ad is installed. 2024-01-11T17:48:38.501 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-11T17:48:38.502 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:48:38.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-11T17:48:38.600 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:48:38.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-11T17:48:38.652 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:48:38.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-11T17:48:38.707 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-11T17:48:38.707 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:48:38.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-01-11T17:48:38.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-11T17:48:38.890 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:48:38.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-01-11T17:48:38.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-11T17:48:39.001 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:48:39.001 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-01-11T17:48:39.036 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-11T17:48:39.109 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-11T17:48:39.110 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:48:39.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-11T17:48:39.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-11T17:48:39.293 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:48:39.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-11T17:48:39.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-11T17:48:39.402 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:48:39.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-11T17:48:39.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-11T17:48:39.510 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-11T17:48:39.511 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:48:39.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-01-11T17:48:39.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-11T17:48:39.690 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:48:39.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-01-11T17:48:39.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-11T17:48:39.799 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:48:39.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-01-11T17:48:39.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-11T17:48:39.907 INFO:teuthology.run_tasks:Running task cephadm... 2024-01-11T17:48:39.996 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-01-11T17:48:39.996 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:48:39.996 INFO:tasks.cephadm:Cluster fsid is a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:48:39.996 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-01-11T17:48:39.997 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.16', 'mon.c': '172.21.15.38'} 2024-01-11T17:48:39.997 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-01-11T17:48:39.997 INFO:tasks.cephadm:First mgr is x 2024-01-11T17:48:39.997 INFO:tasks.cephadm:Normalizing hostnames... 2024-01-11T17:48:39.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-01-11T17:48:40.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-01-11T17:48:40.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-01-11T17:48:40.139 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-01-11T17:48:40.140 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:48:40.278 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c27531add9155519f390b228e498f0a2c5eae117', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/75677/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-446-gc27531ad', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-446.gc27531ad'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-01-10 11:27:48.709749', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240103.072409-1', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-01-11T17:48:40.415 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20240103.072409-1, sha1 c27531add9155519f390b228e498f0a2c5eae117 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=c27531add9155519f390b228e498f0a2c5eae117 2024-01-11T17:48:40.417 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/x86_64/flavors/default/cephadm 2024-01-11T17:48:40.417 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:41.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2280198 Jan 11 17:48 /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:41.089 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:41.542 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2280198 Jan 11 17:48 /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:41.542 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240103.072409-1/c27531add9155519f390b228e498f0a2c5eae117/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:42.172 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2280198 Jan 11 17:48 /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:42.172 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:42.224 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:42.247 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-11T17:48:42.286 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 on all hosts... 2024-01-11T17:48:42.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 pull 2024-01-11T17:48:42.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 pull 2024-01-11T17:48:42.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 pull 2024-01-11T17:48:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117... 2024-01-11T17:48:44.177 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117... 2024-01-11T17:48:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117... 2024-01-11T17:49:15.547 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:49:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev)", 2024-01-11T17:49:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "a1bd98dcc13bc78e92181cf4d645397426ea2b17638630dfc3aaab34745d9e12", 2024-01-11T17:49:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-01-11T17:49:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cdf31b445914a8f1b1613bfa7a222ff5a414ea2566590801dfeb8f3870cce8e3" 2024-01-11T17:49:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T17:49:15.548 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:49:16.236 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-01-11T17:49:16.236 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev)", 2024-01-11T17:49:16.236 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "a1bd98dcc13bc78e92181cf4d645397426ea2b17638630dfc3aaab34745d9e12", 2024-01-11T17:49:16.237 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-01-11T17:49:16.237 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cdf31b445914a8f1b1613bfa7a222ff5a414ea2566590801dfeb8f3870cce8e3" 2024-01-11T17:49:16.237 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-01-11T17:49:16.237 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev)", 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "a1bd98dcc13bc78e92181cf4d645397426ea2b17638630dfc3aaab34745d9e12", 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cdf31b445914a8f1b1613bfa7a222ff5a414ea2566590801dfeb8f3870cce8e3" 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-01-11T17:49:16.738 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-01-11T17:49:16.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-01-11T17:49:16.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-01-11T17:49:16.896 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-01-11T17:49:16.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-01-11T17:49:17.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-01-11T17:49:17.042 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-01-11T17:49:17.078 INFO:tasks.cephadm:Writing seed config... 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [client] debug client = 20 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-01-11T17:49:17.079 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-01-11T17:49:17.080 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-01-11T17:49:17.081 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-01-11T17:49:17.082 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-01-11T17:49:17.083 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:49:17.083 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-01-11T17:49:17.127 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a76d0c8a-b0a9-11ee-95ab-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-01-11T17:49:17.127 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a.service 2024-01-11T17:49:17.172 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x.service 2024-01-11T17:49:17.216 INFO:tasks.cephadm:Bootstrapping... 2024-01-11T17:49:17.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 -v bootstrap --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-01-11T17:49:17.241 INFO:journalctl@ceph.mon.a.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:49:17.284 INFO:journalctl@ceph.mgr.x.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:49:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-01-11T17:49:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117', '-v', 'bootstrap', '--fsid', 'a76d0c8a-b0a9-11ee-95ab-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-01-11T17:49:17.546 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.6.1 2024-01-11T17:49:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-01-11T17:49:17.547 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-01-11T17:49:17.596 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.6.1 2024-01-11T17:49:17.596 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-01-11T17:49:17.596 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-01-11T17:49:17.610 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-01-11T17:49:17.610 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-01-11T17:49:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-01-11T17:49:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-01-11T17:49:17.638 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-01-11T17:49:17.651 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-01-11T17:49:17.652 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-01-11T17:49:17.652 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-01-11T17:49:17.699 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.6.1 2024-01-11T17:49:17.699 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 4.6.1 is present 2024-01-11T17:49:17.699 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-01-11T17:49:17.699 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-01-11T17:49:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-01-11T17:49:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-01-11T17:49:17.726 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-01-11T17:49:17.726 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-01-11T17:49:17.740 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-01-11T17:49:17.753 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-01-11T17:49:17.753 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-01-11T17:49:17.753 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-01-11T17:49:17.753 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:17.753 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 140096469673072 on /run/cephadm/a76d0c8a-b0a9-11ee-95ab-87774f69a715.lock 2024-01-11T17:49:17.754 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 140096469673072 acquired on /run/cephadm/a76d0c8a-b0a9-11ee-95ab-87774f69a715.lock 2024-01-11T17:49:17.754 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-01-11T17:49:17.755 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-01-11T17:49:17.755 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-01-11T17:49:17.763 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-01-11T17:49:17.763 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-01-11T17:49:17.771 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-01-11T17:49:17.771 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-01-11T17:49:17.771 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2024-01-11T17:49:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-01-11T17:49:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-01-11T17:49:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-01-11T17:49:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-01-11T17:49:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T17:49:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-01-11T17:49:17.779 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-01-11T17:49:17.779 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-01-11T17:49:17.779 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-01-11T17:49:17.780 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-01-11T17:49:17.781 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117... 2024-01-11T17:49:18.166 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout a1bd98dcc13bc78e92181cf4d645397426ea2b17638630dfc3aaab34745d9e12 2024-01-11T17:49:18.166 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117... 2024-01-11T17:49:18.167 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-01-11T17:49:18.167 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:01973abcef9c54087b1f6c21ab0e557127409eb7bdbbfb29c9ee042cb86a4004 2024-01-11T17:49:18.167 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:6c507150f7e9f028ef978d633e702dab944bbda82bfbc46f6307495cf6dd9fc7 2024-01-11T17:49:18.167 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:a1bd98dcc13bc78e92181cf4d645397426ea2b17638630dfc3aaab34745d9e12 2024-01-11T17:49:18.167 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-01-11T17:49:18.865 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev) 2024-01-11T17:49:18.865 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev) 2024-01-11T17:49:18.865 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-01-11T17:49:19.410 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-01-11T17:49:19.411 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-01-11T17:49:19.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAfKqBllgdTKRAAYb1rRU5MTwGbMdIGGkFJ+A== 2024-01-11T17:49:20.565 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAgKqBlnv/7DhAAehu2//5DRV8RPt55SpIgFA== 2024-01-11T17:49:21.152 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAgKqBlm+lIMhAAEcZVlhcw/VFwTT/wgXINmA== 2024-01-11T17:49:21.153 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-01-11T17:49:21.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:49:21.715 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-01-11T17:49:22.318 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-01-11T17:49:22.574 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-01-11T17:49:22.791 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-01-11T17:49:23.116 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715.target → /etc/systemd/system/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715.target. 2024-01-11T17:49:23.116 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715.target → /etc/systemd/system/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715.target. 2024-01-11T17:49:23.519 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a 2024-01-11T17:49:23.520 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a.service: Unit ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a.service not loaded. 2024-01-11T17:49:23.727 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715.target.wants/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a.service → /etc/systemd/system/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@.service. 2024-01-11T17:49:23.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:23 smithi007 systemd[1]: Starting Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:49:24.240 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-11T17:49:24.240 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-01-11T17:49:24.254 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:24 smithi007 bash[32835]: 07f5599e18f9e27d9ab27935f3fd25a49a54f19ee32a48fff6a6c4bca1c09fcb 2024-01-11T17:49:24.254 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:24 smithi007 systemd[1]: Started Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:49:24.254 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:24 smithi007 ceph-mon[32873]: mkfs a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-11T17:49:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-01-11T17:49:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-01-11T17:49:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-01-11T17:49:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-01-11T17:49:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-01-11T17:49:24.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:24 smithi007 ceph-mon[32873]: mon.a is new leader, mons a in quorum (ranks 0) 2024-01-11T17:49:25.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-01-11T17:49:25.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:25.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-01-11T17:49:25.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:25.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.527397s) 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.686+0000 7fe7647d0700 1 Processor -- start 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.686+0000 7fe7647d0700 1 -- start start 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.687+0000 7fe7647d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c1072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:25.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.687+0000 7fe7647d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1077f0 con 0x7fe75c106e90 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.688+0000 7fe76256c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c1072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.688+0000 7fe76256c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c1072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34704/0 (socket says 172.21.15.7:34704) 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.688+0000 7fe76256c700 1 -- 172.21.15.7:0/2610423551 learned_addr learned my addr 172.21.15.7:0/2610423551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.688+0000 7fe76256c700 1 -- 172.21.15.7:0/2610423551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c107930 con 0x7fe75c106e90 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.688+0000 7fe76256c700 1 --2- 172.21.15.7:0/2610423551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c1072b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098a0 tx=0x7fe758009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34bfa0a94ed61d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.688+0000 7fe76156a700 1 -- 172.21.15.7:0/2610423551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe758004160 con 0x7fe75c106e90 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe76156a700 1 -- 172.21.15.7:0/2610423551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7580042c0 con 0x7fe75c106e90 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe76156a700 1 -- 172.21.15.7:0/2610423551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7580045a0 con 0x7fe75c106e90 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 -- 172.21.15.7:0/2610423551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 msgr2=0x7fe75c1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 --2- 172.21.15.7:0/2610423551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c1072b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098a0 tx=0x7fe758009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 -- 172.21.15.7:0/2610423551 shutdown_connections 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 --2- 172.21.15.7:0/2610423551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c1072b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 -- 172.21.15.7:0/2610423551 >> 172.21.15.7:0/2610423551 conn(0x7fe75c102470 msgr2=0x7fe75c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 -- 172.21.15.7:0/2610423551 shutdown_connections 2024-01-11T17:49:25.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.689+0000 7fe7647d0700 1 -- 172.21.15.7:0/2610423551 wait complete. 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.690+0000 7fe7647d0700 1 Processor -- start 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.690+0000 7fe7647d0700 1 -- start start 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.691+0000 7fe7647d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.691+0000 7fe7647d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c19ca70 con 0x7fe75c106e90 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.691+0000 7fe76256c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.691+0000 7fe76256c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34720/0 (socket says 172.21.15.7:34720) 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.691+0000 7fe76256c700 1 -- 172.21.15.7:0/2174465001 learned_addr learned my addr 172.21.15.7:0/2174465001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.691+0000 7fe76256c700 1 -- 172.21.15.7:0/2174465001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758009550 con 0x7fe75c106e90 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe76256c700 1 --2- 172.21.15.7:0/2174465001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c19c530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe75c107b20 tx=0x7fe758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe758011630 con 0x7fe75c106e90 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe758011790 con 0x7fe75c106e90 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c19cc70 con 0x7fe75c106e90 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe758011a30 con 0x7fe75c106e90 2024-01-11T17:49:25.052 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c19d110 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fe758021070 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.692+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe75801b440 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.693+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c108170 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.696+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fe758011b90 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.774+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe75c19fc40 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.774+0000 7fe7537fe700 1 -- 172.21.15.7:0/2174465001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe758011d30 con 0x7fe75c106e90 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 msgr2=0x7fe75c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 --2- 172.21.15.7:0/2174465001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c19c530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe75c107b20 tx=0x7fe758004750 comp rx=0 tx=0).stop 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 shutdown_connections 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 --2- 172.21.15.7:0/2174465001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c106e90 0x7fe75c19c530 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 >> 172.21.15.7:0/2174465001 conn(0x7fe75c102470 msgr2=0x7fe75c07a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 shutdown_connections 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:24.775+0000 7fe7647d0700 1 -- 172.21.15.7:0/2174465001 wait complete. 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-01-11T17:49:25.053 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-01-11T17:49:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:25 smithi007 ceph-mon[32873]: mon.a is new leader, mons a in quorum (ranks 0) 2024-01-11T17:49:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:25 smithi007 ceph-mon[32873]: monmap e1: 1 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:49:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:25 smithi007 ceph-mon[32873]: fsmap 2024-01-11T17:49:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:25 smithi007 ceph-mon[32873]: osdmap e1: 0 total, 0 up, 0 in 2024-01-11T17:49:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:25 smithi007 ceph-mon[32873]: mgrmap e1: no daemons active 2024-01-11T17:49:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:25 smithi007 ceph-mon[32873]: from='client.? 172.21.15.7:0/2174465001' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-01-11T17:49:25.931 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-01-11T17:49:25.932 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.541+0000 7fa8488f8700 1 Processor -- start 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.541+0000 7fa8488f8700 1 -- start start 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.541+0000 7fa8488f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa8441072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.541+0000 7fa8488f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8441077f0 con 0x7fa844106e90 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.542+0000 7fa84259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa8441072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.542+0000 7fa84259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa8441072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34722/0 (socket says 172.21.15.7:34722) 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.542+0000 7fa84259c700 1 -- 172.21.15.7:0/3618339679 learned_addr learned my addr 172.21.15.7:0/3618339679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.542+0000 7fa84259c700 1 -- 172.21.15.7:0/3618339679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844107930 con 0x7fa844106e90 2024-01-11T17:49:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.542+0000 7fa84259c700 1 --2- 172.21.15.7:0/3618339679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa8441072b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa8340098a0 tx=0x7fa834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318f6785356db194 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa84159a700 1 -- 172.21.15.7:0/3618339679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa834004160 con 0x7fa844106e90 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa84159a700 1 -- 172.21.15.7:0/3618339679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8340042c0 con 0x7fa844106e90 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa84159a700 1 -- 172.21.15.7:0/3618339679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8340045a0 con 0x7fa844106e90 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 -- 172.21.15.7:0/3618339679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 msgr2=0x7fa8441072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 --2- 172.21.15.7:0/3618339679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa8441072b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa8340098a0 tx=0x7fa834009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 -- 172.21.15.7:0/3618339679 shutdown_connections 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 --2- 172.21.15.7:0/3618339679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa8441072b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 -- 172.21.15.7:0/3618339679 >> 172.21.15.7:0/3618339679 conn(0x7fa844102470 msgr2=0x7fa844104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 -- 172.21.15.7:0/3618339679 shutdown_connections 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.543+0000 7fa8488f8700 1 -- 172.21.15.7:0/3618339679 wait complete. 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.544+0000 7fa8488f8700 1 Processor -- start 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa8488f8700 1 -- start start 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa8488f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa84419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa8488f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84419ca90 con 0x7fa844106e90 2024-01-11T17:49:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa84259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa84419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa84259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa84419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34728/0 (socket says 172.21.15.7:34728) 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa84259c700 1 -- 172.21.15.7:0/4116780978 learned_addr learned my addr 172.21.15.7:0/4116780978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.545+0000 7fa84259c700 1 -- 172.21.15.7:0/4116780978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009550 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa84259c700 1 --2- 172.21.15.7:0/4116780978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa84419c550 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa844107b20 tx=0x7fa834004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa834011630 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834011790 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa834011a30 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84419cc90 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84419d130 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fa834021070 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.546+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa83401b440 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.547+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa844108170 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.550+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fa834011b90 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.624+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa844108300 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.628+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fa834011de0 con 0x7fa844106e90 2024-01-11T17:49:25.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.628+0000 7fa8337fe700 1 -- 172.21.15.7:0/4116780978 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa834035590 con 0x7fa844106e90 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 msgr2=0x7fa84419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 --2- 172.21.15.7:0/4116780978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa84419c550 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa844107b20 tx=0x7fa834004750 comp rx=0 tx=0).stop 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 shutdown_connections 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 --2- 172.21.15.7:0/4116780978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa844106e90 0x7fa84419c550 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 >> 172.21.15.7:0/4116780978 conn(0x7fa844102470 msgr2=0x7fa84407a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 shutdown_connections 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:25.629+0000 7fa8488f8700 1 -- 172.21.15.7:0/4116780978 wait complete. 2024-01-11T17:49:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-01-11T17:49:26.559 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:26 smithi007 ceph-mon[32873]: from='client.? 172.21.15.7:0/4116780978' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-01-11T17:49:26.559 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:26 smithi007 ceph-mon[32873]: from='client.? 172.21.15.7:0/4116780978' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-01-11T17:49:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.350+0000 7f60b39d8700 1 Processor -- start 2024-01-11T17:49:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.351+0000 7f60b39d8700 1 -- start start 2024-01-11T17:49:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.351+0000 7f60b39d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac1072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.351+0000 7f60b39d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac1077f0 con 0x7f60ac106e90 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.351+0000 7f60b1774700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac1072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.351+0000 7f60b1774700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac1072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34736/0 (socket says 172.21.15.7:34736) 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.351+0000 7f60b1774700 1 -- 172.21.15.7:0/993103341 learned_addr learned my addr 172.21.15.7:0/993103341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.352+0000 7f60b1774700 1 -- 172.21.15.7:0/993103341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac107930 con 0x7f60ac106e90 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.352+0000 7f60b1774700 1 --2- 172.21.15.7:0/993103341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac1072b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60a80098a0 tx=0x7f60a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f180d2d8bb31d51 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.352+0000 7f60a3fff700 1 -- 172.21.15.7:0/993103341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60a8004160 con 0x7f60ac106e90 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.352+0000 7f60a3fff700 1 -- 172.21.15.7:0/993103341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60a80042c0 con 0x7f60ac106e90 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.352+0000 7f60a3fff700 1 -- 172.21.15.7:0/993103341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60a8011a00 con 0x7f60ac106e90 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 -- 172.21.15.7:0/993103341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 msgr2=0x7f60ac1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 --2- 172.21.15.7:0/993103341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac1072b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60a80098a0 tx=0x7f60a8009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 -- 172.21.15.7:0/993103341 shutdown_connections 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 --2- 172.21.15.7:0/993103341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac1072b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 -- 172.21.15.7:0/993103341 >> 172.21.15.7:0/993103341 conn(0x7f60ac102470 msgr2=0x7f60ac104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 -- 172.21.15.7:0/993103341 shutdown_connections 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.353+0000 7f60b39d8700 1 -- 172.21.15.7:0/993103341 wait complete. 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.354+0000 7f60b39d8700 1 Processor -- start 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.354+0000 7f60b39d8700 1 -- start start 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.354+0000 7f60b39d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.354+0000 7f60b39d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac19b120 con 0x7f60ac106e90 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.355+0000 7f60b1774700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.355+0000 7f60b1774700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34752/0 (socket says 172.21.15.7:34752) 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.355+0000 7f60b1774700 1 -- 172.21.15.7:0/940723061 learned_addr learned my addr 172.21.15.7:0/940723061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.355+0000 7f60b1774700 1 -- 172.21.15.7:0/940723061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8009550 con 0x7f60ac106e90 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.355+0000 7f60b1774700 1 --2- 172.21.15.7:0/940723061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac19abe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009130 tx=0x7f60a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60a80038b0 con 0x7f60ac106e90 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60a8003a10 con 0x7f60ac106e90 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60a8019550 con 0x7f60ac106e90 2024-01-11T17:49:26.715 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac19b320 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac19b7c0 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f60a80197c0 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.356+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f60a80243f0 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.357+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac062c40 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.360+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f60a8003b80 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.434+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f60ac19e360 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.435+0000 7f60a27fc700 1 -- 172.21.15.7:0/940723061 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f60a801a960 con 0x7f60ac106e90 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 msgr2=0x7f60ac19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 --2- 172.21.15.7:0/940723061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac19abe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009130 tx=0x7f60a8011ad0 comp rx=0 tx=0).stop 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 shutdown_connections 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 --2- 172.21.15.7:0/940723061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60ac106e90 0x7f60ac19abe0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 >> 172.21.15.7:0/940723061 conn(0x7f60ac102470 msgr2=0x7f60ac102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 shutdown_connections 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:26.436+0000 7f60b39d8700 1 -- 172.21.15.7:0/940723061 wait complete. 2024-01-11T17:49:26.716 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-01-11T17:49:26.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:26 smithi007 systemd[1]: Stopping Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:49:27.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:27 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[32850]: 2024-01-11T17:49:27.025+0000 7fca041e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T17:49:27.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:27 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[32850]: 2024-01-11T17:49:27.025+0000 7fca041e0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-01-11T17:49:27.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:27 smithi007 bash[33203]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a 2024-01-11T17:49:28.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:27 smithi007 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a.service: Succeeded. 2024-01-11T17:49:28.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:27 smithi007 systemd[1]: Stopped Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:49:28.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:27 smithi007 systemd[1]: Starting Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:49:29.169 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-01-11T17:49:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 bash[33323]: a03a168ceed1f35fafcd1e2378bc59bdc6a0b401538c2baa9f6569488508d2d0 2024-01-11T17:49:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 systemd[1]: Started Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:49:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: set uid:gid to 167:167 (ceph:ceph) 2024-01-11T17:49:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev), process ceph-mon, pid 7 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: pidfile_write: ignore empty --pid-file 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: load: jerasure load: lrc 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: RocksDB version: 7.9.2 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Git sha 0 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Compile date 2024-01-10 10:08:18 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: DB SUMMARY 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: DB Session ID: 3YUZLXUZL628CC5VBQFX 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: CURRENT file: CURRENT 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: IDENTITY file: IDENTITY 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97913 ; 2024-01-11T17:49:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.error_if_exists: 0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.create_if_missing: 0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.paranoid_checks: 1 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.flush_verify_memtable_count: 1 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.env: 0x55f56e4a1a60 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.fs: PosixFileSystem 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.info_log: 0x55f56f4757a0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_file_opening_threads: 16 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.statistics: (nil) 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.use_fsync: 0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_log_file_size: 0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.log_file_time_to_roll: 0 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.keep_log_file_num: 1000 2024-01-11T17:49:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.recycle_log_file_num: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.allow_fallocate: 1 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.allow_mmap_reads: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.allow_mmap_writes: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.use_direct_reads: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.create_missing_column_families: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.db_log_dir: 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.wal_dir: 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.table_cache_numshardbits: 6 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.WAL_ttl_seconds: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.WAL_size_limit_MB: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.is_fd_close_on_exec: 1 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.advise_random_on_open: 1 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.db_write_buffer_size: 0 2024-01-11T17:49:29.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.write_buffer_manager: 0x55f56f42b4a0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.use_adaptive_mutex: 0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.rate_limiter: (nil) 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.wal_recovery_mode: 2 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.enable_thread_tracking: 0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.enable_pipelined_write: 0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.unordered_write: 0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.row_cache: None 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.wal_filter: None 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-01-11T17:49:29.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.allow_ingest_behind: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.two_write_queues: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.manual_wal_flush: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.wal_compression: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.atomic_flush: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.persist_stats_to_disk: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.write_dbid_to_manifest: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.log_readahead_size: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.best_efforts_recovery: 0 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-01-11T17:49:29.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.allow_data_in_errors: 0 2024-01-11T17:49:29.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.db_host_id: __hostname__ 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.enforce_single_del_contracts: true 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_background_jobs: 2 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_background_compactions: -1 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_subcompactions: 1 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.delayed_write_rate : 16777216 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_total_wal_size: 0 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.stats_dump_period_sec: 600 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.stats_persist_period_sec: 600 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_open_files: -1 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bytes_per_sync: 0 2024-01-11T17:49:29.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.wal_bytes_per_sync: 0 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.strict_bytes_per_sync: 0 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_readahead_size: 0 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_background_flushes: -1 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Compression algorithms supported: 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kZSTD supported: 0 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kXpressCompression supported: 0 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kLZ4HCCompression supported: 1 2024-01-11T17:49:29.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kZlibCompression supported: 1 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kSnappyCompression supported: 1 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kLZ4Compression supported: 1 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: kBZip2Compression supported: 0 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Fast CRC32 supported: Supported on x86 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: DMutex implementation: pthread_mutex_t 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.merge_operator: 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_filter: None 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_filter_factory: None 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.sst_partitioner_factory: None 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.memtable_factory: SkipListFactory 2024-01-11T17:49:29.580 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.table_factory: BlockBasedTable 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f56f4758c0) 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x55f56f4cb090 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-01-11T17:49:29.581 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-01-11T17:49:29.582 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.write_buffer_size: 33554432 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_write_buffer_number: 2 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression: NoCompression 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression: Disabled 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.prefix_extractor: nullptr 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.num_levels: 7 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-01-11T17:49:29.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.window_bits: -14 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.level: 32767 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.strategy: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-01-11T17:49:29.584 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.enabled: false 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.target_file_size_base: 67108864 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.target_file_size_multiplier: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-01-11T17:49:29.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.arena_block_size: 1048576 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.disable_auto_compactions: 0 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-01-11T17:49:29.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.table_properties_collectors: 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.inplace_update_support: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.inplace_update_num_locks: 10000 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.memtable_huge_page_size: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.bloom_locality: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.max_successive_merges: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.optimize_filters_for_hits: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.paranoid_file_checks: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.force_consistency_checks: 1 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.report_bg_io_stats: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.ttl: 2592000 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.periodic_compaction_seconds: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.enable_blob_files: false 2024-01-11T17:49:29.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.min_blob_size: 0 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.blob_file_size: 268435456 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.blob_compression_type: NoCompression 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.enable_blob_garbage_collection: false 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.blob_file_starting_level: 0 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 52b721b4-471f-4504-9457-41a9cb0063ca 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995369189042, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-01-11T17:49:29.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995369192097, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 307, "table_properties": {"data_size": 91293, "index_size": 271, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13606, "raw_average_key_size": 50, "raw_value_size": 83905, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 270, "num_filter_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1704995369, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "52b721b4-471f-4504-9457-41a9cb0063ca", "db_session_id": "3YUZLXUZL628CC5VBQFX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995369192292, "job": 1, "event": "recovery_finished"} 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f56f566000 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: DB pointer 0x55f56f554000 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-11T17:49:29.589 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 92.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 92.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 46.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-01-11T17:49:29.590 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 9.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 9.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x55f56f4cb090#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???) e1 preinit fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:29.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).mds e1 new map 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).mds e1 print_map 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mon.a is new leader, mons a in quorum (ranks 0) 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: monmap e1: 1 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: fsmap 2024-01-11T17:49:29.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: osdmap e1: 0 total, 0 up, 0 in 2024-01-11T17:49:29.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:29 smithi007 ceph-mon[33361]: mgrmap e1: no daemons active 2024-01-11T17:49:29.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.589+0000 7f6e3b4d8700 1 Processor -- start 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.589+0000 7f6e3b4d8700 1 -- start start 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.589+0000 7f6e3b4d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3407a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.589+0000 7f6e3b4d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3407a780 con 0x7f6e3407bd50 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.590+0000 7f6e39274700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3407a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.590+0000 7f6e39274700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3407a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34760/0 (socket says 172.21.15.7:34760) 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.590+0000 7f6e39274700 1 -- 172.21.15.7:0/3619127331 learned_addr learned my addr 172.21.15.7:0/3619127331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.591+0000 7f6e39274700 1 -- 172.21.15.7:0/3619127331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3407a8c0 con 0x7f6e3407bd50 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.591+0000 7f6e39274700 1 --2- 172.21.15.7:0/3619127331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3407a1b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e300098a0 tx=0x7f6e30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6a94e76d0240a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.591+0000 7f6e2bfff700 1 -- 172.21.15.7:0/3619127331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e30004160 con 0x7f6e3407bd50 2024-01-11T17:49:29.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.591+0000 7f6e2bfff700 1 -- 172.21.15.7:0/3619127331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e300042c0 con 0x7f6e3407bd50 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.591+0000 7f6e2bfff700 1 -- 172.21.15.7:0/3619127331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e30011a00 con 0x7f6e3407bd50 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/3619127331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 msgr2=0x7f6e3407a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 --2- 172.21.15.7:0/3619127331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3407a1b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e300098a0 tx=0x7f6e30009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/3619127331 shutdown_connections 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 --2- 172.21.15.7:0/3619127331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3407a1b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/3619127331 >> 172.21.15.7:0/3619127331 conn(0x7f6e34102460 msgr2=0x7f6e34104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/3619127331 shutdown_connections 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.592+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/3619127331 wait complete. 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.593+0000 7f6e3b4d8700 1 Processor -- start 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.594+0000 7f6e3b4d8700 1 -- start start 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.594+0000 7f6e3b4d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.594+0000 7f6e3b4d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3419d240 con 0x7f6e3407bd50 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.594+0000 7f6e39274700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e39274700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34774/0 (socket says 172.21.15.7:34774) 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e39274700 1 -- 172.21.15.7:0/1489166363 learned_addr learned my addr 172.21.15.7:0/1489166363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e39274700 1 -- 172.21.15.7:0/1489166363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30009550 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e39274700 1 --2- 172.21.15.7:0/1489166363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3419cd00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009130 tx=0x7f6e30011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e30003b70 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e30003cd0 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3419d440 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.595+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e300197d0 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.596+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3419d8e0 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.596+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f6e30019a10 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.596+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e3000b7b0 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.597+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3410d1f0 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.600+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f6e3002a060 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.674+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6e3410d400 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.676+0000 7f6e2a7fc700 1 -- 172.21.15.7:0/1489166363 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6e30019e50 con 0x7f6e3407bd50 2024-01-11T17:49:29.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.676+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 msgr2=0x7f6e3419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.676+0000 7f6e3b4d8700 1 --2- 172.21.15.7:0/1489166363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3419cd00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009130 tx=0x7f6e30011ad0 comp rx=0 tx=0).stop 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.676+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 shutdown_connections 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.676+0000 7f6e3b4d8700 1 --2- 172.21.15.7:0/1489166363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e3407bd50 0x7f6e3419cd00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.676+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 >> 172.21.15.7:0/1489166363 conn(0x7f6e34102460 msgr2=0x7f6e34102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.677+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 shutdown_connections 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:29.677+0000 7f6e3b4d8700 1 -- 172.21.15.7:0/1489166363 wait complete. 2024-01-11T17:49:29.983 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-01-11T17:49:29.996 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-01-11T17:49:29.996 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-01-11T17:49:29.997 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-01-11T17:49:29.997 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-01-11T17:49:30.365 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x 2024-01-11T17:49:30.365 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x.service: Unit ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x.service not loaded. 2024-01-11T17:49:30.579 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715.target.wants/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@.service. 2024-01-11T17:49:30.676 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:30 smithi007 systemd[1]: Starting Ceph mgr.x for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:49:30.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:30 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1489166363' entity='client.admin' 2024-01-11T17:49:31.111 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-11T17:49:31.111 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-01-11T17:49:31.128 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-11T17:49:31.128 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-01-11T17:49:31.128 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-01-11T17:49:31.128 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-01-11T17:49:31.146 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-11T17:49:31.146 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-01-11T17:49:31.163 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-11T17:49:31.163 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-01-11T17:49:31.163 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-01-11T17:49:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-01-11T17:49:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-01-11T17:49:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-01-11T17:49:31.367 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:31 smithi007 bash[33596]: ee005514df344a318bdcb730244fb06a4a45f43958b6f9ac315eb0442fa03571 2024-01-11T17:49:31.367 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:31 smithi007 systemd[1]: Started Ceph mgr.x for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:49:31.897 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:31 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/198130256' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-11T17:49:31.897 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:31.645+0000 7fa1facaf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T17:49:31.897 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:31.776+0000 7fa1facaf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T17:49:31.897 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:31.897+0000 7fa1facaf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:31.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:31.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-11T17:49:31.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-11T17:49:31.990 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-01-11T17:49:24.249203+0000", 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.604+0000 7f3075f9c700 1 Processor -- start 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.604+0000 7f3075f9c700 1 -- start start 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.605+0000 7f3075f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307007a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.605+0000 7f3075f9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307007a780 con 0x7f307007bd50 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.605+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307007a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.605+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307007a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34796/0 (socket says 172.21.15.7:34796) 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.605+0000 7f306f7fe700 1 -- 172.21.15.7:0/258349399 learned_addr learned my addr 172.21.15.7:0/258349399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.606+0000 7f306f7fe700 1 -- 172.21.15.7:0/258349399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307007a8c0 con 0x7f307007bd50 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.606+0000 7f306f7fe700 1 --2- 172.21.15.7:0/258349399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307007a1b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f30600098a0 tx=0x7f3060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5999ed1fab015c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.606+0000 7f306e7fc700 1 -- 172.21.15.7:0/258349399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3060004160 con 0x7f307007bd50 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.606+0000 7f306e7fc700 1 -- 172.21.15.7:0/258349399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30600042c0 con 0x7f307007bd50 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.606+0000 7f306e7fc700 1 -- 172.21.15.7:0/258349399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3060011a00 con 0x7f307007bd50 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 -- 172.21.15.7:0/258349399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 msgr2=0x7f307007a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 --2- 172.21.15.7:0/258349399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307007a1b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f30600098a0 tx=0x7f3060009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 -- 172.21.15.7:0/258349399 shutdown_connections 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 --2- 172.21.15.7:0/258349399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307007a1b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 -- 172.21.15.7:0/258349399 >> 172.21.15.7:0/258349399 conn(0x7f3070102470 msgr2=0x7f3070104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:31.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 -- 172.21.15.7:0/258349399 shutdown_connections 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.607+0000 7f3075f9c700 1 -- 172.21.15.7:0/258349399 wait complete. 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.608+0000 7f3075f9c700 1 Processor -- start 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.608+0000 7f3075f9c700 1 -- start start 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.609+0000 7f3075f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.609+0000 7f3075f9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307019d320 con 0x7f307007bd50 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.609+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.609+0000 7f306f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34798/0 (socket says 172.21.15.7:34798) 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.609+0000 7f306f7fe700 1 -- 172.21.15.7:0/198130256 learned_addr learned my addr 172.21.15.7:0/198130256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.609+0000 7f306f7fe700 1 -- 172.21.15.7:0/198130256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060009550 con 0x7f307007bd50 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f306f7fe700 1 --2- 172.21.15.7:0/198130256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307019cde0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3060011a20 tx=0x7f3060011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3060003af0 con 0x7f307007bd50 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3060003c50 con 0x7f307007bd50 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307019d520 con 0x7f307007bd50 2024-01-11T17:49:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3060019760 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307019d9c0 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f3060019950 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.610+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f306000b720 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.611+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307010d220 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.614+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f306002a060 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.692+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f30701a0540 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.693+0000 7f306cff9700 1 -- 172.21.15.7:0/198130256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f3060019d90 con 0x7f307007bd50 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 msgr2=0x7f307019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 --2- 172.21.15.7:0/198130256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307019cde0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3060011a20 tx=0x7f3060011b00 comp rx=0 tx=0).stop 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 shutdown_connections 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 --2- 172.21.15.7:0/198130256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f307007bd50 0x7f307019cde0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 >> 172.21.15.7:0/198130256 conn(0x7f3070102470 msgr2=0x7f3070102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 shutdown_connections 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:31.694+0000 7f3075f9c700 1 -- 172.21.15.7:0/198130256 wait complete. 2024-01-11T17:49:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-01-11T17:49:32.320 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:32.018+0000 7fa1facaf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T17:49:33.210 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:32.901+0000 7fa1facaf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:49:33.470 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:33.210+0000 7fa1facaf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T17:49:33.471 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:33.326+0000 7fa1facaf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T17:49:33.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:33.470+0000 7fa1facaf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T17:49:33.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:33.671+0000 7fa1facaf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T17:49:34.147 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:33.848+0000 7fa1facaf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T17:49:34.148 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:33.965+0000 7fa1facaf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T17:49:34.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:34.488+0000 7fa1facaf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T17:49:34.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-01-11T17:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-11T17:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-11T17:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-01-11T17:49:24.249203+0000", 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.424+0000 7fcd0bc24700 1 Processor -- start 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.425+0000 7fcd0bc24700 1 -- start start 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.425+0000 7fcd0bc24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0407a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.425+0000 7fcd0bc24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0407a780 con 0x7fcd0407bd50 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.426+0000 7fcd099c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0407a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.426+0000 7fcd099c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0407a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40216/0 (socket says 172.21.15.7:40216) 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.426+0000 7fcd099c0700 1 -- 172.21.15.7:0/668981907 learned_addr learned my addr 172.21.15.7:0/668981907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.426+0000 7fcd099c0700 1 -- 172.21.15.7:0/668981907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0407a8c0 con 0x7fcd0407bd50 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.426+0000 7fcd099c0700 1 --2- 172.21.15.7:0/668981907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0407a1b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd000098a0 tx=0x7fcd00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd38b0343491a28e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd089be700 1 -- 172.21.15.7:0/668981907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd00004160 con 0x7fcd0407bd50 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd089be700 1 -- 172.21.15.7:0/668981907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd000042c0 con 0x7fcd0407bd50 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd089be700 1 -- 172.21.15.7:0/668981907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd00011a00 con 0x7fcd0407bd50 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 -- 172.21.15.7:0/668981907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 msgr2=0x7fcd0407a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 --2- 172.21.15.7:0/668981907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0407a1b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd000098a0 tx=0x7fcd00009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 -- 172.21.15.7:0/668981907 shutdown_connections 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 --2- 172.21.15.7:0/668981907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0407a1b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 -- 172.21.15.7:0/668981907 >> 172.21.15.7:0/668981907 conn(0x7fcd04102470 msgr2=0x7fcd04104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 -- 172.21.15.7:0/668981907 shutdown_connections 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.427+0000 7fcd0bc24700 1 -- 172.21.15.7:0/668981907 wait complete. 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.428+0000 7fcd0bc24700 1 Processor -- start 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.429+0000 7fcd0bc24700 1 -- start start 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.429+0000 7fcd0bc24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.429+0000 7fcd0bc24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419d320 con 0x7fcd0407bd50 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.429+0000 7fcd099c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.429+0000 7fcd099c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40224/0 (socket says 172.21.15.7:40224) 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.429+0000 7fcd099c0700 1 -- 172.21.15.7:0/1868680386 learned_addr learned my addr 172.21.15.7:0/1868680386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fcd099c0700 1 -- 172.21.15.7:0/1868680386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009550 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fcd099c0700 1 --2- 172.21.15.7:0/1868680386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0419cde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd00011a20 tx=0x7fcd00011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd00003af0 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd00003c50 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd00019760 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0419d520 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.430+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0419d9c0 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.431+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fcd00019950 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.431+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcd0000b720 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.431+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0410d220 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.434+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fcd0002a060 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.513+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcd041a0540 con 0x7fcd0407bd50 2024-01-11T17:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.513+0000 7fccfaffd700 1 -- 172.21.15.7:0/1868680386 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fcd00019d90 con 0x7fcd0407bd50 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.514+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 msgr2=0x7fcd0419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.514+0000 7fcd0bc24700 1 --2- 172.21.15.7:0/1868680386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0419cde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd00011a20 tx=0x7fcd00011b00 comp rx=0 tx=0).stop 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.514+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 shutdown_connections 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.514+0000 7fcd0bc24700 1 --2- 172.21.15.7:0/1868680386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0407bd50 0x7fcd0419cde0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.514+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 >> 172.21.15.7:0/1868680386 conn(0x7fcd04102470 msgr2=0x7fcd04102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.514+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 shutdown_connections 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:34.515+0000 7fcd0bc24700 1 -- 172.21.15.7:0/1868680386 wait complete. 2024-01-11T17:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-01-11T17:49:35.205 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:34.866+0000 7fa1facaf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T17:49:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:35 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1868680386' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-11T17:49:35.571 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:35.231+0000 7fa1facaf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T17:49:35.571 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:35.360+0000 7fa1facaf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T17:49:36.672 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:36.345+0000 7fa1facaf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T17:49:36.672 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:36.473+0000 7fa1facaf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T17:49:37.008 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:36.671+0000 7fa1facaf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-01-11T17:49:37.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-11T17:49:37.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-11T17:49:37.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.720 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-11T17:49:37.721 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:37.721 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-11T17:49:37.721 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:37 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1879171701' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-11T17:49:37.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-11T17:49:37.723 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:37.549+0000 7fa1facaf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T17:49:37.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-01-11T17:49:37.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:37.728 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-01-11T17:49:24.249203+0000", 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.276+0000 7fcfea19c700 1 Processor -- start 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.277+0000 7fcfea19c700 1 -- start start 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.277+0000 7fcfea19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe407a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.277+0000 7fcfea19c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe407a780 con 0x7fcfe407bd50 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.278+0000 7fcfe37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe407a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:37.729 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.278+0000 7fcfe37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe407a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40238/0 (socket says 172.21.15.7:40238) 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.278+0000 7fcfe37fe700 1 -- 172.21.15.7:0/1958570425 learned_addr learned my addr 172.21.15.7:0/1958570425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.278+0000 7fcfe37fe700 1 -- 172.21.15.7:0/1958570425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe407a8c0 con 0x7fcfe407bd50 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.278+0000 7fcfe37fe700 1 --2- 172.21.15.7:0/1958570425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe407a1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40098a0 tx=0x7fcfd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f418eaed53414d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.279+0000 7fcfe27fc700 1 -- 172.21.15.7:0/1958570425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4004160 con 0x7fcfe407bd50 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.279+0000 7fcfe27fc700 1 -- 172.21.15.7:0/1958570425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfd40042c0 con 0x7fcfe407bd50 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.279+0000 7fcfe27fc700 1 -- 172.21.15.7:0/1958570425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4011a00 con 0x7fcfe407bd50 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.279+0000 7fcfea19c700 1 -- 172.21.15.7:0/1958570425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 msgr2=0x7fcfe407a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.279+0000 7fcfea19c700 1 --2- 172.21.15.7:0/1958570425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe407a1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40098a0 tx=0x7fcfd4009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.280+0000 7fcfea19c700 1 -- 172.21.15.7:0/1958570425 shutdown_connections 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.280+0000 7fcfea19c700 1 --2- 172.21.15.7:0/1958570425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe407a1b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.280+0000 7fcfea19c700 1 -- 172.21.15.7:0/1958570425 >> 172.21.15.7:0/1958570425 conn(0x7fcfe4102470 msgr2=0x7fcfe4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.280+0000 7fcfea19c700 1 -- 172.21.15.7:0/1958570425 shutdown_connections 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.280+0000 7fcfea19c700 1 -- 172.21.15.7:0/1958570425 wait complete. 2024-01-11T17:49:37.730 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.281+0000 7fcfea19c700 1 Processor -- start 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.281+0000 7fcfea19c700 1 -- start start 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.282+0000 7fcfea19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.282+0000 7fcfea19c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe419d2e0 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.282+0000 7fcfe37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.282+0000 7fcfe37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40252/0 (socket says 172.21.15.7:40252) 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.282+0000 7fcfe37fe700 1 -- 172.21.15.7:0/1879171701 learned_addr learned my addr 172.21.15.7:0/1879171701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.282+0000 7fcfe37fe700 1 -- 172.21.15.7:0/1879171701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4009550 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfe37fe700 1 --2- 172.21.15.7:0/1879171701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe419cda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4011a20 tx=0x7fcfd4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4003af0 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfd4003c50 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe419d4e0 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4019760 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe419d980 con 0x7fcfe407bd50 2024-01-11T17:49:37.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fcfd4019950 con 0x7fcfe407bd50 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.283+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcfd400b720 con 0x7fcfe407bd50 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.284+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe410d220 con 0x7fcfe407bd50 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.288+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fcfd402a060 con 0x7fcfe407bd50 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.367+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcfe41a0520 con 0x7fcfe407bd50 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.368+0000 7fcfe0ff9700 1 -- 172.21.15.7:0/1879171701 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fcfd4019d90 con 0x7fcfe407bd50 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 msgr2=0x7fcfe419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 --2- 172.21.15.7:0/1879171701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe419cda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4011a20 tx=0x7fcfd4011b00 comp rx=0 tx=0).stop 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 shutdown_connections 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 --2- 172.21.15.7:0/1879171701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe407bd50 0x7fcfe419cda0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 >> 172.21.15.7:0/1879171701 conn(0x7fcfe4102470 msgr2=0x7fcfe4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 shutdown_connections 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:37.369+0000 7fcfea19c700 1 -- 172.21.15.7:0/1879171701 wait complete. 2024-01-11T17:49:37.732 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-01-11T17:49:38.068 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:37.803+0000 7fa1facaf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T17:49:38.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:38.068+0000 7fa1facaf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T17:49:39.571 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:39.245+0000 7fa1facaf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T17:49:40.073 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:39.630+0000 7fa1facaf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:49:40.073 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:39.787+0000 7fa1facaf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:40.098+0000 7fa1facaf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: Activating manager daemon x 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: mgrmap e2: x(active, starting, since 0.0045071s) 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: Manager daemon x is now available 2024-01-11T17:49:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-11T17:49:40.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' 2024-01-11T17:49:40.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-11T17:49:40.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' 2024-01-11T17:49:40.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:40 smithi007 ceph-mon[33361]: from='mgr.14100 172.21.15.7:0/603628798' entity='mgr.x' 2024-01-11T17:49:42.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:42 smithi007 ceph-mon[33361]: mgrmap e3: x(active, since 1.0097s) 2024-01-11T17:49:43.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:43 smithi007 ceph-mon[33361]: mgrmap e4: x(active, since 2s) 2024-01-11T17:49:45.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:45 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3586331027' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.400 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-11T17:49:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-01-11T17:49:24.249203+0000", 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.872+0000 7fd052177700 1 Processor -- start 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.872+0000 7fd052177700 1 -- start start 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.872+0000 7fd052177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.872+0000 7fd052177700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c07a780 con 0x7fd04c07bd50 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.873+0000 7fd04b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.873+0000 7fd04b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c07a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49826/0 (socket says 172.21.15.7:49826) 2024-01-11T17:49:47.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.873+0000 7fd04b7fe700 1 -- 172.21.15.7:0/806249230 learned_addr learned my addr 172.21.15.7:0/806249230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.873+0000 7fd04b7fe700 1 -- 172.21.15.7:0/806249230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c07a8c0 con 0x7fd04c07bd50 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.873+0000 7fd04b7fe700 1 --2- 172.21.15.7:0/806249230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c07a1b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0098a0 tx=0x7fd03c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8696e7f4207f9b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd04a7fc700 1 -- 172.21.15.7:0/806249230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd03c004160 con 0x7fd04c07bd50 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd04a7fc700 1 -- 172.21.15.7:0/806249230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd03c0042c0 con 0x7fd04c07bd50 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd04a7fc700 1 -- 172.21.15.7:0/806249230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd03c011a00 con 0x7fd04c07bd50 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 -- 172.21.15.7:0/806249230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 msgr2=0x7fd04c07a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 --2- 172.21.15.7:0/806249230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c07a1b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0098a0 tx=0x7fd03c009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 -- 172.21.15.7:0/806249230 shutdown_connections 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 --2- 172.21.15.7:0/806249230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c07a1b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 -- 172.21.15.7:0/806249230 >> 172.21.15.7:0/806249230 conn(0x7fd04c102470 msgr2=0x7fd04c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 -- 172.21.15.7:0/806249230 shutdown_connections 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.874+0000 7fd052177700 1 -- 172.21.15.7:0/806249230 wait complete. 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.875+0000 7fd052177700 1 Processor -- start 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd052177700 1 -- start start 2024-01-11T17:49:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd052177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd052177700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c19d2e0 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd04b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd04b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49834/0 (socket says 172.21.15.7:49834) 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd04b7fe700 1 -- 172.21.15.7:0/3586331027 learned_addr learned my addr 172.21.15.7:0/3586331027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.876+0000 7fd04b7fe700 1 -- 172.21.15.7:0/3586331027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c009550 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.877+0000 7fd04b7fe700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c19cda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd03c011a20 tx=0x7fd03c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.877+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd03c003af0 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.877+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd03c003c50 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.877+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd03c019760 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.877+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c19d4e0 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.877+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c19d980 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.878+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 45868+0+0 (secure 0 0 0) 0x7fd03c01ab20 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.878+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c10d220 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.878+0000 7fd048ff9700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd034038e10 0x7fd03403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.878+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd03c04d080 con 0x7fd04c07bd50 2024-01-11T17:49:47.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.879+0000 7fd04affd700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd034038e10 0x7fd03403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.879+0000 7fd04affd700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd034038e10 0x7fd03403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd040006fd0 tx=0x7fd040006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:44.885+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd03c02dbd0 con 0x7fd04c07bd50 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.187+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd04c1a0750 con 0x7fd04c07bd50 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.188+0000 7fd048ff9700 1 -- 172.21.15.7:0/3586331027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fd03c02d7c0 con 0x7fd04c07bd50 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd034038e10 msgr2=0x7fd03403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd034038e10 0x7fd03403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd040006fd0 tx=0x7fd040006e40 comp rx=0 tx=0).stop 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 msgr2=0x7fd04c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c19cda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd03c011a20 tx=0x7fd03c011b00 comp rx=0 tx=0).stop 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 shutdown_connections 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd034038e10 0x7fd03403b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 --2- 172.21.15.7:0/3586331027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04c07bd50 0x7fd04c19cda0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.190+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 >> 172.21.15.7:0/3586331027 conn(0x7fd04c102470 msgr2=0x7fd04c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.191+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 shutdown_connections 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:45.191+0000 7fd052177700 1 -- 172.21.15.7:0/3586331027 wait complete. 2024-01-11T17:49:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-01-11T17:49:48.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:48 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3425253724' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-01-11T17:49:48.361 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:48.361 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-01-11T17:49:48.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.822+0000 7f30e271c700 1 Processor -- start 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.822+0000 7f30e271c700 1 -- start start 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.823+0000 7f30e271c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.823+0000 7f30e271c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc07a780 con 0x7f30dc07bd50 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.823+0000 7f30dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.823+0000 7f30dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc07a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49840/0 (socket says 172.21.15.7:49840) 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.823+0000 7f30dbfff700 1 -- 172.21.15.7:0/29163470 learned_addr learned my addr 172.21.15.7:0/29163470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.823+0000 7f30dbfff700 1 -- 172.21.15.7:0/29163470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc07a8c0 con 0x7f30dc07bd50 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.824+0000 7f30dbfff700 1 --2- 172.21.15.7:0/29163470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc07a1b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098a0 tx=0x7f30cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dbfa82dac7d740 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.824+0000 7f30daffd700 1 -- 172.21.15.7:0/29163470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc004160 con 0x7f30dc07bd50 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.824+0000 7f30daffd700 1 -- 172.21.15.7:0/29163470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30cc0042c0 con 0x7f30dc07bd50 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.824+0000 7f30daffd700 1 -- 172.21.15.7:0/29163470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc011a00 con 0x7f30dc07bd50 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.824+0000 7f30e271c700 1 -- 172.21.15.7:0/29163470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 msgr2=0x7f30dc07a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.824+0000 7f30e271c700 1 --2- 172.21.15.7:0/29163470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc07a1b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098a0 tx=0x7f30cc009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.825+0000 7f30e271c700 1 -- 172.21.15.7:0/29163470 shutdown_connections 2024-01-11T17:49:48.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.825+0000 7f30e271c700 1 --2- 172.21.15.7:0/29163470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc07a1b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.825+0000 7f30e271c700 1 -- 172.21.15.7:0/29163470 >> 172.21.15.7:0/29163470 conn(0x7f30dc102470 msgr2=0x7f30dc104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.825+0000 7f30e271c700 1 -- 172.21.15.7:0/29163470 shutdown_connections 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.825+0000 7f30e271c700 1 -- 172.21.15.7:0/29163470 wait complete. 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30e271c700 1 Processor -- start 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30e271c700 1 -- start start 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30e271c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30e271c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc19d2e0 con 0x7f30dc07bd50 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49844/0 (socket says 172.21.15.7:49844) 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.826+0000 7f30dbfff700 1 -- 172.21.15.7:0/3425253724 learned_addr learned my addr 172.21.15.7:0/3425253724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30dbfff700 1 -- 172.21.15.7:0/3425253724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc009550 con 0x7f30dc07bd50 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30dbfff700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc19cda0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30cc011a20 tx=0x7f30cc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc003af0 con 0x7f30dc07bd50 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30cc003c50 con 0x7f30dc07bd50 2024-01-11T17:49:48.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc019760 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30dc19d4e0 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.827+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc19d980 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.828+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 45868+0+0 (secure 0 0 0) 0x7f30cc01ab20 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.828+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30dc10d220 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.829+0000 7f30d97fa700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f30c4038e10 0x7f30c403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.829+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f30cc04d0e0 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.829+0000 7f30db7fe700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f30c4038e10 0x7f30c403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.830+0000 7f30db7fe700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f30c4038e10 0x7f30c403b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006fd0 tx=0x7f30d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:47.834+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f30cc080380 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.043+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f30dc10d430 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.044+0000 7f30d97fa700 1 -- 172.21.15.7:0/3425253724 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f30cc01a7b0 con 0x7f30dc07bd50 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f30c4038e10 msgr2=0x7f30c403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f30c4038e10 0x7f30c403b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006fd0 tx=0x7f30d0006e40 comp rx=0 tx=0).stop 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 msgr2=0x7f30dc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:48.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc19cda0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30cc011a20 tx=0x7f30cc011b00 comp rx=0 tx=0).stop 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 shutdown_connections 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f30c4038e10 0x7f30c403b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 --2- 172.21.15.7:0/3425253724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30dc07bd50 0x7f30dc19cda0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 >> 172.21.15.7:0/3425253724 conn(0x7f30dc102470 msgr2=0x7f30dc102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 shutdown_connections 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.047+0000 7f30e271c700 1 -- 172.21.15.7:0/3425253724 wait complete. 2024-01-11T17:49:48.366 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-01-11T17:49:49.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.757+0000 7fd1bd1fb700 1 Processor -- start 2024-01-11T17:49:49.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.758+0000 7fd1bd1fb700 1 -- start start 2024-01-11T17:49:49.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.758+0000 7fd1bd1fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b807a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.758+0000 7fd1bd1fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b807a780 con 0x7fd1b807bd50 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.758+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b807a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.758+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b807a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49852/0 (socket says 172.21.15.7:49852) 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.758+0000 7fd1b6d9d700 1 -- 172.21.15.7:0/135818849 learned_addr learned my addr 172.21.15.7:0/135818849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.759+0000 7fd1b6d9d700 1 -- 172.21.15.7:0/135818849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b807a8c0 con 0x7fd1b807bd50 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.759+0000 7fd1b6d9d700 1 --2- 172.21.15.7:0/135818849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b807a1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098a0 tx=0x7fd1a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0af65ffd3e83e6d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.759+0000 7fd1b5d9b700 1 -- 172.21.15.7:0/135818849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1a8004160 con 0x7fd1b807bd50 2024-01-11T17:49:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.759+0000 7fd1b5d9b700 1 -- 172.21.15.7:0/135818849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1a80042c0 con 0x7fd1b807bd50 2024-01-11T17:49:49.449 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:49 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1246247163' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-01-11T17:49:49.449 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: ignoring --setuser ceph since I am not root 2024-01-11T17:49:49.449 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: ignoring --setgroup ceph since I am not root 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.759+0000 7fd1b5d9b700 1 -- 172.21.15.7:0/135818849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1a8011a00 con 0x7fd1b807bd50 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/135818849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 msgr2=0x7fd1b807a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 --2- 172.21.15.7:0/135818849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b807a1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098a0 tx=0x7fd1a8009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/135818849 shutdown_connections 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 --2- 172.21.15.7:0/135818849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b807a1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/135818849 >> 172.21.15.7:0/135818849 conn(0x7fd1b8102460 msgr2=0x7fd1b8104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/135818849 shutdown_connections 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.760+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/135818849 wait complete. 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.761+0000 7fd1bd1fb700 1 Processor -- start 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.762+0000 7fd1bd1fb700 1 -- start start 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.762+0000 7fd1bd1fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:49.453 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.762+0000 7fd1bd1fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b819d240 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.762+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.762+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49858/0 (socket says 172.21.15.7:49858) 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.762+0000 7fd1b6d9d700 1 -- 172.21.15.7:0/1246247163 learned_addr learned my addr 172.21.15.7:0/1246247163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd1b6d9d700 1 -- 172.21.15.7:0/1246247163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8009550 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd1b6d9d700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b819cd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009130 tx=0x7fd1a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1a8003b70 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1a8003cd0 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1a80197d0 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b819d440 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.763+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b819d8e0 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.764+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 45868+0+0 (secure 0 0 0) 0x7fd1a801abb0 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.764+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b810d1f0 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.765+0000 7fd19ffff700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd1a0038e10 0x7fd1a003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.765+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd1a804d020 con 0x7fd1b807bd50 2024-01-11T17:49:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.765+0000 7fd1b659c700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd1a0038e10 0x7fd1a003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.766+0000 7fd1b659c700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd1a0038e10 0x7fd1a003b2d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac006fd0 tx=0x7fd1ac006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:48.770+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd1a8080380 con 0x7fd1b807bd50 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.034+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd1b81a06a0 con 0x7fd1b807bd50 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.098+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd1a801a430 con 0x7fd1b807bd50 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.100+0000 7fd19ffff700 1 -- 172.21.15.7:0/1246247163 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) v1 ==== 45879+0+0 (secure 0 0 0) 0x7fd1a801a7b0 con 0x7fd1b807bd50 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd1a0038e10 msgr2=0x7fd1a003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd1a0038e10 0x7fd1a003b2d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac006fd0 tx=0x7fd1ac006e40 comp rx=0 tx=0).stop 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 msgr2=0x7fd1b819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b819cd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009130 tx=0x7fd1a8011ad0 comp rx=0 tx=0).stop 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 shutdown_connections 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fd1a0038e10 0x7fd1a003b2d0 secure :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac006fd0 tx=0x7fd1ac006e40 comp rx=0 tx=0).stop 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 --2- 172.21.15.7:0/1246247163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b807bd50 0x7fd1b819cd00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 >> 172.21.15.7:0/1246247163 conn(0x7fd1b8102460 msgr2=0x7fd1b8102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 shutdown_connections 2024-01-11T17:49:49.456 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.102+0000 7fd1bd1fb700 1 -- 172.21.15.7:0/1246247163 wait complete. 2024-01-11T17:49:49.975 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:49.724+0000 7f7103b67200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T17:49:49.975 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:49.853+0000 7f7103b67200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T17:49:50.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:50 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1246247163' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-01-11T17:49:50.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:50 smithi007 ceph-mon[33361]: mgrmap e5: x(active, since 9s) 2024-01-11T17:49:50.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:49.974+0000 7f7103b67200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T17:49:50.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:50 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:50.090+0000 7f7103b67200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T17:49:50.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:49:50.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.872+0000 7fac7888b700 1 Processor -- start 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.872+0000 7fac7888b700 1 -- start start 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.872+0000 7fac7888b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac741072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.872+0000 7fac7888b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac741077f0 con 0x7fac74106e90 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.873+0000 7fac7259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac741072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.873+0000 7fac7259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac741072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49876/0 (socket says 172.21.15.7:49876) 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.873+0000 7fac7259c700 1 -- 172.21.15.7:0/1319045781 learned_addr learned my addr 172.21.15.7:0/1319045781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.873+0000 7fac7259c700 1 -- 172.21.15.7:0/1319045781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac74107930 con 0x7fac74106e90 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.873+0000 7fac7259c700 1 --2- 172.21.15.7:0/1319045781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac741072b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac640098a0 tx=0x7fac64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f2ef1d4896a633 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7159a700 1 -- 172.21.15.7:0/1319045781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac64004160 con 0x7fac74106e90 2024-01-11T17:49:50.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7159a700 1 -- 172.21.15.7:0/1319045781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac640042c0 con 0x7fac74106e90 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7159a700 1 -- 172.21.15.7:0/1319045781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac64011a00 con 0x7fac74106e90 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7888b700 1 -- 172.21.15.7:0/1319045781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 msgr2=0x7fac741072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7888b700 1 --2- 172.21.15.7:0/1319045781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac741072b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac640098a0 tx=0x7fac64009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7888b700 1 -- 172.21.15.7:0/1319045781 shutdown_connections 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7888b700 1 --2- 172.21.15.7:0/1319045781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac741072b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.874+0000 7fac7888b700 1 -- 172.21.15.7:0/1319045781 >> 172.21.15.7:0/1319045781 conn(0x7fac74102470 msgr2=0x7fac74104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.875+0000 7fac7888b700 1 -- 172.21.15.7:0/1319045781 shutdown_connections 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.875+0000 7fac7888b700 1 -- 172.21.15.7:0/1319045781 wait complete. 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7888b700 1 Processor -- start 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7888b700 1 -- start start 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7888b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac7419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7888b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7419d320 con 0x7fac74106e90 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac7419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:50.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac7419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49892/0 (socket says 172.21.15.7:49892) 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.876+0000 7fac7259c700 1 -- 172.21.15.7:0/114389983 learned_addr learned my addr 172.21.15.7:0/114389983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac7259c700 1 -- 172.21.15.7:0/114389983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64009550 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac7259c700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac7419cde0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac64011a20 tx=0x7fac64011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac64003af0 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac64003c50 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac64019760 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7419d520 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.877+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7419d9c0 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.878+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac74108c50 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.879+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 45879+0+0 (secure 0 0 0) 0x7fac6401a430 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.879+0000 7fac637fe700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 0x7fac5c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.879+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fac6404d050 con 0x7fac74106e90 2024-01-11T17:49:50.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.879+0000 7fac71d9b700 1 -- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 msgr2=0x7fac5c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.879+0000 7fac71d9b700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 0x7fac5c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:49.884+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fac6402d7c0 con 0x7fac74106e90 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.080+0000 7fac71d9b700 1 -- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 msgr2=0x7fac5c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.080+0000 7fac71d9b700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 0x7fac5c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.156+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fac741a06b0 con 0x7fac74106e90 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.157+0000 7fac637fe700 1 -- 172.21.15.7:0/114389983 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fac6402d7c0 con 0x7fac74106e90 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 msgr2=0x7fac5c03b320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 0x7fac5c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 msgr2=0x7fac7419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac7419cde0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac64011a20 tx=0x7fac64011b00 comp rx=0 tx=0).stop 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 shutdown_connections 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7fac5c038e60 0x7fac5c03b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 --2- 172.21.15.7:0/114389983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac74106e90 0x7fac7419cde0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 >> 172.21.15.7:0/114389983 conn(0x7fac74102470 msgr2=0x7fac74102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 shutdown_connections 2024-01-11T17:49:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.159+0000 7fac7888b700 1 -- 172.21.15.7:0/114389983 wait complete. 2024-01-11T17:49:50.491 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-01-11T17:49:50.491 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-01-11T17:49:51.280 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:51 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/114389983' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-01-11T17:49:51.280 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:50 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:50.971+0000 7f7103b67200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:49:51.540 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:51 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:51.280+0000 7f7103b67200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T17:49:51.540 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:51 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:51.396+0000 7f7103b67200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T17:49:51.820 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:51 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:51.540+0000 7f7103b67200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T17:49:51.877 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:51 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:51.740+0000 7f7103b67200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T17:49:52.320 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:51 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:51.915+0000 7f7103b67200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T17:49:52.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:52 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:52.033+0000 7f7103b67200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T17:49:52.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:52 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:52.516+0000 7f7103b67200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T17:49:53.243 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:52 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:52.877+0000 7f7103b67200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T17:49:53.571 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:53 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:53.243+0000 7f7103b67200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T17:49:53.571 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:53 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:53.371+0000 7f7103b67200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T17:49:54.681 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:54 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:54.354+0000 7f7103b67200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T17:49:54.682 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:54 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:54.482+0000 7f7103b67200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T17:49:55.070 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:54 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:54.681+0000 7f7103b67200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T17:49:55.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:55 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:55.526+0000 7f7103b67200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T17:49:55.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:55 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:55.776+0000 7f7103b67200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T17:49:56.320 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:56 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:56.042+0000 7f7103b67200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T17:49:57.570 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:57 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:57.215+0000 7f7103b67200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T17:49:57.973 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:57 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:57.585+0000 7f7103b67200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:49:57.973 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:57 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:57.706+0000 7f7103b67200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:49:57 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:49:57.972+0000 7f7103b67200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: Active manager daemon x restarted 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: Activating manager daemon x 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: osdmap e2: 0 total, 0 up, 0 in 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: mgrmap e6: x(active, starting, since 0.00408498s) 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-11T17:49:58.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: Manager daemon x is now available 2024-01-11T17:49:58.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:49:58.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:49:58.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:58 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: Found migration_current of "None". Setting to last migration. 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: mgrmap e7: x(active, since 1.00684s) 2024-01-11T17:49:59.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:49:59 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:49:59.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:49:59.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.950+0000 7f49473be700 1 Processor -- start 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.950+0000 7f49473be700 1 -- start start 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.950+0000 7f49473be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940106170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.950+0000 7f49473be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940106740 con 0x7f4940105d50 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.951+0000 7f494515a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940106170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.951+0000 7f494515a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940106170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49906/0 (socket says 172.21.15.7:49906) 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.951+0000 7f494515a700 1 -- 172.21.15.7:0/2850518998 learned_addr learned my addr 172.21.15.7:0/2850518998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.951+0000 7f494515a700 1 -- 172.21.15.7:0/2850518998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940106f50 con 0x7f4940105d50 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.951+0000 7f494515a700 1 --2- 172.21.15.7:0/2850518998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940106170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f493c0098a0 tx=0x7f493c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5ce35a51332082 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f4937fff700 1 -- 172.21.15.7:0/2850518998 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f493c004160 con 0x7f4940105d50 2024-01-11T17:49:59.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f4937fff700 1 -- 172.21.15.7:0/2850518998 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f493c0042c0 con 0x7f4940105d50 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f4937fff700 1 -- 172.21.15.7:0/2850518998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f493c011a00 con 0x7f4940105d50 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f49473be700 1 -- 172.21.15.7:0/2850518998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 msgr2=0x7f4940106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f49473be700 1 --2- 172.21.15.7:0/2850518998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940106170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f493c0098a0 tx=0x7f493c009bb0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f49473be700 1 -- 172.21.15.7:0/2850518998 shutdown_connections 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.952+0000 7f49473be700 1 --2- 172.21.15.7:0/2850518998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940106170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.953+0000 7f49473be700 1 -- 172.21.15.7:0/2850518998 >> 172.21.15.7:0/2850518998 conn(0x7f4940101310 msgr2=0x7f4940103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.953+0000 7f49473be700 1 -- 172.21.15.7:0/2850518998 shutdown_connections 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.953+0000 7f49473be700 1 -- 172.21.15.7:0/2850518998 wait complete. 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f49473be700 1 Processor -- start 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f49473be700 1 -- start start 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f49473be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940190d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f49473be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49401912d0 con 0x7f4940105d50 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f494515a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940190d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:59.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f494515a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940190d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49916/0 (socket says 172.21.15.7:49916) 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.954+0000 7f494515a700 1 -- 172.21.15.7:0/1230021612 learned_addr learned my addr 172.21.15.7:0/1230021612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f494515a700 1 -- 172.21.15.7:0/1230021612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c009550 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f494515a700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940190d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f493c011710 tx=0x7f493c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f493c003aa0 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f493c003c00 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f493c024800 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494018f4c0 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.955+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494018f960 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.957+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f493c01abd0 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.957+0000 7f49367fc700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.957+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 --> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f492c03ba30 con 0x7f492c038e60 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.957+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f493c04cb20 con 0x7f4940105d50 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.957+0000 7f4944959700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:59.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:50.957+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:51.158+0000 7f4944959700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:51.158+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:51.559+0000 7f4944959700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:51.559+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:52.360+0000 7f4944959700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:52.360+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:53.961+0000 7f4944959700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:53.961+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:57.164+0000 7f4944959700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1981932676 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:57.164+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:57.977+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) v1 ==== 45646+0+0 (secure 0 0 0) 0x7f493c01a430 con 0x7f4940105d50 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:57.977+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 msgr2=0x7f492c03b320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:57.977+0000 7f49367fc700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.979+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) v1 ==== 45773+0+0 (secure 0 0 0) 0x7f493c04e690 con 0x7f4940105d50 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.979+0000 7f49367fc700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f492c03c740 0x7f492c03eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.979+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f492c03ba30 con 0x7f492c03c740 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.980+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f492c03c740 0x7f492c03eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.981+0000 7f4944959700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f492c03c740 0x7f492c03eb30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4930003a10 tx=0x7f49300092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.982+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f492c03ba30 con 0x7f492c03c740 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.989+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4940107d50 con 0x7f492c03c740 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.989+0000 7f49367fc700 1 -- 172.21.15.7:0/1230021612 <== mgr.14120 v2:172.21.15.7:6800/867047662 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f493000a050 con 0x7f492c03c740 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f492c03c740 msgr2=0x7f492c03eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f492c03c740 0x7f492c03eb30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4930003a10 tx=0x7f49300092b0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 msgr2=0x7f4940190d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940190d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f493c011710 tx=0x7f493c011ad0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 shutdown_connections 2024-01-11T17:49:59.348 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f492c03c740 0x7f492c03eb30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4940105d50 0x7f4940190d90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 --2- 172.21.15.7:0/1230021612 >> [v2:172.21.15.7:6800/1981932676,v1:172.21.15.7:6801/1981932676] conn(0x7f492c038e60 0x7f492c03b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 >> 172.21.15.7:0/1230021612 conn(0x7f4940101310 msgr2=0x7f4940102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 shutdown_connections 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:58.990+0000 7f49473be700 1 -- 172.21.15.7:0/1230021612 wait complete. 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-01-11T17:49:59.349 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-01-11T17:50:00.312 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.780+0000 7fbbd1ae4700 1 Processor -- start 2024-01-11T17:50:00.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:00 smithi007 ceph-mon[33361]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-01-11T17:50:00.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:00 smithi007 ceph-mon[33361]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-01-11T17:50:00.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:00 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:00.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:00 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:00.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:00 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:00.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.780+0000 7fbbd1ae4700 1 -- start start 2024-01-11T17:50:00.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.780+0000 7fbbd1ae4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:00.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.780+0000 7fbbd1ae4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1077c0 con 0x7fbbcc106e60 2024-01-11T17:50:00.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.780+0000 7fbbcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:00.318 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59398/0 (socket says 172.21.15.7:59398) 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbcb7fe700 1 -- 172.21.15.7:0/817654603 learned_addr learned my addr 172.21.15.7:0/817654603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbcb7fe700 1 -- 172.21.15.7:0/817654603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc107900 con 0x7fbbcc106e60 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbcb7fe700 1 --2- 172.21.15.7:0/817654603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc107280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0098a0 tx=0x7fbbbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa731ddfad92bd22 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbca7fc700 1 -- 172.21.15.7:0/817654603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbbc004160 con 0x7fbbcc106e60 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbca7fc700 1 -- 172.21.15.7:0/817654603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbbbc0042c0 con 0x7fbbcc106e60 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.781+0000 7fbbca7fc700 1 -- 172.21.15.7:0/817654603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbbc011a00 con 0x7fbbcc106e60 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/817654603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 msgr2=0x7fbbcc107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 --2- 172.21.15.7:0/817654603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc107280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0098a0 tx=0x7fbbbc009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/817654603 shutdown_connections 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 --2- 172.21.15.7:0/817654603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc107280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/817654603 >> 172.21.15.7:0/817654603 conn(0x7fbbcc102460 msgr2=0x7fbbcc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/817654603 shutdown_connections 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.782+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/817654603 wait complete. 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.784+0000 7fbbd1ae4700 1 Processor -- start 2024-01-11T17:50:00.319 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.784+0000 7fbbd1ae4700 1 -- start start 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.784+0000 7fbbd1ae4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc07ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.784+0000 7fbbd1ae4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc07bf80 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc07ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbcb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc07ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59410/0 (socket says 172.21.15.7:59410) 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbcb7fe700 1 -- 172.21.15.7:0/4270188912 learned_addr learned my addr 172.21.15.7:0/4270188912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbcb7fe700 1 -- 172.21.15.7:0/4270188912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc009550 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbcb7fe700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc07ba40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fb0 tx=0x7fbbbc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbbc003a40 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbbbc003ba0 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbbc01a750 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.785+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc07a170 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.786+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc07a610 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.787+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc062c40 con 0x7fbbcc106e60 2024-01-11T17:50:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.787+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 7) v1 ==== 45773+0+0 (secure 0 0 0) 0x7fbbbc024b60 con 0x7fbbcc106e60 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.787+0000 7fbbc8ff9700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbbb4038d00 0x7fbbb403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.787+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbbbc04cd80 con 0x7fbbcc106e60 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.787+0000 7fbbcaffd700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbbb4038d00 0x7fbbb403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.788+0000 7fbbcaffd700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbbb4038d00 0x7fbbb403b1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0006fd0 tx=0x7fbbc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:49:59.793+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fbbbc024430 con 0x7fbbcc106e60 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.023+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbcc07aca0 con 0x7fbbb4038d00 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.029+0000 7fbbc8ff9700 1 -- 172.21.15.7:0/4270188912 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbbcc07aca0 con 0x7fbbb4038d00 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.031+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbbb4038d00 msgr2=0x7fbbb403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.031+0000 7fbbd1ae4700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbbb4038d00 0x7fbbb403b1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0006fd0 tx=0x7fbbc0006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 msgr2=0x7fbbcc07ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc07ba40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fb0 tx=0x7fbbbc011ad0 comp rx=0 tx=0).stop 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 shutdown_connections 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbbb4038d00 0x7fbbb403b1c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 --2- 172.21.15.7:0/4270188912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbcc106e60 0x7fbbcc07ba40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 >> 172.21.15.7:0/4270188912 conn(0x7fbbcc102460 msgr2=0x7fbbcc102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:00.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 shutdown_connections 2024-01-11T17:50:00.322 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.032+0000 7fbbd1ae4700 1 -- 172.21.15.7:0/4270188912 wait complete. 2024-01-11T17:50:01.279 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-01-11T17:50:01.279 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.727+0000 7f4944dd7700 1 Processor -- start 2024-01-11T17:50:01.279 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.727+0000 7f4944dd7700 1 -- start start 2024-01-11T17:50:01.279 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.728+0000 7f4944dd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f4940109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.728+0000 7f4944dd7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940074ba0 con 0x7f49401079a0 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.728+0000 7f493e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f4940109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.728+0000 7f493e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f4940109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59420/0 (socket says 172.21.15.7:59420) 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.728+0000 7f493e59c700 1 -- 172.21.15.7:0/1037268437 learned_addr learned my addr 172.21.15.7:0/1037268437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.729+0000 7f493e59c700 1 -- 172.21.15.7:0/1037268437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940074ce0 con 0x7f49401079a0 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.729+0000 7f493e59c700 1 --2- 172.21.15.7:0/1037268437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f4940109d90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49300098a0 tx=0x7f4930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36be24689f98bb28 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.729+0000 7f493d59a700 1 -- 172.21.15.7:0/1037268437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4930004160 con 0x7f49401079a0 2024-01-11T17:50:01.280 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:01 smithi007 ceph-mon[33361]: [11/Jan/2024:17:49:59] ENGINE Bus STARTING 2024-01-11T17:50:01.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:01 smithi007 ceph-mon[33361]: [11/Jan/2024:17:49:59] ENGINE Serving on http://172.21.15.7:8765 2024-01-11T17:50:01.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:01 smithi007 ceph-mon[33361]: [11/Jan/2024:17:49:59] ENGINE Serving on https://172.21.15.7:7150 2024-01-11T17:50:01.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:01 smithi007 ceph-mon[33361]: [11/Jan/2024:17:49:59] ENGINE Bus STARTED 2024-01-11T17:50:01.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:01 smithi007 ceph-mon[33361]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:01.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:01 smithi007 ceph-mon[33361]: mgrmap e8: x(active, since 2s) 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.729+0000 7f493d59a700 1 -- 172.21.15.7:0/1037268437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49300042c0 con 0x7f49401079a0 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.729+0000 7f493d59a700 1 -- 172.21.15.7:0/1037268437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4930011a00 con 0x7f49401079a0 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 -- 172.21.15.7:0/1037268437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 msgr2=0x7f4940109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 --2- 172.21.15.7:0/1037268437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f4940109d90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49300098a0 tx=0x7f4930009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 -- 172.21.15.7:0/1037268437 shutdown_connections 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 --2- 172.21.15.7:0/1037268437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f4940109d90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 -- 172.21.15.7:0/1037268437 >> 172.21.15.7:0/1037268437 conn(0x7f4940101310 msgr2=0x7f4940103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 -- 172.21.15.7:0/1037268437 shutdown_connections 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.730+0000 7f4944dd7700 1 -- 172.21.15.7:0/1037268437 wait complete. 2024-01-11T17:50:01.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.731+0000 7f4944dd7700 1 Processor -- start 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.731+0000 7f4944dd7700 1 -- start start 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.731+0000 7f4944dd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f494019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.731+0000 7f4944dd7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494019b0c0 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.732+0000 7f493e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f494019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.732+0000 7f493e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f494019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59432/0 (socket says 172.21.15.7:59432) 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.732+0000 7f493e59c700 1 -- 172.21.15.7:0/446770432 learned_addr learned my addr 172.21.15.7:0/446770432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.732+0000 7f493e59c700 1 -- 172.21.15.7:0/446770432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930009550 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.732+0000 7f493e59c700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f494019ab80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4930011a20 tx=0x7f4930011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.733+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4930003af0 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.733+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494019b2c0 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.733+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4930003c50 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.733+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4930019760 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.733+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494019b760 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.734+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494010af90 con 0x7f49401079a0 2024-01-11T17:50:01.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.734+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f493001ab20 con 0x7f49401079a0 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.734+0000 7f492f7fe700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f4928038e10 0x7f492803b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.734+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f493004d230 con 0x7f49401079a0 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.734+0000 7f493dd9b700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f4928038e10 0x7f492803b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.735+0000 7f493dd9b700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f4928038e10 0x7f492803b2d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4934006fd0 tx=0x7f4934006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.739+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f493001a7b0 con 0x7f49401079a0 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.951+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f4940102580 con 0x7f4928038e10 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.952+0000 7f492f7fe700 1 -- 172.21.15.7:0/446770432 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f4940102580 con 0x7f4928038e10 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f4928038e10 msgr2=0x7f492803b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f4928038e10 0x7f492803b2d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4934006fd0 tx=0x7f4934006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 msgr2=0x7f494019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f494019ab80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4930011a20 tx=0x7f4930011b00 comp rx=0 tx=0).stop 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 shutdown_connections 2024-01-11T17:50:01.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f4928038e10 0x7f492803b2d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 --2- 172.21.15.7:0/446770432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49401079a0 0x7f494019ab80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 >> 172.21.15.7:0/446770432 conn(0x7f4940101310 msgr2=0x7f4940101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 shutdown_connections 2024-01-11T17:50:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:00.955+0000 7f4944dd7700 1 -- 172.21.15.7:0/446770432 wait complete. 2024-01-11T17:50:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-01-11T17:50:02.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-mon[33361]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:02.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.693+0000 7f599e60c700 1 Processor -- start 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.693+0000 7f599e60c700 1 -- start start 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.694+0000 7f599e60c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599807a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.694+0000 7f599e60c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599807a760 con 0x7f599807bd30 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.694+0000 7f5997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599807a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.694+0000 7f5997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599807a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59434/0 (socket says 172.21.15.7:59434) 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.694+0000 7f5997fff700 1 -- 172.21.15.7:0/1144962335 learned_addr learned my addr 172.21.15.7:0/1144962335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.695+0000 7f5997fff700 1 -- 172.21.15.7:0/1144962335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599807a8a0 con 0x7f599807bd30 2024-01-11T17:50:02.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.695+0000 7f5997fff700 1 --2- 172.21.15.7:0/1144962335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599807a190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f59880098a0 tx=0x7f5988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf44c3b01a0eb77 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:02.954 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: Generating public/private rsa key pair. 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: Your identification has been saved in /tmp/tmpwl5ahwk2/key. 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: Your public key has been saved in /tmp/tmpwl5ahwk2/key.pub. 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: The key fingerprint is: 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: SHA256:if/bEDx7f5PnlYjCTZ8z5c8d0maBrHdOZggd66+OCis ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: The key's randomart image is: 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: +---[RSA 3072]----+ 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | | 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | | 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | . | 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | . o o + | 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | . S +o = o | 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | .. o+* B o| 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | ..oo+.@ %o| 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | E o..+o.&+B| 2024-01-11T17:50:02.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: | .. .+oooo=B| 2024-01-11T17:50:02.956 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: +----[SHA256]-----+ 2024-01-11T17:50:02.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.695+0000 7f5996ffd700 1 -- 172.21.15.7:0/1144962335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5988004160 con 0x7f599807bd30 2024-01-11T17:50:02.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.695+0000 7f5996ffd700 1 -- 172.21.15.7:0/1144962335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59880042c0 con 0x7f599807bd30 2024-01-11T17:50:02.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.695+0000 7f5996ffd700 1 -- 172.21.15.7:0/1144962335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5988011a00 con 0x7f599807bd30 2024-01-11T17:50:02.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 -- 172.21.15.7:0/1144962335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 msgr2=0x7f599807a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 --2- 172.21.15.7:0/1144962335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599807a190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f59880098a0 tx=0x7f5988009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 -- 172.21.15.7:0/1144962335 shutdown_connections 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 --2- 172.21.15.7:0/1144962335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599807a190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 -- 172.21.15.7:0/1144962335 >> 172.21.15.7:0/1144962335 conn(0x7f5998102470 msgr2=0x7f5998104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 -- 172.21.15.7:0/1144962335 shutdown_connections 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.696+0000 7f599e60c700 1 -- 172.21.15.7:0/1144962335 wait complete. 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.697+0000 7f599e60c700 1 Processor -- start 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.697+0000 7f599e60c700 1 -- start start 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.697+0000 7f599e60c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.697+0000 7f599e60c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599819d2e0 con 0x7f599807bd30 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.697+0000 7f5997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.697+0000 7f5997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59442/0 (socket says 172.21.15.7:59442) 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f5997fff700 1 -- 172.21.15.7:0/1036803708 learned_addr learned my addr 172.21.15.7:0/1036803708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:02.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f5997fff700 1 -- 172.21.15.7:0/1036803708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5988009550 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f5997fff700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599819cda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5988011a20 tx=0x7f5988011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5988003af0 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5988003c50 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5988019760 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599819d4e0 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.698+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599819d980 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.699+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f598801ab20 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.699+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599810d1e0 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.700+0000 7f59957fa700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f5980038e60 0x7f598003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.700+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f598804d1e0 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.700+0000 7f59977fe700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f5980038e60 0x7f598003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.701+0000 7f59977fe700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f5980038e60 0x7f598003b320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f598c006fd0 tx=0x7f598c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.705+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f598801a7b0 con 0x7f599807bd30 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:01.920+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f59981036b0 con 0x7f5980038e60 2024-01-11T17:50:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.680+0000 7f59957fa700 1 -- 172.21.15.7:0/1036803708 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f59981036b0 con 0x7f5980038e60 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f5980038e60 msgr2=0x7f598003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f5980038e60 0x7f598003b320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f598c006fd0 tx=0x7f598c006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 msgr2=0x7f599819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599819cda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5988011a20 tx=0x7f5988011b00 comp rx=0 tx=0).stop 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 shutdown_connections 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f5980038e60 0x7f598003b320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 --2- 172.21.15.7:0/1036803708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599807bd30 0x7f599819cda0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 >> 172.21.15.7:0/1036803708 conn(0x7f5998102470 msgr2=0x7f5998102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.682+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 shutdown_connections 2024-01-11T17:50:02.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:02.683+0000 7f599e60c700 1 -- 172.21.15.7:0/1036803708 wait complete. 2024-01-11T17:50:03.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:03 smithi007 ceph-mon[33361]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:03.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:03 smithi007 ceph-mon[33361]: Generating ssh key... 2024-01-11T17:50:03.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:03 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:03.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:03 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:03.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:50:03.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a24f3700 1 Processor -- start 2024-01-11T17:50:03.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a24f3700 1 -- start start 2024-01-11T17:50:03.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a24f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a24f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c07a780 con 0x7fd69c07bd50 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a14f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a14f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c07a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49392/0 (socket says 172.21.15.7:49392) 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.352+0000 7fd6a14f1700 1 -- 172.21.15.7:0/3891318046 learned_addr learned my addr 172.21.15.7:0/3891318046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.353+0000 7fd6a14f1700 1 -- 172.21.15.7:0/3891318046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c07a8c0 con 0x7fd69c07bd50 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.353+0000 7fd6a14f1700 1 --2- 172.21.15.7:0/3891318046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c07a1b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6980098a0 tx=0x7fd698009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d909cc5b406b5411 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.353+0000 7fd693fff700 1 -- 172.21.15.7:0/3891318046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd698004160 con 0x7fd69c07bd50 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.353+0000 7fd693fff700 1 -- 172.21.15.7:0/3891318046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6980042c0 con 0x7fd69c07bd50 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.353+0000 7fd693fff700 1 -- 172.21.15.7:0/3891318046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd698011a00 con 0x7fd69c07bd50 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3891318046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 msgr2=0x7fd69c07a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 --2- 172.21.15.7:0/3891318046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c07a1b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6980098a0 tx=0x7fd698009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3891318046 shutdown_connections 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 --2- 172.21.15.7:0/3891318046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c07a1b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3891318046 >> 172.21.15.7:0/3891318046 conn(0x7fd69c102240 msgr2=0x7fd69c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:03.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3891318046 shutdown_connections 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.354+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3891318046 wait complete. 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.355+0000 7fd6a24f3700 1 Processor -- start 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.355+0000 7fd6a24f3700 1 -- start start 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.355+0000 7fd6a24f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.355+0000 7fd6a24f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c19d0f0 con 0x7fd69c07bd50 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.356+0000 7fd6a14f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.356+0000 7fd6a14f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49400/0 (socket says 172.21.15.7:49400) 2024-01-11T17:50:03.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.356+0000 7fd6a14f1700 1 -- 172.21.15.7:0/3386790667 learned_addr learned my addr 172.21.15.7:0/3386790667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:03.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.356+0000 7fd6a14f1700 1 -- 172.21.15.7:0/3386790667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009550 con 0x7fd69c07bd50 2024-01-11T17:50:03.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.356+0000 7fd6a14f1700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c19cbb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd698011a20 tx=0x7fd698011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:03.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.357+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd698003af0 con 0x7fd69c07bd50 2024-01-11T17:50:03.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.357+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd698003c50 con 0x7fd69c07bd50 2024-01-11T17:50:03.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.357+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd698019760 con 0x7fd69c07bd50 2024-01-11T17:50:03.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.357+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c19d2f0 con 0x7fd69c07bd50 2024-01-11T17:50:03.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.357+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c19d790 con 0x7fd69c07bd50 2024-01-11T17:50:03.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.358+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c10d010 con 0x7fd69c07bd50 2024-01-11T17:50:03.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.358+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7fd69801ab20 con 0x7fd69c07bd50 2024-01-11T17:50:03.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.358+0000 7fd6927fc700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd688038e60 0x7fd68803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:03.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.358+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd69804d1b0 con 0x7fd69c07bd50 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.359+0000 7fd6a0cf0700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd688038e60 0x7fd68803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.359+0000 7fd6a0cf0700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd688038e60 0x7fd68803b320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd68c006fd0 tx=0x7fd68c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.363+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd69802db90 con 0x7fd69c07bd50 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.578+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd69c1034b0 con 0x7fd688038e60 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.579+0000 7fd6927fc700 1 -- 172.21.15.7:0/3386790667 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd69c1034b0 con 0x7fd688038e60 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd688038e60 msgr2=0x7fd68803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd688038e60 0x7fd68803b320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd68c006fd0 tx=0x7fd68c006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:03.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 msgr2=0x7fd69c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:03.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c19cbb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd698011a20 tx=0x7fd698011b00 comp rx=0 tx=0).stop 2024-01-11T17:50:03.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 shutdown_connections 2024-01-11T17:50:03.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd688038e60 0x7fd68803b320 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:03.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 --2- 172.21.15.7:0/3386790667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd69c07bd50 0x7fd69c19cbb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:03.929 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 >> 172.21.15.7:0/3386790667 conn(0x7fd69c102240 msgr2=0x7fd69c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:03.929 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 shutdown_connections 2024-01-11T17:50:03.929 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:03.581+0000 7fd6a24f3700 1 -- 172.21.15.7:0/3386790667 wait complete. 2024-01-11T17:50:03.929 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-01-11T17:50:03.929 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-01-11T17:50:03.929 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-01-11T17:50:05.258 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:05 smithi007 ceph-mon[33361]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:06.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:06 smithi007 ceph-mon[33361]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:07.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:07 smithi007 ceph-mon[33361]: Deploying cephadm binary to smithi007 2024-01-11T17:50:09.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-01-11T17:50:09.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.340+0000 7fbe876fd700 1 Processor -- start 2024-01-11T17:50:09.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.340+0000 7fbe876fd700 1 -- start start 2024-01-11T17:50:09.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.340+0000 7fbe876fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8007a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:09.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.340+0000 7fbe876fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8007a7a0 con 0x7fbe8007bd70 2024-01-11T17:50:09.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.340+0000 7fbe85499700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8007a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:09.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe85499700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8007a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49402/0 (socket says 172.21.15.7:49402) 2024-01-11T17:50:09.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe85499700 1 -- 172.21.15.7:0/792767203 learned_addr learned my addr 172.21.15.7:0/792767203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:09.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe85499700 1 -- 172.21.15.7:0/792767203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8007a8e0 con 0x7fbe8007bd70 2024-01-11T17:50:09.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe85499700 1 --2- 172.21.15.7:0/792767203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8007a1d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0098a0 tx=0x7fbe7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a453f3df761292db server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:09.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe77fff700 1 -- 172.21.15.7:0/792767203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe7c004160 con 0x7fbe8007bd70 2024-01-11T17:50:09.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe77fff700 1 -- 172.21.15.7:0/792767203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe7c0042c0 con 0x7fbe8007bd70 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.341+0000 7fbe77fff700 1 -- 172.21.15.7:0/792767203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe7c011a00 con 0x7fbe8007bd70 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 -- 172.21.15.7:0/792767203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 msgr2=0x7fbe8007a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 --2- 172.21.15.7:0/792767203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8007a1d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0098a0 tx=0x7fbe7c009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 -- 172.21.15.7:0/792767203 shutdown_connections 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 --2- 172.21.15.7:0/792767203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8007a1d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 -- 172.21.15.7:0/792767203 >> 172.21.15.7:0/792767203 conn(0x7fbe80102460 msgr2=0x7fbe80104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 -- 172.21.15.7:0/792767203 shutdown_connections 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.342+0000 7fbe876fd700 1 -- 172.21.15.7:0/792767203 wait complete. 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.344+0000 7fbe876fd700 1 Processor -- start 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.344+0000 7fbe876fd700 1 -- start start 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.344+0000 7fbe876fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:09.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.344+0000 7fbe876fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8019d2d0 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe85499700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe85499700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49408/0 (socket says 172.21.15.7:49408) 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe85499700 1 -- 172.21.15.7:0/2250610999 learned_addr learned my addr 172.21.15.7:0/2250610999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe85499700 1 -- 172.21.15.7:0/2250610999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c009550 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe85499700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8019cd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009fb0 tx=0x7fbe7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe7c003a40 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe7c003ba0 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe7c01a750 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.345+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8019d4d0 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.346+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8019d970 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.347+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7fbe7c024b60 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.347+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8010d1d0 con 0x7fbe8007bd70 2024-01-11T17:50:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.347+0000 7fbe767fc700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbe6c038e10 0x7fbe6c03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.347+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe7c04cf40 con 0x7fbe8007bd70 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.347+0000 7fbe84c98700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbe6c038e10 0x7fbe6c03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.348+0000 7fbe84c98700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbe6c038e10 0x7fbe6c03b2d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbe70006fd0 tx=0x7fbe70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.352+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fbe7c024430 con 0x7fbe8007bd70 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:04.566+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe80103720 con 0x7fbe6c038e10 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.325+0000 7fbe767fc700 1 -- 172.21.15.7:0/2250610999 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fbe80103720 con 0x7fbe6c038e10 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbe6c038e10 msgr2=0x7fbe6c03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbe6c038e10 0x7fbe6c03b2d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbe70006fd0 tx=0x7fbe70006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 msgr2=0x7fbe8019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8019cd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009fb0 tx=0x7fbe7c011ad0 comp rx=0 tx=0).stop 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 shutdown_connections 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fbe6c038e10 0x7fbe6c03b2d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 --2- 172.21.15.7:0/2250610999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8007bd70 0x7fbe8019cd90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:09.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.328+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 >> 172.21.15.7:0/2250610999 conn(0x7fbe80102460 msgr2=0x7fbe80103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:09.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.329+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 shutdown_connections 2024-01-11T17:50:09.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:09.329+0000 7fbe876fd700 1 -- 172.21.15.7:0/2250610999 wait complete. 2024-01-11T17:50:09.643 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-01-11T17:50:10.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.058+0000 7fa64f18b700 1 Processor -- start 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.059+0000 7fa64f18b700 1 -- start start 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.059+0000 7fa64f18b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64807a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.059+0000 7fa64f18b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64807a7a0 con 0x7fa64807bd70 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.059+0000 7fa64cf27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64807a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.059+0000 7fa64cf27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64807a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49412/0 (socket says 172.21.15.7:49412) 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.059+0000 7fa64cf27700 1 -- 172.21.15.7:0/766263695 learned_addr learned my addr 172.21.15.7:0/766263695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.060+0000 7fa64cf27700 1 -- 172.21.15.7:0/766263695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64807a8e0 con 0x7fa64807bd70 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.060+0000 7fa64cf27700 1 --2- 172.21.15.7:0/766263695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64807a1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0098a0 tx=0x7fa63c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af7ea00286842f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.060+0000 7fa6477fe700 1 -- 172.21.15.7:0/766263695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c004160 con 0x7fa64807bd70 2024-01-11T17:50:10.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.060+0000 7fa6477fe700 1 -- 172.21.15.7:0/766263695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa63c0042c0 con 0x7fa64807bd70 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.060+0000 7fa6477fe700 1 -- 172.21.15.7:0/766263695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c011a00 con 0x7fa64807bd70 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 -- 172.21.15.7:0/766263695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 msgr2=0x7fa64807a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 --2- 172.21.15.7:0/766263695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64807a1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0098a0 tx=0x7fa63c009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 -- 172.21.15.7:0/766263695 shutdown_connections 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 --2- 172.21.15.7:0/766263695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64807a1d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 -- 172.21.15.7:0/766263695 >> 172.21.15.7:0/766263695 conn(0x7fa648102460 msgr2=0x7fa648104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 -- 172.21.15.7:0/766263695 shutdown_connections 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.061+0000 7fa64f18b700 1 -- 172.21.15.7:0/766263695 wait complete. 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.062+0000 7fa64f18b700 1 Processor -- start 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.062+0000 7fa64f18b700 1 -- start start 2024-01-11T17:50:10.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64f18b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:10.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:10 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:10.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:10 smithi007 ceph-mon[33361]: Added host smithi007 2024-01-11T17:50:10.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:10 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:10.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:10 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:10.642 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64f18b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819d2d0 con 0x7fa64807bd70 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64cf27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64cf27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49422/0 (socket says 172.21.15.7:49422) 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64cf27700 1 -- 172.21.15.7:0/3368021362 learned_addr learned my addr 172.21.15.7:0/3368021362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64cf27700 1 -- 172.21.15.7:0/3368021362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c009550 con 0x7fa64807bd70 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.063+0000 7fa64cf27700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64819cd90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009130 tx=0x7fa63c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.064+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c003ab0 con 0x7fa64807bd70 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.064+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa63c003c10 con 0x7fa64807bd70 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.064+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64819d4d0 con 0x7fa64807bd70 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.064+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c0247d0 con 0x7fa64807bd70 2024-01-11T17:50:10.643 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.064+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64819d970 con 0x7fa64807bd70 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.065+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64810d1d0 con 0x7fa64807bd70 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.065+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7fa63c01abb0 con 0x7fa64807bd70 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.065+0000 7fa645ffb700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fa638038e60 0x7fa63803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.066+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa63c04cfc0 con 0x7fa64807bd70 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.066+0000 7fa647fff700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fa638038e60 0x7fa63803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.066+0000 7fa647fff700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fa638038e60 0x7fa63803b320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa634006fd0 tx=0x7fa634006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.071+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa63c018920 con 0x7fa64807bd70 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.293+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa648103720 con 0x7fa638038e60 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.298+0000 7fa645ffb700 1 -- 172.21.15.7:0/3368021362 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa648103720 con 0x7fa638038e60 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fa638038e60 msgr2=0x7fa63803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:10.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fa638038e60 0x7fa63803b320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa634006fd0 tx=0x7fa634006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 msgr2=0x7fa64819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64819cd90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009130 tx=0x7fa63c011ad0 comp rx=0 tx=0).stop 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 shutdown_connections 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fa638038e60 0x7fa63803b320 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 --2- 172.21.15.7:0/3368021362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64807bd70 0x7fa64819cd90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 >> 172.21.15.7:0/3368021362 conn(0x7fa648102460 msgr2=0x7fa648103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 shutdown_connections 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:10.301+0000 7fa64f18b700 1 -- 172.21.15.7:0/3368021362 wait complete. 2024-01-11T17:50:10.645 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.078+0000 7f84ab962700 1 Processor -- start 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.078+0000 7f84ab962700 1 -- start start 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.079+0000 7f84ab962700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a4109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.079+0000 7f84ab962700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a4074bc0 con 0x7f84a41079a0 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.079+0000 7f84a96fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a4109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:11.644 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.079+0000 7f84a96fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a4109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49430/0 (socket says 172.21.15.7:49430) 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.079+0000 7f84a96fe700 1 -- 172.21.15.7:0/707224903 learned_addr learned my addr 172.21.15.7:0/707224903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.079+0000 7f84a96fe700 1 -- 172.21.15.7:0/707224903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4074d00 con 0x7f84a41079a0 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.080+0000 7f84a96fe700 1 --2- 172.21.15.7:0/707224903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a4109d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84a00098a0 tx=0x7f84a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb50e9748d1408f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.080+0000 7f849bfff700 1 -- 172.21.15.7:0/707224903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a0004160 con 0x7f84a41079a0 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.080+0000 7f849bfff700 1 -- 172.21.15.7:0/707224903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84a00042c0 con 0x7f84a41079a0 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.080+0000 7f849bfff700 1 -- 172.21.15.7:0/707224903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a0011a00 con 0x7f84a41079a0 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.080+0000 7f84ab962700 1 -- 172.21.15.7:0/707224903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 msgr2=0x7f84a4109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.080+0000 7f84ab962700 1 --2- 172.21.15.7:0/707224903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a4109d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84a00098a0 tx=0x7f84a0009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.081+0000 7f84ab962700 1 -- 172.21.15.7:0/707224903 shutdown_connections 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.081+0000 7f84ab962700 1 --2- 172.21.15.7:0/707224903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a4109d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.081+0000 7f84ab962700 1 -- 172.21.15.7:0/707224903 >> 172.21.15.7:0/707224903 conn(0x7f84a4101350 msgr2=0x7f84a4103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.081+0000 7f84ab962700 1 -- 172.21.15.7:0/707224903 shutdown_connections 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.081+0000 7f84ab962700 1 -- 172.21.15.7:0/707224903 wait complete. 2024-01-11T17:50:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.082+0000 7f84ab962700 1 Processor -- start 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.082+0000 7f84ab962700 1 -- start start 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.083+0000 7f84ab962700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.083+0000 7f84ab962700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419b0a0 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.083+0000 7f84a96fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.083+0000 7f84a96fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49440/0 (socket says 172.21.15.7:49440) 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.083+0000 7f84a96fe700 1 -- 172.21.15.7:0/3068432351 learned_addr learned my addr 172.21.15.7:0/3068432351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.083+0000 7f84a96fe700 1 -- 172.21.15.7:0/3068432351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a0009550 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.084+0000 7f84a96fe700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a419ab60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009130 tx=0x7f84a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.084+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a0003b70 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.084+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84a0003cd0 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.084+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a00197d0 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.084+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a419b2a0 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.084+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a419b740 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.085+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f84a001a430 con 0x7f84a41079a0 2024-01-11T17:50:11.646 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.085+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a410af90 con 0x7f84a41079a0 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.086+0000 7f849a7fc700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f8490038e10 0x7f849003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.086+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f84a004d160 con 0x7f84a41079a0 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.086+0000 7f84a8efd700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f8490038e10 0x7f849003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.087+0000 7f84a8efd700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f8490038e10 0x7f849003b2d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8494006fd0 tx=0x7f8494006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.091+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f84a002d960 con 0x7f84a41079a0 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.320+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f84a4102530 con 0x7f8490038e10 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.324+0000 7f849a7fc700 1 -- 172.21.15.7:0/3068432351 <== mgr.14120 v2:172.21.15.7:6800/867047662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f84a4102530 con 0x7f8490038e10 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f8490038e10 msgr2=0x7f849003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f8490038e10 0x7f849003b2d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8494006fd0 tx=0x7f8494006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 msgr2=0x7f84a419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a419ab60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009130 tx=0x7f84a0011ad0 comp rx=0 tx=0).stop 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 shutdown_connections 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f8490038e10 0x7f849003b2d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:11.647 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 --2- 172.21.15.7:0/3068432351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a41079a0 0x7f84a419ab60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:11.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 >> 172.21.15.7:0/3068432351 conn(0x7f84a4101350 msgr2=0x7f84a4101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:11.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 shutdown_connections 2024-01-11T17:50:11.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:11.327+0000 7f84ab962700 1 -- 172.21.15.7:0/3068432351 wait complete. 2024-01-11T17:50:11.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:11 smithi007 ceph-mon[33361]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:11.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:11 smithi007 ceph-mon[33361]: Saving service mon spec with placement count:5 2024-01-11T17:50:12.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:12 smithi007 ceph-mon[33361]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:12.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:12 smithi007 ceph-mon[33361]: Saving service mgr spec with placement count:2 2024-01-11T17:50:12.587 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:12 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f143e9bf700 1 Processor -- start 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f143e9bf700 1 -- start start 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f143e9bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143807a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f143e9bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143807a760 con 0x7f143807bd30 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f1437fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143807a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f1437fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143807a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45754/0 (socket says 172.21.15.7:45754) 2024-01-11T17:50:12.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.178+0000 7f1437fff700 1 -- 172.21.15.7:0/377509363 learned_addr learned my addr 172.21.15.7:0/377509363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:12.712 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.179+0000 7f1437fff700 1 -- 172.21.15.7:0/377509363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143807a8a0 con 0x7f143807bd30 2024-01-11T17:50:12.712 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.179+0000 7f1437fff700 1 --2- 172.21.15.7:0/377509363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143807a190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f1428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eff728bcd767a92 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:12.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.179+0000 7f1436ffd700 1 -- 172.21.15.7:0/377509363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1428004160 con 0x7f143807bd30 2024-01-11T17:50:12.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f1436ffd700 1 -- 172.21.15.7:0/377509363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14280042c0 con 0x7f143807bd30 2024-01-11T17:50:12.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f1436ffd700 1 -- 172.21.15.7:0/377509363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1428011a00 con 0x7f143807bd30 2024-01-11T17:50:12.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 -- 172.21.15.7:0/377509363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 msgr2=0x7f143807a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:12.716 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 --2- 172.21.15.7:0/377509363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143807a190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f1428009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 -- 172.21.15.7:0/377509363 shutdown_connections 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 --2- 172.21.15.7:0/377509363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143807a190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 -- 172.21.15.7:0/377509363 >> 172.21.15.7:0/377509363 conn(0x7f1438102470 msgr2=0x7f1438104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 -- 172.21.15.7:0/377509363 shutdown_connections 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.180+0000 7f143e9bf700 1 -- 172.21.15.7:0/377509363 wait complete. 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.182+0000 7f143e9bf700 1 Processor -- start 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.182+0000 7f143e9bf700 1 -- start start 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.183+0000 7f143e9bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.183+0000 7f143e9bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143819d2e0 con 0x7f143807bd30 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.183+0000 7f1437fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.183+0000 7f1437fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45764/0 (socket says 172.21.15.7:45764) 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.183+0000 7f1437fff700 1 -- 172.21.15.7:0/2326219023 learned_addr learned my addr 172.21.15.7:0/2326219023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.183+0000 7f1437fff700 1 -- 172.21.15.7:0/2326219023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1428009550 con 0x7f143807bd30 2024-01-11T17:50:12.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.184+0000 7f1437fff700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143819cda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1428011a20 tx=0x7f1428011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.184+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1428003af0 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.184+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1428003c50 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.184+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143819d4e0 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.184+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1428019760 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.184+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143819d980 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.185+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f1428019950 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.186+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143810d1e0 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.186+0000 7f14357fa700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f1420038e60 0x7f142003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.186+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f142804d1b0 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.186+0000 7f14377fe700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f1420038e60 0x7f142003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.187+0000 7f14377fe700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f1420038e60 0x7f142003b320 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f142c006fd0 tx=0x7f142c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.192+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1428018ba0 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.418+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f143810d3f0 con 0x7f143807bd30 2024-01-11T17:50:12.718 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.421+0000 7f14357fa700 1 -- 172.21.15.7:0/2326219023 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) v1 ==== 142+0+0 (secure 0 0 0) 0x7f14280187c0 con 0x7f143807bd30 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f1420038e60 msgr2=0x7f142003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f1420038e60 0x7f142003b320 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f142c006fd0 tx=0x7f142c006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 msgr2=0x7f143819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143819cda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1428011a20 tx=0x7f1428011b00 comp rx=0 tx=0).stop 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 shutdown_connections 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f1420038e60 0x7f142003b320 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 --2- 172.21.15.7:0/2326219023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f143807bd30 0x7f143819cda0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 >> 172.21.15.7:0/2326219023 conn(0x7f1438102470 msgr2=0x7f1438102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 shutdown_connections 2024-01-11T17:50:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:12.424+0000 7f143e9bf700 1 -- 172.21.15.7:0/2326219023 wait complete. 2024-01-11T17:50:13.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2326219023' entity='client.admin' 2024-01-11T17:50:13.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:13 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:13.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:13 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:13.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1603781529' entity='client.admin' 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.158+0000 7f24f7a7d700 1 Processor -- start 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.158+0000 7f24f7a7d700 1 -- start start 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.158+0000 7f24f7a7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f007a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.158+0000 7f24f7a7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f007a780 con 0x7f24f007bd50 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.159+0000 7f24f5819700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f007a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.159+0000 7f24f5819700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f007a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45780/0 (socket says 172.21.15.7:45780) 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.159+0000 7f24f5819700 1 -- 172.21.15.7:0/46649108 learned_addr learned my addr 172.21.15.7:0/46649108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.159+0000 7f24f5819700 1 -- 172.21.15.7:0/46649108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f007a8c0 con 0x7f24f007bd50 2024-01-11T17:50:13.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.159+0000 7f24f5819700 1 --2- 172.21.15.7:0/46649108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f007a1b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0098a0 tx=0x7f24ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d8bddba5d09725 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.159+0000 7f24f4817700 1 -- 172.21.15.7:0/46649108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24ec004160 con 0x7f24f007bd50 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f4817700 1 -- 172.21.15.7:0/46649108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24ec0042c0 con 0x7f24f007bd50 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f4817700 1 -- 172.21.15.7:0/46649108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24ec0119b0 con 0x7f24f007bd50 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 -- 172.21.15.7:0/46649108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 msgr2=0x7f24f007a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 --2- 172.21.15.7:0/46649108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f007a1b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0098a0 tx=0x7f24ec009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 -- 172.21.15.7:0/46649108 shutdown_connections 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 --2- 172.21.15.7:0/46649108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f007a1b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 -- 172.21.15.7:0/46649108 >> 172.21.15.7:0/46649108 conn(0x7f24f0102460 msgr2=0x7f24f0104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 -- 172.21.15.7:0/46649108 shutdown_connections 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.160+0000 7f24f7a7d700 1 -- 172.21.15.7:0/46649108 wait complete. 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.161+0000 7f24f7a7d700 1 Processor -- start 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.162+0000 7f24f7a7d700 1 -- start start 2024-01-11T17:50:13.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.162+0000 7f24f7a7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.162+0000 7f24f7a7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f019d260 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.162+0000 7f24f5819700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.162+0000 7f24f5819700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45792/0 (socket says 172.21.15.7:45792) 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.162+0000 7f24f5819700 1 -- 172.21.15.7:0/1603781529 learned_addr learned my addr 172.21.15.7:0/1603781529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24f5819700 1 -- 172.21.15.7:0/1603781529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec009550 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24f5819700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f019cd20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006af0 tx=0x7f24ec011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24ec003a40 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24ec003ba0 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24ec0196c0 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f019d460 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.163+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f019d900 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.165+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f24ec01aa80 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.165+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f010d2a0 con 0x7f24f007bd50 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.165+0000 7f24e6ffd700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f24dc038e10 0x7f24dc03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:13.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.165+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f24ec04ce80 con 0x7f24f007bd50 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.165+0000 7f24f5018700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f24dc038e10 0x7f24dc03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.166+0000 7f24f5018700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f24dc038e10 0x7f24dc03b2d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f24e0006fd0 tx=0x7f24e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.170+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f24ec01a430 con 0x7f24f007bd50 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.381+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f24f010d4b0 con 0x7f24f007bd50 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.383+0000 7f24e6ffd700 1 -- 172.21.15.7:0/1603781529 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f24ec00b3e0 con 0x7f24f007bd50 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f24dc038e10 msgr2=0x7f24dc03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f24dc038e10 0x7f24dc03b2d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f24e0006fd0 tx=0x7f24e0006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 msgr2=0x7f24f019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f019cd20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006af0 tx=0x7f24ec011a10 comp rx=0 tx=0).stop 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 shutdown_connections 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f24dc038e10 0x7f24dc03b2d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 --2- 172.21.15.7:0/1603781529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f007bd50 0x7f24f019cd20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 >> 172.21.15.7:0/1603781529 conn(0x7f24f0102460 msgr2=0x7f24f0102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 shutdown_connections 2024-01-11T17:50:13.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:13.386+0000 7f24f7a7d700 1 -- 172.21.15.7:0/1603781529 wait complete. 2024-01-11T17:50:13.874 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-01-11T17:50:15.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:15 smithi007 ceph-mon[33361]: from='mgr.14120 172.21.15.7:0/3677868654' entity='mgr.x' 2024-01-11T17:50:15.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:15 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1125265406' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-01-11T17:50:15.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: ignoring --setuser ceph since I am not root 2024-01-11T17:50:15.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: ignoring --setgroup ceph since I am not root 2024-01-11T17:50:15.881 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.306+0000 7f2eb70e4700 1 Processor -- start 2024-01-11T17:50:15.881 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.307+0000 7f2eb70e4700 1 -- start start 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.307+0000 7f2eb70e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb0109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.307+0000 7f2eb70e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb01099e0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.307+0000 7f2eb4e80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb0109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.307+0000 7f2eb4e80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb0109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45798/0 (socket says 172.21.15.7:45798) 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.307+0000 7f2eb4e80700 1 -- 172.21.15.7:0/2395698377 learned_addr learned my addr 172.21.15.7:0/2395698377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.308+0000 7f2eb4e80700 1 -- 172.21.15.7:0/2395698377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb010a1f0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.308+0000 7f2eb4e80700 1 --2- 172.21.15.7:0/2395698377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb0109410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40098a0 tx=0x7f2ea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33d37c1433d398b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.308+0000 7f2eaf7fe700 1 -- 172.21.15.7:0/2395698377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4004160 con 0x7f2eb0108ff0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.308+0000 7f2eaf7fe700 1 -- 172.21.15.7:0/2395698377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ea40042c0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.308+0000 7f2eaf7fe700 1 -- 172.21.15.7:0/2395698377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ea40119b0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 -- 172.21.15.7:0/2395698377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 msgr2=0x7f2eb0109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 --2- 172.21.15.7:0/2395698377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb0109410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40098a0 tx=0x7f2ea4009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 -- 172.21.15.7:0/2395698377 shutdown_connections 2024-01-11T17:50:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 --2- 172.21.15.7:0/2395698377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb0109410 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 -- 172.21.15.7:0/2395698377 >> 172.21.15.7:0/2395698377 conn(0x7f2eb007c450 msgr2=0x7f2eb007c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 -- 172.21.15.7:0/2395698377 shutdown_connections 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.309+0000 7f2eb70e4700 1 -- 172.21.15.7:0/2395698377 wait complete. 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.310+0000 7f2eb70e4700 1 Processor -- start 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.310+0000 7f2eb70e4700 1 -- start start 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.310+0000 7f2eb70e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.310+0000 7f2eb70e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb019d290 con 0x7f2eb0108ff0 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.311+0000 7f2eb4e80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.311+0000 7f2eb4e80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45804/0 (socket says 172.21.15.7:45804) 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.311+0000 7f2eb4e80700 1 -- 172.21.15.7:0/1125265406 learned_addr learned my addr 172.21.15.7:0/1125265406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.311+0000 7f2eb4e80700 1 -- 172.21.15.7:0/1125265406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea4009550 con 0x7f2eb0108ff0 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.311+0000 7f2eb4e80700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb019cd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4003e60 tx=0x7f2ea4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.312+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4003a60 con 0x7f2eb0108ff0 2024-01-11T17:50:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.312+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ea4003bc0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.312+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb019d490 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.312+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ea40196c0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.312+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb019d930 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.313+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 45879+0+0 (secure 0 0 0) 0x7f2ea401a430 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.313+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb010d420 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.313+0000 7f2eadffb700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f2ea0038e10 0x7f2ea003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.313+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2ea404d150 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.314+0000 7f2eaffff700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f2ea0038e10 0x7f2ea003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.314+0000 7f2eaffff700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f2ea0038e10 0x7f2ea003b2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c006fd0 tx=0x7f2e9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.320+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2ea4010560 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:14.595+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2eb01a0210 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.505+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2ea40281c0 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.506+0000 7f2eadffb700 1 -- 172.21.15.7:0/1125265406 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) v1 ==== 45892+0+0 (secure 0 0 0) 0x7f2ea401ac10 con 0x7f2eb0108ff0 2024-01-11T17:50:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f2ea0038e10 msgr2=0x7f2ea003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f2ea0038e10 0x7f2ea003b2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c006fd0 tx=0x7f2e9c006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 msgr2=0x7f2eb019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb019cd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4003e60 tx=0x7f2ea4011a30 comp rx=0 tx=0).stop 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 shutdown_connections 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7f2ea0038e10 0x7f2ea003b2d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 --2- 172.21.15.7:0/1125265406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2eb0108ff0 0x7f2eb019cd50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.509+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 >> 172.21.15.7:0/1125265406 conn(0x7f2eb007c450 msgr2=0x7f2eb0106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.510+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 shutdown_connections 2024-01-11T17:50:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:15.510+0000 7f2eb70e4700 1 -- 172.21.15.7:0/1125265406 wait complete. 2024-01-11T17:50:16.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:16.209+0000 7fdc786d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T17:50:16.475 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:16.346+0000 7fdc786d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T17:50:16.476 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:16.474+0000 7fdc786d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T17:50:16.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:16 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1125265406' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-01-11T17:50:16.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:16 smithi007 ceph-mon[33361]: mgrmap e9: x(active, since 17s) 2024-01-11T17:50:16.784 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:16.591+0000 7fdc786d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T17:50:16.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.316+0000 7fc7d9b7c700 1 Processor -- start 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.316+0000 7fc7d9b7c700 1 -- start start 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.317+0000 7fc7d9b7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d407a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.317+0000 7fc7d9b7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d407a760 con 0x7fc7d407bd30 2024-01-11T17:50:16.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.317+0000 7fc7d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d407a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.317+0000 7fc7d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d407a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45834/0 (socket says 172.21.15.7:45834) 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.317+0000 7fc7d37fe700 1 -- 172.21.15.7:0/2427276575 learned_addr learned my addr 172.21.15.7:0/2427276575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.318+0000 7fc7d37fe700 1 -- 172.21.15.7:0/2427276575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d407a8a0 con 0x7fc7d407bd30 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.318+0000 7fc7d37fe700 1 --2- 172.21.15.7:0/2427276575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d407a190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40098a0 tx=0x7fc7c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bad2193f8736ac server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.318+0000 7fc7d27fc700 1 -- 172.21.15.7:0/2427276575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7c4004160 con 0x7fc7d407bd30 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.318+0000 7fc7d27fc700 1 -- 172.21.15.7:0/2427276575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7c40042c0 con 0x7fc7d407bd30 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.318+0000 7fc7d27fc700 1 -- 172.21.15.7:0/2427276575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7c40119b0 con 0x7fc7d407bd30 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2427276575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 msgr2=0x7fc7d407a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 --2- 172.21.15.7:0/2427276575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d407a190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40098a0 tx=0x7fc7c4009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2427276575 shutdown_connections 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 --2- 172.21.15.7:0/2427276575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d407a190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2427276575 >> 172.21.15.7:0/2427276575 conn(0x7fc7d4102470 msgr2=0x7fc7d4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2427276575 shutdown_connections 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.319+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2427276575 wait complete. 2024-01-11T17:50:16.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.320+0000 7fc7d9b7c700 1 Processor -- start 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.320+0000 7fc7d9b7c700 1 -- start start 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.321+0000 7fc7d9b7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.321+0000 7fc7d9b7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d419d300 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.321+0000 7fc7d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.321+0000 7fc7d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45848/0 (socket says 172.21.15.7:45848) 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.321+0000 7fc7d37fe700 1 -- 172.21.15.7:0/2571983597 learned_addr learned my addr 172.21.15.7:0/2571983597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.321+0000 7fc7d37fe700 1 -- 172.21.15.7:0/2571983597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009550 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.322+0000 7fc7d37fe700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d419cdc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4011980 tx=0x7fc7c4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.322+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7c4003af0 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.322+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7c4003c50 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.322+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d419d500 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.322+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7c4019760 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.322+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d419d9a0 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.323+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 45892+0+0 (secure 0 0 0) 0x7fc7c40199a0 con 0x7fc7d407bd30 2024-01-11T17:50:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.323+0000 7fc7d0ff9700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 0x7fc7bc03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.323+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d410d290 con 0x7fc7d407bd30 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.324+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc7c404d2a0 con 0x7fc7d407bd30 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.324+0000 7fc7d2ffd700 1 -- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 msgr2=0x7fc7bc03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.324+0000 7fc7d2ffd700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 0x7fc7bc03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.329+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc7c402d7c0 con 0x7fc7d407bd30 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.525+0000 7fc7d2ffd700 1 -- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 msgr2=0x7fc7bc03b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.525+0000 7fc7d2ffd700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 0x7fc7bc03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.615+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc7d41a0280 con 0x7fc7d407bd30 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.616+0000 7fc7d0ff9700 1 -- 172.21.15.7:0/2571983597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc7c404a070 con 0x7fc7d407bd30 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 msgr2=0x7fc7bc03b320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 0x7fc7bc03b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 msgr2=0x7fc7d419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d419cdc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4011980 tx=0x7fc7c4011a60 comp rx=0 tx=0).stop 2024-01-11T17:50:16.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 shutdown_connections 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fc7bc038e60 0x7fc7bc03b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 --2- 172.21.15.7:0/2571983597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7d407bd30 0x7fc7d419cdc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 >> 172.21.15.7:0/2571983597 conn(0x7fc7d4102470 msgr2=0x7fc7d4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 shutdown_connections 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:16.619+0000 7fc7d9b7c700 1 -- 172.21.15.7:0/2571983597 wait complete. 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-01-11T17:50:16.961 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-01-11T17:50:17.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:17 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2571983597' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-01-11T17:50:17.779 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:17.464+0000 7fdc786d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:50:18.039 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:17.777+0000 7fdc786d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T17:50:18.039 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:17.893+0000 7fdc786d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T17:50:18.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:18.038+0000 7fdc786d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T17:50:18.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:18.239+0000 7fdc786d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T17:50:18.820 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:18.415+0000 7fdc786d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T17:50:18.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:18.531+0000 7fdc786d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T17:50:19.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:19.015+0000 7fdc786d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T17:50:19.746 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:19.379+0000 7fdc786d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T17:50:20.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:19.745+0000 7fdc786d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T17:50:20.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:19.873+0000 7fdc786d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T17:50:21.204 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:20 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:20.874+0000 7fdc786d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T17:50:21.205 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:21.002+0000 7fdc786d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T17:50:21.205 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:21.203+0000 7fdc786d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T17:50:22.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:22.049+0000 7fdc786d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T17:50:22.321 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:22.296+0000 7fdc786d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T17:50:22.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:22.563+0000 7fdc786d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T17:50:24.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:23.732+0000 7fdc786d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T17:50:24.495 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:24.103+0000 7fdc786d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:50:24.495 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:24.226+0000 7fdc786d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T17:50:24.821 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:50:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:50:24.494+0000 7fdc786d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T17:50:25.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: Active manager daemon x restarted 2024-01-11T17:50:25.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: Activating manager daemon x 2024-01-11T17:50:25.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: osdmap e3: 0 total, 0 up, 0 in 2024-01-11T17:50:25.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: mgrmap e10: x(active, starting, since 0.00396813s) 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: Manager daemon x is now available 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-11T17:50:25.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-11T17:50:25.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-01-11T17:50:25.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.410+0000 7fd9df73a700 1 Processor -- start 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.410+0000 7fd9df73a700 1 -- start start 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.410+0000 7fd9df73a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d807a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.410+0000 7fd9df73a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d807a760 con 0x7fd9d807bd30 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.411+0000 7fd9dd4d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d807a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.411+0000 7fd9dd4d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d807a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45862/0 (socket says 172.21.15.7:45862) 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.411+0000 7fd9dd4d6700 1 -- 172.21.15.7:0/1721214548 learned_addr learned my addr 172.21.15.7:0/1721214548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.411+0000 7fd9dd4d6700 1 -- 172.21.15.7:0/1721214548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d807a8a0 con 0x7fd9d807bd30 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.412+0000 7fd9dd4d6700 1 --2- 172.21.15.7:0/1721214548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d807a190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098a0 tx=0x7fd9d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272cd53f44158cb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.412+0000 7fd9cffff700 1 -- 172.21.15.7:0/1721214548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d4004160 con 0x7fd9d807bd30 2024-01-11T17:50:25.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.412+0000 7fd9cffff700 1 -- 172.21.15.7:0/1721214548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d40042c0 con 0x7fd9d807bd30 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.412+0000 7fd9cffff700 1 -- 172.21.15.7:0/1721214548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d40119b0 con 0x7fd9d807bd30 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.412+0000 7fd9df73a700 1 -- 172.21.15.7:0/1721214548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 msgr2=0x7fd9d807a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.412+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1721214548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d807a190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098a0 tx=0x7fd9d4009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.413+0000 7fd9df73a700 1 -- 172.21.15.7:0/1721214548 shutdown_connections 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.413+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1721214548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d807a190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.413+0000 7fd9df73a700 1 -- 172.21.15.7:0/1721214548 >> 172.21.15.7:0/1721214548 conn(0x7fd9d8102470 msgr2=0x7fd9d8104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.413+0000 7fd9df73a700 1 -- 172.21.15.7:0/1721214548 shutdown_connections 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.413+0000 7fd9df73a700 1 -- 172.21.15.7:0/1721214548 wait complete. 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.414+0000 7fd9df73a700 1 Processor -- start 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.414+0000 7fd9df73a700 1 -- start start 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.414+0000 7fd9df73a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d810a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.414+0000 7fd9df73a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d810a6a0 con 0x7fd9d807bd30 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9dd4d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d810a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:25.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9dd4d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d810a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45872/0 (socket says 172.21.15.7:45872) 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9dd4d6700 1 -- 172.21.15.7:0/1807112902 learned_addr learned my addr 172.21.15.7:0/1807112902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9dd4d6700 1 -- 172.21.15.7:0/1807112902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4009550 con 0x7fd9d807bd30 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9dd4d6700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d810a160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4011980 tx=0x7fd9d4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d4003af0 con 0x7fd9d807bd30 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d4003c50 con 0x7fd9d807bd30 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d810a8a0 con 0x7fd9d807bd30 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.415+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d4019760 con 0x7fd9d807bd30 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.416+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d8106ca0 con 0x7fd9d807bd30 2024-01-11T17:50:25.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.417+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 45892+0+0 (secure 0 0 0) 0x7fd9d401ab10 con 0x7fd9d807bd30 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.417+0000 7fd9ce7fc700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.418+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 --> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9c403ba30 con 0x7fd9c4038e60 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.418+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd9d404d320 con 0x7fd9d807bd30 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.418+0000 7fd9dccd5700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.418+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.619+0000 7fd9dccd5700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:17.619+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-11T17:50:25.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:18.020+0000 7fd9dccd5700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:18.020+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:18.821+0000 7fd9dccd5700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:18.821+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:20.422+0000 7fd9dccd5700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:20.422+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:23.626+0000 7fd9dccd5700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/867047662 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:23.626+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:24.498+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) v1 ==== 45659+0+0 (secure 0 0 0) 0x7fd9d4019d20 con 0x7fd9d807bd30 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:24.498+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 msgr2=0x7fd9c403b320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:50:25.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:24.498+0000 7fd9ce7fc700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.501+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) v1 ==== 45786+0+0 (secure 0 0 0) 0x7fd9d404f010 con 0x7fd9d807bd30 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.501+0000 7fd9ce7fc700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd9c403c8d0 0x7fd9c403ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.502+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9c403ba30 con 0x7fd9c403c8d0 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.503+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd9c403c8d0 0x7fd9c403ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.504+0000 7fd9dccd5700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd9c403c8d0 0x7fd9c403ecc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8003a10 tx=0x7fd9c80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.505+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fd9c403ba30 con 0x7fd9c403c8d0 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.519+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd9d810d290 con 0x7fd9c403c8d0 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.519+0000 7fd9ce7fc700 1 -- 172.21.15.7:0/1807112902 <== mgr.14152 v2:172.21.15.7:6800/314922909 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd9d810d290 con 0x7fd9c403c8d0 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd9c403c8d0 msgr2=0x7fd9c403ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd9c403c8d0 0x7fd9c403ecc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8003a10 tx=0x7fd9c80092b0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 msgr2=0x7fd9d810a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d810a160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4011980 tx=0x7fd9d4011a60 comp rx=0 tx=0).stop 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 shutdown_connections 2024-01-11T17:50:25.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d807bd30 0x7fd9d810a160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd9c403c8d0 0x7fd9c403ecc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 --2- 172.21.15.7:0/1807112902 >> [v2:172.21.15.7:6800/867047662,v1:172.21.15.7:6801/867047662] conn(0x7fd9c4038e60 0x7fd9c403b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 >> 172.21.15.7:0/1807112902 conn(0x7fd9d8102470 msgr2=0x7fd9d8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 shutdown_connections 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:25.520+0000 7fd9df73a700 1 -- 172.21.15.7:0/1807112902 wait complete. 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-01-11T17:50:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-01-11T17:50:26.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:26 smithi007 ceph-mon[33361]: mgrmap e11: x(active, since 1.00869s) 2024-01-11T17:50:26.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:26 smithi007 ceph-mon[33361]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-01-11T17:50:26.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:26 smithi007 ceph-mon[33361]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-01-11T17:50:26.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:27.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.392+0000 7fb5c1132700 1 Processor -- start 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.392+0000 7fb5c1132700 1 -- start start 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.393+0000 7fb5c1132700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.393+0000 7fb5c1132700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc07a780 con 0x7fb5bc07bd50 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.393+0000 7fb5bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.394+0000 7fb5bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc07a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46570/0 (socket says 172.21.15.7:46570) 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.394+0000 7fb5bad9d700 1 -- 172.21.15.7:0/3055351400 learned_addr learned my addr 172.21.15.7:0/3055351400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.394+0000 7fb5bad9d700 1 -- 172.21.15.7:0/3055351400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc07a8c0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.394+0000 7fb5bad9d700 1 --2- 172.21.15.7:0/3055351400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc07a1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098a0 tx=0x7fb5ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dae78089bdc102c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5b9d9b700 1 -- 172.21.15.7:0/3055351400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac004160 con 0x7fb5bc07bd50 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5b9d9b700 1 -- 172.21.15.7:0/3055351400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5ac0042c0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5b9d9b700 1 -- 172.21.15.7:0/3055351400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac0119b0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5c1132700 1 -- 172.21.15.7:0/3055351400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 msgr2=0x7fb5bc07a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5c1132700 1 --2- 172.21.15.7:0/3055351400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc07a1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098a0 tx=0x7fb5ac009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5c1132700 1 -- 172.21.15.7:0/3055351400 shutdown_connections 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5c1132700 1 --2- 172.21.15.7:0/3055351400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc07a1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5c1132700 1 -- 172.21.15.7:0/3055351400 >> 172.21.15.7:0/3055351400 conn(0x7fb5bc102470 msgr2=0x7fb5bc104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.395+0000 7fb5c1132700 1 -- 172.21.15.7:0/3055351400 shutdown_connections 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.396+0000 7fb5c1132700 1 -- 172.21.15.7:0/3055351400 wait complete. 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5c1132700 1 Processor -- start 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5c1132700 1 -- start start 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5c1132700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5c1132700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc19d300 con 0x7fb5bc07bd50 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5bad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46578/0 (socket says 172.21.15.7:46578) 2024-01-11T17:50:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.397+0000 7fb5bad9d700 1 -- 172.21.15.7:0/221409643 learned_addr learned my addr 172.21.15.7:0/221409643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.398+0000 7fb5bad9d700 1 -- 172.21.15.7:0/221409643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac009550 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.398+0000 7fb5bad9d700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc19cdc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011980 tx=0x7fb5ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.398+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac003af0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.398+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5ac003c50 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.398+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac019760 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.398+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc19d500 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.399+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc19d9a0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.400+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) v1 ==== 45786+0+0 (secure 0 0 0) 0x7fb5ac01ab10 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.400+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc10d2d0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.400+0000 7fb5a3fff700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5a4038e00 0x7fb5a403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:27.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.400+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb5ac04d1a0 con 0x7fb5bc07bd50 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.401+0000 7fb5ba59c700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5a4038e00 0x7fb5a403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.401+0000 7fb5ba59c700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5a4038e00 0x7fb5a403b2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.406+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb5ac01a430 con 0x7fb5bc07bd50 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.645+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5bc1a02e0 con 0x7fb5a4038e00 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.865+0000 7fb5a3fff700 1 -- 172.21.15.7:0/221409643 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb5bc1a02e0 con 0x7fb5a4038e00 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5a4038e00 msgr2=0x7fb5a403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5a4038e00 0x7fb5a403b2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 msgr2=0x7fb5bc19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc19cdc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac011980 tx=0x7fb5ac011a60 comp rx=0 tx=0).stop 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 shutdown_connections 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5a4038e00 0x7fb5a403b2c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 --2- 172.21.15.7:0/221409643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc07bd50 0x7fb5bc19cdc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.868+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 >> 172.21.15.7:0/221409643 conn(0x7fb5bc102470 msgr2=0x7fb5bc102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.869+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 shutdown_connections 2024-01-11T17:50:27.202 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:26.869+0000 7fb5c1132700 1 -- 172.21.15.7:0/221409643 wait complete. 2024-01-11T17:50:27.203 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-01-11T17:50:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:27 smithi007 ceph-mon[33361]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:27 smithi007 ceph-mon[33361]: mgrmap e12: x(active, since 2s) 2024-01-11T17:50:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:28.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QBaOIEcHchmXtK0T0tTHc.x5ngPLrf9S0fyUyPTtszG3QyNiISNaG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1704995428, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-01-11T17:50:28.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.640+0000 7f36ff089700 1 Processor -- start 2024-01-11T17:50:28.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.640+0000 7f36ff089700 1 -- start start 2024-01-11T17:50:28.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.640+0000 7f36ff089700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f81072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.640+0000 7f36ff089700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81077f0 con 0x7f36f8106e90 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.641+0000 7f36fce25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f81072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.641+0000 7f36fce25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f81072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46584/0 (socket says 172.21.15.7:46584) 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.641+0000 7f36fce25700 1 -- 172.21.15.7:0/3714947130 learned_addr learned my addr 172.21.15.7:0/3714947130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.641+0000 7f36fce25700 1 -- 172.21.15.7:0/3714947130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f8107930 con 0x7f36f8106e90 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.641+0000 7f36fce25700 1 --2- 172.21.15.7:0/3714947130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f81072b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0098a0 tx=0x7f36ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd0476b9ec3130 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.642+0000 7f36f77fe700 1 -- 172.21.15.7:0/3714947130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36ec004160 con 0x7f36f8106e90 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.642+0000 7f36f77fe700 1 -- 172.21.15.7:0/3714947130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ec0042c0 con 0x7f36f8106e90 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.642+0000 7f36f77fe700 1 -- 172.21.15.7:0/3714947130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36ec0119b0 con 0x7f36f8106e90 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.642+0000 7f36ff089700 1 -- 172.21.15.7:0/3714947130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 msgr2=0x7f36f81072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.642+0000 7f36ff089700 1 --2- 172.21.15.7:0/3714947130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f81072b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0098a0 tx=0x7f36ec009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.643+0000 7f36ff089700 1 -- 172.21.15.7:0/3714947130 shutdown_connections 2024-01-11T17:50:28.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.643+0000 7f36ff089700 1 --2- 172.21.15.7:0/3714947130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f81072b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.643+0000 7f36ff089700 1 -- 172.21.15.7:0/3714947130 >> 172.21.15.7:0/3714947130 conn(0x7f36f8102470 msgr2=0x7f36f8104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.643+0000 7f36ff089700 1 -- 172.21.15.7:0/3714947130 shutdown_connections 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.643+0000 7f36ff089700 1 -- 172.21.15.7:0/3714947130 wait complete. 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.644+0000 7f36ff089700 1 Processor -- start 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.644+0000 7f36ff089700 1 -- start start 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.645+0000 7f36ff089700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f807ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.645+0000 7f36ff089700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f807bf80 con 0x7f36f8106e90 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.645+0000 7f36fce25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f807ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.645+0000 7f36fce25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f807ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46600/0 (socket says 172.21.15.7:46600) 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.645+0000 7f36fce25700 1 -- 172.21.15.7:0/1010317884 learned_addr learned my addr 172.21.15.7:0/1010317884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.645+0000 7f36fce25700 1 -- 172.21.15.7:0/1010317884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec009550 con 0x7f36f8106e90 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.646+0000 7f36fce25700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f807ba40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36ec011980 tx=0x7f36ec011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.646+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36ec003af0 con 0x7f36f8106e90 2024-01-11T17:50:28.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.646+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ec003c50 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.646+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f807a170 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.646+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36ec019760 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.646+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f807a610 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.647+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 45892+0+0 (secure 0 0 0) 0x7f36ec01a430 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.647+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f8062c40 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.648+0000 7f36f5ffb700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36e8038ec0 0x7f36e803b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.648+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f36ec04d360 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.648+0000 7f36f7fff700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36e8038ec0 0x7f36e803b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.649+0000 7f36f7fff700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36e8038ec0 0x7f36e803b380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f36e4006fd0 tx=0x7f36e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.653+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f36ec02d950 con 0x7f36f8106e90 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:27.882+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f36f8062a60 con 0x7f36e8038ec0 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.132+0000 7f36f5ffb700 1 -- 172.21.15.7:0/1010317884 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f36f8062a60 con 0x7f36e8038ec0 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36e8038ec0 msgr2=0x7f36e803b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36e8038ec0 0x7f36e803b380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f36e4006fd0 tx=0x7f36e4006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 msgr2=0x7f36f807ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f807ba40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36ec011980 tx=0x7f36ec011a60 comp rx=0 tx=0).stop 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 shutdown_connections 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36e8038ec0 0x7f36e803b380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 --2- 172.21.15.7:0/1010317884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f8106e90 0x7f36f807ba40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.135+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 >> 172.21.15.7:0/1010317884 conn(0x7f36f8102470 msgr2=0x7f36f8102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.136+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 shutdown_connections 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:28.136+0000 7f36ff089700 1 -- 172.21.15.7:0/1010317884 wait complete. 2024-01-11T17:50:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-01-11T17:50:28.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:28 smithi007 ceph-mon[33361]: [11/Jan/2024:17:50:27] ENGINE Bus STARTING 2024-01-11T17:50:28.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:28 smithi007 ceph-mon[33361]: [11/Jan/2024:17:50:27] ENGINE Serving on http://172.21.15.7:8765 2024-01-11T17:50:28.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:28 smithi007 ceph-mon[33361]: [11/Jan/2024:17:50:27] ENGINE Serving on https://172.21.15.7:7150 2024-01-11T17:50:28.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:28 smithi007 ceph-mon[33361]: [11/Jan/2024:17:50:27] ENGINE Bus STARTED 2024-01-11T17:50:28.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:29.729 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:29 smithi007 ceph-mon[33361]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:29.729 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:29 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3778493169' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.185+0000 7fc8f0938700 1 Processor -- start 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.185+0000 7fc8f0938700 1 -- start start 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.185+0000 7fc8f0938700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec07a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.185+0000 7fc8f0938700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec07a780 con 0x7fc8ec07bd50 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.186+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec07a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.186+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec07a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46614/0 (socket says 172.21.15.7:46614) 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.186+0000 7fc8eb7fe700 1 -- 172.21.15.7:0/3417710814 learned_addr learned my addr 172.21.15.7:0/3417710814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.186+0000 7fc8eb7fe700 1 -- 172.21.15.7:0/3417710814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec07a8c0 con 0x7fc8ec07bd50 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.186+0000 7fc8eb7fe700 1 --2- 172.21.15.7:0/3417710814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec07a1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0098a0 tx=0x7fc8dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13de9640253a8488 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8ea7fc700 1 -- 172.21.15.7:0/3417710814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc004160 con 0x7fc8ec07bd50 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8ea7fc700 1 -- 172.21.15.7:0/3417710814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc0042c0 con 0x7fc8ec07bd50 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8ea7fc700 1 -- 172.21.15.7:0/3417710814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc0119b0 con 0x7fc8ec07bd50 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 -- 172.21.15.7:0/3417710814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 msgr2=0x7fc8ec07a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 --2- 172.21.15.7:0/3417710814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec07a1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0098a0 tx=0x7fc8dc009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 -- 172.21.15.7:0/3417710814 shutdown_connections 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 --2- 172.21.15.7:0/3417710814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec07a1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 -- 172.21.15.7:0/3417710814 >> 172.21.15.7:0/3417710814 conn(0x7fc8ec102240 msgr2=0x7fc8ec104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 -- 172.21.15.7:0/3417710814 shutdown_connections 2024-01-11T17:50:29.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.187+0000 7fc8f0938700 1 -- 172.21.15.7:0/3417710814 wait complete. 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.188+0000 7fc8f0938700 1 Processor -- start 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8f0938700 1 -- start start 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8f0938700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8f0938700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec19d110 con 0x7fc8ec07bd50 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46616/0 (socket says 172.21.15.7:46616) 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8eb7fe700 1 -- 172.21.15.7:0/3778493169 learned_addr learned my addr 172.21.15.7:0/3778493169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.189+0000 7fc8eb7fe700 1 -- 172.21.15.7:0/3778493169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc009550 con 0x7fc8ec07bd50 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.190+0000 7fc8eb7fe700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec19cbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc003e60 tx=0x7fc8dc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.190+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc0038b0 con 0x7fc8ec07bd50 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.190+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc003a10 con 0x7fc8ec07bd50 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.190+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec19d310 con 0x7fc8ec07bd50 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.190+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc019530 con 0x7fc8ec07bd50 2024-01-11T17:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.190+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec19d7b0 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.191+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec10d0c0 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.191+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 45892+0+0 (secure 0 0 0) 0x7fc8dc01a980 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.192+0000 7fc8e8ff9700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8d4038e10 0x7fc8d403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.192+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc8dc04d250 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.192+0000 7fc8eaffd700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8d4038e10 0x7fc8d403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.193+0000 7fc8eaffd700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8d4038e10 0x7fc8d403b2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0006fd0 tx=0x7fc8e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.197+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc8dc003b80 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.406+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc8ec1a0090 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.407+0000 7fc8e8ff9700 1 -- 172.21.15.7:0/3778493169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc8dc018350 con 0x7fc8ec07bd50 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8d4038e10 msgr2=0x7fc8d403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8d4038e10 0x7fc8d403b2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0006fd0 tx=0x7fc8e0006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 msgr2=0x7fc8ec19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec19cbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc003e60 tx=0x7fc8dc011a30 comp rx=0 tx=0).stop 2024-01-11T17:50:29.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 shutdown_connections 2024-01-11T17:50:29.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8d4038e10 0x7fc8d403b2d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:29.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 --2- 172.21.15.7:0/3778493169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8ec07bd50 0x7fc8ec19cbd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:29.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 >> 172.21.15.7:0/3778493169 conn(0x7fc8ec102240 msgr2=0x7fc8ec102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:29.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 shutdown_connections 2024-01-11T17:50:29.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:29.409+0000 7fc8f0938700 1 -- 172.21.15.7:0/3778493169 wait complete. 2024-01-11T17:50:29.753 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-11T17:50:29.753 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-01-11T17:50:29.769 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-11T17:50:29.769 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-01-11T17:50:29.770 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-01-11T17:50:29.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: Password: p1thhvd48z 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:29.773 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config directory 2024-01-11T17:50:29.774 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.208+0000 7f7d22bf5700 1 Processor -- start 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.208+0000 7f7d22bf5700 1 -- start start 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.209+0000 7f7d22bf5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.209+0000 7f7d22bf5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c074bc0 con 0x7f7d1c1079a0 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.209+0000 7f7d20991700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.209+0000 7f7d20991700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46630/0 (socket says 172.21.15.7:46630) 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.209+0000 7f7d20991700 1 -- 172.21.15.7:0/1324415694 learned_addr learned my addr 172.21.15.7:0/1324415694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.210+0000 7f7d20991700 1 -- 172.21.15.7:0/1324415694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c074d00 con 0x7f7d1c1079a0 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.210+0000 7f7d20991700 1 --2- 172.21.15.7:0/1324415694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c109d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d100098a0 tx=0x7f7d10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af0e283cd8bc5ea server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:30.753 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.210+0000 7f7d1b7fe700 1 -- 172.21.15.7:0/1324415694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d10004160 con 0x7f7d1c1079a0 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.210+0000 7f7d1b7fe700 1 -- 172.21.15.7:0/1324415694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d100042c0 con 0x7f7d1c1079a0 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.210+0000 7f7d1b7fe700 1 -- 172.21.15.7:0/1324415694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d100119b0 con 0x7f7d1c1079a0 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 -- 172.21.15.7:0/1324415694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 msgr2=0x7f7d1c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 --2- 172.21.15.7:0/1324415694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c109d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d100098a0 tx=0x7f7d10009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 -- 172.21.15.7:0/1324415694 shutdown_connections 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 --2- 172.21.15.7:0/1324415694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c109d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 -- 172.21.15.7:0/1324415694 >> 172.21.15.7:0/1324415694 conn(0x7f7d1c101350 msgr2=0x7f7d1c103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 -- 172.21.15.7:0/1324415694 shutdown_connections 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.211+0000 7f7d22bf5700 1 -- 172.21.15.7:0/1324415694 wait complete. 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.212+0000 7f7d22bf5700 1 Processor -- start 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.212+0000 7f7d22bf5700 1 -- start start 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.212+0000 7f7d22bf5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c1071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.212+0000 7f7d22bf5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c107710 con 0x7f7d1c1079a0 2024-01-11T17:50:30.754 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d20991700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c1071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d20991700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c1071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46632/0 (socket says 172.21.15.7:46632) 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d20991700 1 -- 172.21.15.7:0/3546347205 learned_addr learned my addr 172.21.15.7:0/3546347205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d20991700 1 -- 172.21.15.7:0/3546347205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10009550 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d20991700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c1071d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d10011670 tx=0x7f7d10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d10003b20 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d10003c80 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c105950 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.213+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d10019770 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.214+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c105df0 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.215+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c10b040 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.215+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 45892+0+0 (secure 0 0 0) 0x7f7d1001ab60 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.215+0000 7f7d19ffb700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7d0c038ec0 0x7f7d0c03b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.215+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d1004d350 con 0x7f7d1c1079a0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.215+0000 7f7d1bfff700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7d0c038ec0 0x7f7d0c03b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:30.755 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.216+0000 7f7d1bfff700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7d0c038ec0 0x7f7d0c03b380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.220+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f7d1001a430 con 0x7f7d1c1079a0 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.430+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7d1c10b250 con 0x7f7d1c1079a0 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.431+0000 7f7d19ffb700 1 -- 172.21.15.7:0/3546347205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7d1002db40 con 0x7f7d1c1079a0 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7d0c038ec0 msgr2=0x7f7d0c03b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7d0c038ec0 0x7f7d0c03b380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 msgr2=0x7f7d1c1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c1071d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d10011670 tx=0x7f7d10011a30 comp rx=0 tx=0).stop 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 shutdown_connections 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7d0c038ec0 0x7f7d0c03b380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 --2- 172.21.15.7:0/3546347205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d1c1079a0 0x7f7d1c1071d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.433+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 >> 172.21.15.7:0/3546347205 conn(0x7f7d1c101350 msgr2=0x7f7d1c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.434+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 shutdown_connections 2024-01-11T17:50:30.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:30.434+0000 7f7d22bf5700 1 -- 172.21.15.7:0/3546347205 wait complete. 2024-01-11T17:50:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:31 smithi007 ceph-mon[33361]: mgrmap e13: x(active, since 6s) 2024-01-11T17:50:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:31 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3464741087' entity='client.admin' 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.166+0000 7fe884f20700 1 Processor -- start 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.166+0000 7fe884f20700 1 -- start start 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.166+0000 7fe884f20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe880109d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.166+0000 7fe884f20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880074c00 con 0x7fe880107960 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.167+0000 7fe87e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe880109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.167+0000 7fe87e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe880109d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46640/0 (socket says 172.21.15.7:46640) 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.167+0000 7fe87e59c700 1 -- 172.21.15.7:0/2380950896 learned_addr learned my addr 172.21.15.7:0/2380950896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.167+0000 7fe87e59c700 1 -- 172.21.15.7:0/2380950896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88010a290 con 0x7fe880107960 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.167+0000 7fe87e59c700 1 --2- 172.21.15.7:0/2380950896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe880109d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098a0 tx=0x7fe870009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fcf2e1616b35f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe87d59a700 1 -- 172.21.15.7:0/2380950896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe870004160 con 0x7fe880107960 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe87d59a700 1 -- 172.21.15.7:0/2380950896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8700042c0 con 0x7fe880107960 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe87d59a700 1 -- 172.21.15.7:0/2380950896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8700119b0 con 0x7fe880107960 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe884f20700 1 -- 172.21.15.7:0/2380950896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 msgr2=0x7fe880109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe884f20700 1 --2- 172.21.15.7:0/2380950896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe880109d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098a0 tx=0x7fe870009bb0 comp rx=0 tx=0).stop 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe884f20700 1 -- 172.21.15.7:0/2380950896 shutdown_connections 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe884f20700 1 --2- 172.21.15.7:0/2380950896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe880109d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe884f20700 1 -- 172.21.15.7:0/2380950896 >> 172.21.15.7:0/2380950896 conn(0x7fe880101350 msgr2=0x7fe880103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.168+0000 7fe884f20700 1 -- 172.21.15.7:0/2380950896 shutdown_connections 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.169+0000 7fe884f20700 1 -- 172.21.15.7:0/2380950896 wait complete. 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.169+0000 7fe884f20700 1 Processor -- start 2024-01-11T17:50:31.838 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.170+0000 7fe884f20700 1 -- start start 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.170+0000 7fe884f20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe8801071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.170+0000 7fe884f20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880107720 con 0x7fe880107960 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.170+0000 7fe87e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe8801071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.170+0000 7fe87e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe8801071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46644/0 (socket says 172.21.15.7:46644) 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.170+0000 7fe87e59c700 1 -- 172.21.15.7:0/3464741087 learned_addr learned my addr 172.21.15.7:0/3464741087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe87e59c700 1 -- 172.21.15.7:0/3464741087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870009550 con 0x7fe880107960 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe87e59c700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe8801071e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe870011670 tx=0x7fe870011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe870003b20 con 0x7fe880107960 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe870003c80 con 0x7fe880107960 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe880105910 con 0x7fe880107960 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe870019770 con 0x7fe880107960 2024-01-11T17:50:31.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.171+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe880105db0 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.172+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88010b020 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.172+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7fe87001ab60 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.173+0000 7fe86f7fe700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe868038f60 0x7fe86803b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.173+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe87004d410 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.173+0000 7fe87dd9b700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe868038f60 0x7fe86803b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.174+0000 7fe87dd9b700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe868038f60 0x7fe86803b420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe874006fd0 tx=0x7fe874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.178+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe87001a430 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.500+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe88010b230 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.503+0000 7fe86f7fe700 1 -- 172.21.15.7:0/3464741087 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe87002db40 con 0x7fe880107960 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe868038f60 msgr2=0x7fe86803b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe868038f60 0x7fe86803b420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe874006fd0 tx=0x7fe874006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:31.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 msgr2=0x7fe8801071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe8801071e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe870011670 tx=0x7fe870011a30 comp rx=0 tx=0).stop 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 shutdown_connections 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe868038f60 0x7fe86803b420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 --2- 172.21.15.7:0/3464741087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe880107960 0x7fe8801071e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 >> 172.21.15.7:0/3464741087 conn(0x7fe880101350 msgr2=0x7fe880101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 shutdown_connections 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-01-11T17:50:31.505+0000 7fe884f20700 1 -- 172.21.15.7:0/3464741087 wait complete. 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-01-11T17:50:31.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:31.842 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-01-11T17:50:31.903 INFO:tasks.cephadm:Fetching config... 2024-01-11T17:50:31.903 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:50:31.904 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-01-11T17:50:31.946 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-01-11T17:50:31.947 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:50:31.947 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-01-11T17:50:32.035 INFO:tasks.cephadm:Fetching mon keyring... 2024-01-11T17:50:32.035 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:50:32.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/keyring of=/dev/stdout 2024-01-11T17:50:32.159 INFO:tasks.cephadm:Fetching pub ssh key... 2024-01-11T17:50:32.159 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:50:32.159 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-01-11T17:50:32.243 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-01-11T17:50:32.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-11T17:50:32.383 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:50:32.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-11T17:50:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:50:32.480 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-11T17:50:32.530 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXwFaDh/UWDCrBCmc66firmeafgtTuaqDuYwJaN8IZbGhXWTCmFR/Pt1vCn8NXj9DRbxPYHUVGxKQv35EtO8fb1PpAg0cl7EMvUdX7RMBjdcGBABGP+DRXpfs8pWNIPKJrOj8jopNj4/t7Hdb73wrA68i9pODWMi4AexQw8lemZ4LFuR94fcxT80U8/LFEzpkd1FjdDejR0D2DwijAi3EleZIPh8jg2YMD9vJo5zAMVV1Xr41HeH82P/ypheqzqL4ZKDmIGTja0heQ3VaASzA2K8H0MYZWmo6ah9j2lhrXPcEMlJpitPi2yDyG0XQPl1u+6q2OJB9GF9ZCh0bx5iifA1MyF1VOSt2Pxy4innGBeRYHsCKzkGFgQyJBygebBu5qY7ivYd5UerDI0/AgHOBQDFHOBbp6RjML3AR1KhXQj8Kvgnqynb8nw/mOfzgeqlV7J2joQc7uPnRkAEves7ldEs1sM/4Ir1UITdH2V/BUMrpZJYOMAtL+1+Ph5K9npWE= ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:50:32.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-01-11T17:50:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 -- 172.21.15.7:0/3732483030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 msgr2=0x7f35bc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 --2- 172.21.15.7:0/3732483030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc101460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009910 tx=0x7f35ac009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 -- 172.21.15.7:0/3732483030 shutdown_connections 2024-01-11T17:50:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 --2- 172.21.15.7:0/3732483030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc101460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 -- 172.21.15.7:0/3732483030 >> 172.21.15.7:0/3732483030 conn(0x7f35bc0fc5e0 msgr2=0x7f35bc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 -- 172.21.15.7:0/3732483030 shutdown_connections 2024-01-11T17:50:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.473+0000 7f35c291d700 1 -- 172.21.15.7:0/3732483030 wait complete. 2024-01-11T17:50:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.474+0000 7f35c291d700 1 Processor -- start 2024-01-11T17:50:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.475+0000 7f35c291d700 1 -- start start 2024-01-11T17:50:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.475+0000 7f35c291d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.475+0000 7f35c291d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc198bd0 con 0x7f35bc101040 2024-01-11T17:50:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.475+0000 7f35bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.475+0000 7f35bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35298/0 (socket says 172.21.15.7:35298) 2024-01-11T17:50:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.475+0000 7f35bbfff700 1 -- 172.21.15.7:0/2440752699 learned_addr learned my addr 172.21.15.7:0/2440752699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35bbfff700 1 -- 172.21.15.7:0/2440752699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac0095f0 con 0x7f35bc101040 2024-01-11T17:50:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35bbfff700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc198690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005950 tx=0x7f35ac0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35ac01d070 con 0x7f35bc101040 2024-01-11T17:50:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ac00bc10 con 0x7f35bc101040 2024-01-11T17:50:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35ac003a10 con 0x7f35bc101040 2024-01-11T17:50:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc198dd0 con 0x7f35bc101040 2024-01-11T17:50:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.476+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc199270 con 0x7f35bc101040 2024-01-11T17:50:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.477+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc192160 con 0x7f35bc101040 2024-01-11T17:50:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.477+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f35ac004390 con 0x7f35bc101040 2024-01-11T17:50:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.478+0000 7f35b97fa700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35a4038f10 0x7f35a403b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.478+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f35ac04cdd0 con 0x7f35bc101040 2024-01-11T17:50:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.478+0000 7f35bb7fe700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35a4038f10 0x7f35a403b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.478+0000 7f35bb7fe700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35a4038f10 0x7f35a403b3d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f35b0006fd0 tx=0x7f35b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.483+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f35ac02abf0 con 0x7f35bc101040 2024-01-11T17:50:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.694+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f35bc062c40 con 0x7f35bc101040 2024-01-11T17:50:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.697+0000 7f35b97fa700 1 -- 172.21.15.7:0/2440752699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f35ac026070 con 0x7f35bc101040 2024-01-11T17:50:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35a4038f10 msgr2=0x7f35a403b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35a4038f10 0x7f35a403b3d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f35b0006fd0 tx=0x7f35b0006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 msgr2=0x7f35bc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc198690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005950 tx=0x7f35ac0057c0 comp rx=0 tx=0).stop 2024-01-11T17:50:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 shutdown_connections 2024-01-11T17:50:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35a4038f10 0x7f35a403b3d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 --2- 172.21.15.7:0/2440752699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc101040 0x7f35bc198690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 >> 172.21.15.7:0/2440752699 conn(0x7f35bc0fc5e0 msgr2=0x7f35bc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 shutdown_connections 2024-01-11T17:50:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:34.700+0000 7f35c291d700 1 -- 172.21.15.7:0/2440752699 wait complete. 2024-01-11T17:50:35.075 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-01-11T17:50:35.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2440752699' entity='client.admin' 2024-01-11T17:50:35.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:35.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:35.485 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 -- 172.21.15.7:0/517304132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 msgr2=0x7fc144101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 --2- 172.21.15.7:0/517304132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc144101460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc134009910 tx=0x7fc134009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 -- 172.21.15.7:0/517304132 shutdown_connections 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 --2- 172.21.15.7:0/517304132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc144101460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 -- 172.21.15.7:0/517304132 >> 172.21.15.7:0/517304132 conn(0x7fc1440fc5e0 msgr2=0x7fc1440fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 -- 172.21.15.7:0/517304132 shutdown_connections 2024-01-11T17:50:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.657+0000 7fc148c24700 1 -- 172.21.15.7:0/517304132 wait complete. 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.658+0000 7fc148c24700 1 Processor -- start 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc148c24700 1 -- start start 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc148c24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc1441986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc148c24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144198c00 con 0x7fc144101040 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc14259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc1441986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc14259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc1441986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35322/0 (socket says 172.21.15.7:35322) 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc14259c700 1 -- 172.21.15.7:0/3815281259 learned_addr learned my addr 172.21.15.7:0/3815281259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.659+0000 7fc14259c700 1 -- 172.21.15.7:0/3815281259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340095f0 con 0x7fc144101040 2024-01-11T17:50:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.660+0000 7fc14259c700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc1441986c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc134005950 tx=0x7fc1340057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.660+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc13401d070 con 0x7fc144101040 2024-01-11T17:50:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.660+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc13400bc10 con 0x7fc144101040 2024-01-11T17:50:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.660+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc144198e00 con 0x7fc144101040 2024-01-11T17:50:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.660+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc134003a10 con 0x7fc144101040 2024-01-11T17:50:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.660+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1441992a0 con 0x7fc144101040 2024-01-11T17:50:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.661+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7fc134004390 con 0x7fc144101040 2024-01-11T17:50:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.661+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc144192160 con 0x7fc144101040 2024-01-11T17:50:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.662+0000 7fc1337fe700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc12c038f10 0x7fc12c03b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.662+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc13404cdd0 con 0x7fc144101040 2024-01-11T17:50:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.662+0000 7fc141d9b700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc12c038f10 0x7fc12c03b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.662+0000 7fc141d9b700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc12c038f10 0x7fc12c03b3d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc138006fd0 tx=0x7fc138006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.666+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc13402ae20 con 0x7fc144101040 2024-01-11T17:50:36.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.882+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fc14402d3c0 con 0x7fc12c038f10 2024-01-11T17:50:36.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.886+0000 7fc1337fe700 1 -- 172.21.15.7:0/3815281259 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc14402d3c0 con 0x7fc12c038f10 2024-01-11T17:50:36.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.888+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc12c038f10 msgr2=0x7fc12c03b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:36.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.888+0000 7fc148c24700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc12c038f10 0x7fc12c03b3d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc138006fd0 tx=0x7fc138006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:36.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.888+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 msgr2=0x7fc1441986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:36.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc1441986c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc134005950 tx=0x7fc1340057c0 comp rx=0 tx=0).stop 2024-01-11T17:50:36.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 shutdown_connections 2024-01-11T17:50:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc12c038f10 0x7fc12c03b3d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 --2- 172.21.15.7:0/3815281259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144101040 0x7fc1441986c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 >> 172.21.15.7:0/3815281259 conn(0x7fc1440fc5e0 msgr2=0x7fc1440fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 shutdown_connections 2024-01-11T17:50:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:36.889+0000 7fc148c24700 1 -- 172.21.15.7:0/3815281259 wait complete. 2024-01-11T17:50:37.246 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi016 2024-01-11T17:50:37.246 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:50:37.246 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.conf 2024-01-11T17:50:37.268 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:50:37.268 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:50:37.329 INFO:tasks.cephadm:Adding host smithi016 to orchestrator... 2024-01-11T17:50:37.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch host add smithi016 2024-01-11T17:50:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:38.230 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:38.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:38.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:38.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:50:38.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:50:38.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: Updating smithi007:/etc/ceph/ceph.conf 2024-01-11T17:50:38.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:37 smithi007 ceph-mon[33361]: Updating smithi007:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:50:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 -- 172.21.15.7:0/1656894877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 msgr2=0x7f3288073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 --2- 172.21.15.7:0/1656894877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f3288073d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3284009910 tx=0x7f3284009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 -- 172.21.15.7:0/1656894877 shutdown_connections 2024-01-11T17:50:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 --2- 172.21.15.7:0/1656894877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f3288073d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 -- 172.21.15.7:0/1656894877 >> 172.21.15.7:0/1656894877 conn(0x7f32880fcaf0 msgr2=0x7f32880fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 -- 172.21.15.7:0/1656894877 shutdown_connections 2024-01-11T17:50:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.959+0000 7f328fba7700 1 -- 172.21.15.7:0/1656894877 wait complete. 2024-01-11T17:50:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.960+0000 7f328fba7700 1 Processor -- start 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.960+0000 7f328fba7700 1 -- start start 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.960+0000 7f328fba7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f328819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.960+0000 7f328fba7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328819d370 con 0x7f32880739b0 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.961+0000 7f328d943700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f328819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.961+0000 7f328d943700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f328819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35344/0 (socket says 172.21.15.7:35344) 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.961+0000 7f328d943700 1 -- 172.21.15.7:0/4239118860 learned_addr learned my addr 172.21.15.7:0/4239118860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.961+0000 7f328d943700 1 -- 172.21.15.7:0/4239118860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840095f0 con 0x7f32880739b0 2024-01-11T17:50:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.961+0000 7f328d943700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f328819ce30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3284006010 tx=0x7f3284004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.962+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f328401d070 con 0x7f32880739b0 2024-01-11T17:50:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.962+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f328400bbb0 con 0x7f32880739b0 2024-01-11T17:50:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.962+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328819d5d0 con 0x7f32880739b0 2024-01-11T17:50:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.962+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3284003880 con 0x7f32880739b0 2024-01-11T17:50:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.962+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328819da50 con 0x7f32880739b0 2024-01-11T17:50:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.963+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3288062c40 con 0x7f32880739b0 2024-01-11T17:50:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.963+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f328400bd20 con 0x7f32880739b0 2024-01-11T17:50:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.964+0000 7f327effd700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3274038f10 0x7f327403b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.964+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f328404cc70 con 0x7f32880739b0 2024-01-11T17:50:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.964+0000 7f328d142700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3274038f10 0x7f327403b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.965+0000 7f328d142700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3274038f10 0x7f327403b3d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3278006fd0 tx=0x7f3278006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:38.969+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f328402aba0 con 0x7f32880739b0 2024-01-11T17:50:39.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:38 smithi007 ceph-mon[33361]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:50:39.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:38 smithi007 ceph-mon[33361]: Updating smithi007:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.client.admin.keyring 2024-01-11T17:50:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:39.196+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}) v1 -- 0x7f3288111a50 con 0x7f3274038f10 2024-01-11T17:50:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:40 smithi007 ceph-mon[33361]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:41.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:41 smithi007 ceph-mon[33361]: Deploying cephadm binary to smithi016 2024-01-11T17:50:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.289+0000 7f327effd700 1 -- 172.21.15.7:0/4239118860 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f3288111a50 con 0x7f3274038f10 2024-01-11T17:50:43.290 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi016' with addr '172.21.15.16' 2024-01-11T17:50:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3274038f10 msgr2=0x7f327403b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3274038f10 0x7f327403b3d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3278006fd0 tx=0x7f3278006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 msgr2=0x7f328819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f328819ce30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3284006010 tx=0x7f3284004ff0 comp rx=0 tx=0).stop 2024-01-11T17:50:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 shutdown_connections 2024-01-11T17:50:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3274038f10 0x7f327403b3d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 --2- 172.21.15.7:0/4239118860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32880739b0 0x7f328819ce30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 >> 172.21.15.7:0/4239118860 conn(0x7f32880fcaf0 msgr2=0x7f32880fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 shutdown_connections 2024-01-11T17:50:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:43.292+0000 7f328fba7700 1 -- 172.21.15.7:0/4239118860 wait complete. 2024-01-11T17:50:43.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch host ls --format=json 2024-01-11T17:50:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:44.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:44.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:44 smithi007 ceph-mon[33361]: Added host smithi016 2024-01-11T17:50:44.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:44.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:44.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 -- 172.21.15.7:0/208260253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 msgr2=0x7f3db4101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 --2- 172.21.15.7:0/208260253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db4101620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009910 tx=0x7f3da4009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 -- 172.21.15.7:0/208260253 shutdown_connections 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 --2- 172.21.15.7:0/208260253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db4101620 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 -- 172.21.15.7:0/208260253 >> 172.21.15.7:0/208260253 conn(0x7f3db40fcaf0 msgr2=0x7f3db40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 -- 172.21.15.7:0/208260253 shutdown_connections 2024-01-11T17:50:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.209+0000 7f3dba094700 1 -- 172.21.15.7:0/208260253 wait complete. 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.210+0000 7f3dba094700 1 Processor -- start 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.210+0000 7f3dba094700 1 -- start start 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.211+0000 7f3dba094700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.211+0000 7f3dba094700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db419eca0 con 0x7f3db4101240 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.211+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.211+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49592/0 (socket says 172.21.15.7:49592) 2024-01-11T17:50:45.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.211+0000 7f3db37fe700 1 -- 172.21.15.7:0/1601195912 learned_addr learned my addr 172.21.15.7:0/1601195912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:45.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.211+0000 7f3db37fe700 1 -- 172.21.15.7:0/1601195912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40095f0 con 0x7f3db4101240 2024-01-11T17:50:45.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.212+0000 7f3db37fe700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db419e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006010 tx=0x7f3da4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:45.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.212+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3da401d070 con 0x7f3db4101240 2024-01-11T17:50:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.212+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3da400bc10 con 0x7f3db4101240 2024-01-11T17:50:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.212+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3da40038a0 con 0x7f3db4101240 2024-01-11T17:50:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.212+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db419eea0 con 0x7f3db4101240 2024-01-11T17:50:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.212+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db419f2c0 con 0x7f3db4101240 2024-01-11T17:50:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.213+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f3da400b6f0 con 0x7f3db4101240 2024-01-11T17:50:45.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.213+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db4062c40 con 0x7f3db4101240 2024-01-11T17:50:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.214+0000 7f3db0ff9700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d9c038f10 0x7f3d9c03b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.214+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3da404cbd0 con 0x7f3db4101240 2024-01-11T17:50:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.214+0000 7f3db2ffd700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d9c038f10 0x7f3d9c03b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.215+0000 7f3db2ffd700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d9c038f10 0x7f3d9c03b3d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3da8006fd0 tx=0x7f3da8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.219+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3da400bd80 con 0x7f3db4101240 2024-01-11T17:50:45.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.437+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3db410b440 con 0x7f3d9c038f10 2024-01-11T17:50:45.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.438+0000 7f3db0ff9700 1 -- 172.21.15.7:0/1601195912 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f3db410b440 con 0x7f3d9c038f10 2024-01-11T17:50:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:45.439 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}] 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d9c038f10 msgr2=0x7f3d9c03b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d9c038f10 0x7f3d9c03b3d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3da8006fd0 tx=0x7f3da8006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 msgr2=0x7f3db419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db419e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006010 tx=0x7f3da4004ff0 comp rx=0 tx=0).stop 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 shutdown_connections 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d9c038f10 0x7f3d9c03b3d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 --2- 172.21.15.7:0/1601195912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3db4101240 0x7f3db419e760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 >> 172.21.15.7:0/1601195912 conn(0x7f3db40fcaf0 msgr2=0x7f3db4073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 shutdown_connections 2024-01-11T17:50:45.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:45.441+0000 7f3dba094700 1 -- 172.21.15.7:0/1601195912 wait complete. 2024-01-11T17:50:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:45 smithi007 ceph-mon[33361]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:45.762 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:45.796 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi038 2024-01-11T17:50:45.797 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:50:45.797 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/etc/ceph/ceph.conf 2024-01-11T17:50:45.819 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:50:45.819 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:50:45.880 INFO:tasks.cephadm:Adding host smithi038 to orchestrator... 2024-01-11T17:50:45.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch host add smithi038 2024-01-11T17:50:46.291 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:46.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:46 smithi007 ceph-mon[33361]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:50:47.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 -- 172.21.15.7:0/1462522294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 msgr2=0x7f02b0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:47.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 --2- 172.21.15.7:0/1462522294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b0101620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009910 tx=0x7f02ac009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:47.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 -- 172.21.15.7:0/1462522294 shutdown_connections 2024-01-11T17:50:47.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 --2- 172.21.15.7:0/1462522294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b0101620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:47.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 -- 172.21.15.7:0/1462522294 >> 172.21.15.7:0/1462522294 conn(0x7f02b00fcaf0 msgr2=0x7f02b00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:47.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 -- 172.21.15.7:0/1462522294 shutdown_connections 2024-01-11T17:50:47.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.450+0000 7f02b82eb700 1 -- 172.21.15.7:0/1462522294 wait complete. 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.451+0000 7f02b82eb700 1 Processor -- start 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.451+0000 7f02b82eb700 1 -- start start 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.452+0000 7f02b82eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.452+0000 7f02b82eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b019ec40 con 0x7f02b0101240 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.452+0000 7f02b6087700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.452+0000 7f02b6087700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49608/0 (socket says 172.21.15.7:49608) 2024-01-11T17:50:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.452+0000 7f02b6087700 1 -- 172.21.15.7:0/1806483179 learned_addr learned my addr 172.21.15.7:0/1806483179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.452+0000 7f02b6087700 1 -- 172.21.15.7:0/1806483179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0095f0 con 0x7f02b0101240 2024-01-11T17:50:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.453+0000 7f02b6087700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b019e700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02ac006010 tx=0x7f02ac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.453+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02ac01d070 con 0x7f02b0101240 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.453+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02ac00bc10 con 0x7f02b0101240 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.453+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b019ee40 con 0x7f02b0101240 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.453+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02ac0038a0 con 0x7f02b0101240 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.453+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b019f260 con 0x7f02b0101240 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.454+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b0062c40 con 0x7f02b0101240 2024-01-11T17:50:47.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.454+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f02ac00b6f0 con 0x7f02b0101240 2024-01-11T17:50:47.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.455+0000 7f02a77fe700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f029c038eb0 0x7f029c03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:47.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.455+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02ac04ccc0 con 0x7f02b0101240 2024-01-11T17:50:47.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.455+0000 7f02b5886700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f029c038eb0 0x7f029c03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:47.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.456+0000 7f02b5886700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f029c038eb0 0x7f029c03b370 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006fd0 tx=0x7f02a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.459+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f02ac053080 con 0x7f02b0101240 2024-01-11T17:50:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:47.675+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "target": ["mon-mgr", ""]}) v1 -- 0x7f02b010b3a0 con 0x7f029c038eb0 2024-01-11T17:50:47.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:47 smithi007 ceph-mon[33361]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: Updating smithi016:/etc/ceph/ceph.conf 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: Updating smithi016:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: Deploying cephadm binary to smithi038 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:50:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:48 smithi007 ceph-mon[33361]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:50.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:50 smithi007 ceph-mon[33361]: Updating smithi016:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.client.admin.keyring 2024-01-11T17:50:51.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:51 smithi007 ceph-mon[33361]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:51.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.716+0000 7f02a77fe700 1 -- 172.21.15.7:0/1806483179 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f02b010b3a0 con 0x7f029c038eb0 2024-01-11T17:50:51.717 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi038' with addr '172.21.15.38' 2024-01-11T17:50:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f029c038eb0 msgr2=0x7f029c03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f029c038eb0 0x7f029c03b370 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006fd0 tx=0x7f02a0006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 msgr2=0x7f02b019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b019e700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02ac006010 tx=0x7f02ac004ff0 comp rx=0 tx=0).stop 2024-01-11T17:50:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 shutdown_connections 2024-01-11T17:50:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f029c038eb0 0x7f029c03b370 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:51.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 --2- 172.21.15.7:0/1806483179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b0101240 0x7f02b019e700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:51.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 >> 172.21.15.7:0/1806483179 conn(0x7f02b00fcaf0 msgr2=0x7f02b0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:51.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 shutdown_connections 2024-01-11T17:50:51.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:51.719+0000 7f02b82eb700 1 -- 172.21.15.7:0/1806483179 wait complete. 2024-01-11T17:50:52.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch host ls --format=json 2024-01-11T17:50:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:52.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:52.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:52.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:52.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:52.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:52.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:53.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:53 smithi007 ceph-mon[33361]: Added host smithi038 2024-01-11T17:50:53.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:53 smithi007 ceph-mon[33361]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:53.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:53.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 -- 172.21.15.7:0/3937207831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 msgr2=0x7f63f8101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 --2- 172.21.15.7:0/3937207831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f8101620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009910 tx=0x7f63f4009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 -- 172.21.15.7:0/3937207831 shutdown_connections 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 --2- 172.21.15.7:0/3937207831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f8101620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 -- 172.21.15.7:0/3937207831 >> 172.21.15.7:0/3937207831 conn(0x7f63f80fcaf0 msgr2=0x7f63f80fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 -- 172.21.15.7:0/3937207831 shutdown_connections 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.592+0000 7f63ff7fd700 1 -- 172.21.15.7:0/3937207831 wait complete. 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.593+0000 7f63ff7fd700 1 Processor -- start 2024-01-11T17:50:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.594+0000 7f63ff7fd700 1 -- start start 2024-01-11T17:50:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.594+0000 7f63ff7fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.594+0000 7f63ff7fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f819cb20 con 0x7f63f8101240 2024-01-11T17:50:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.594+0000 7f63fd599700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.594+0000 7f63fd599700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39574/0 (socket says 172.21.15.7:39574) 2024-01-11T17:50:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.594+0000 7f63fd599700 1 -- 172.21.15.7:0/232701072 learned_addr learned my addr 172.21.15.7:0/232701072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63fd599700 1 -- 172.21.15.7:0/232701072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40095f0 con 0x7f63f8101240 2024-01-11T17:50:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63fd599700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f819c5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006010 tx=0x7f63f4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63f401d070 con 0x7f63f8101240 2024-01-11T17:50:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f819cd20 con 0x7f63f8101240 2024-01-11T17:50:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f400bc10 con 0x7f63f8101240 2024-01-11T17:50:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63f40038a0 con 0x7f63f8101240 2024-01-11T17:50:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.595+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81966b0 con 0x7f63f8101240 2024-01-11T17:50:53.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.596+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f63f400b6f0 con 0x7f63f8101240 2024-01-11T17:50:53.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.596+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f819d140 con 0x7f63f8101240 2024-01-11T17:50:53.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.597+0000 7f63ee7fc700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63e4038eb0 0x7f63e403b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:53.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.597+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f63f404cc70 con 0x7f63f8101240 2024-01-11T17:50:53.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.597+0000 7f63fcd98700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63e4038eb0 0x7f63e403b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:53.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.598+0000 7f63fcd98700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63e4038eb0 0x7f63e403b370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f63e8006fd0 tx=0x7f63e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.602+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f63f400bd80 con 0x7f63f8101240 2024-01-11T17:50:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.821+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63f8197c70 con 0x7f63e4038eb0 2024-01-11T17:50:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.822+0000 7f63ee7fc700 1 -- 172.21.15.7:0/232701072 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+237 (secure 0 0 0) 0x7f63f8197c70 con 0x7f63e4038eb0 2024-01-11T17:50:53.822 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:50:53.822 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}] 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.824+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63e4038eb0 msgr2=0x7f63e403b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.824+0000 7f63ff7fd700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63e4038eb0 0x7f63e403b370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f63e8006fd0 tx=0x7f63e8006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.824+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 msgr2=0x7f63f819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.824+0000 7f63ff7fd700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f819c5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006010 tx=0x7f63f4004ff0 comp rx=0 tx=0).stop 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.825+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 shutdown_connections 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.825+0000 7f63ff7fd700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63e4038eb0 0x7f63e403b370 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.825+0000 7f63ff7fd700 1 --2- 172.21.15.7:0/232701072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63f8101240 0x7f63f819c5e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.825+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 >> 172.21.15.7:0/232701072 conn(0x7f63f80fcaf0 msgr2=0x7f63f8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.825+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 shutdown_connections 2024-01-11T17:50:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:53.825+0000 7f63ff7fd700 1 -- 172.21.15.7:0/232701072 wait complete. 2024-01-11T17:50:54.204 INFO:tasks.cephadm:Setting crush tunables to default 2024-01-11T17:50:54.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd crush tunables default 2024-01-11T17:50:54.618 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:50:55.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:55 smithi007 ceph-mon[33361]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:50:55.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:55 smithi007 ceph-mon[33361]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 -- 172.21.15.7:0/2359144973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 msgr2=0x7f7c8c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 --2- 172.21.15.7:0/2359144973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c073d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009910 tx=0x7f7c7c009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 -- 172.21.15.7:0/2359144973 shutdown_connections 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 --2- 172.21.15.7:0/2359144973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c073d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 -- 172.21.15.7:0/2359144973 >> 172.21.15.7:0/2359144973 conn(0x7f7c8c0fcaf0 msgr2=0x7f7c8c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 -- 172.21.15.7:0/2359144973 shutdown_connections 2024-01-11T17:50:55.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.757+0000 7f7c91131700 1 -- 172.21.15.7:0/2359144973 wait complete. 2024-01-11T17:50:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.758+0000 7f7c91131700 1 Processor -- start 2024-01-11T17:50:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.759+0000 7f7c91131700 1 -- start start 2024-01-11T17:50:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.759+0000 7f7c91131700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:55.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.759+0000 7f7c91131700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c19d370 con 0x7f7c8c0739b0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.759+0000 7f7c8ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.759+0000 7f7c8ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39602/0 (socket says 172.21.15.7:39602) 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.759+0000 7f7c8ad9d700 1 -- 172.21.15.7:0/250824849 learned_addr learned my addr 172.21.15.7:0/250824849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c8ad9d700 1 -- 172.21.15.7:0/250824849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0095f0 con 0x7f7c8c0739b0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c8ad9d700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c19ce30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c004f40 tx=0x7f7c7c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c7c01d070 con 0x7f7c8c0739b0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c7c004370 con 0x7f7c8c0739b0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c19d5d0 con 0x7f7c8c0739b0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c7c0038e0 con 0x7f7c8c0739b0 2024-01-11T17:50:55.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.760+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c19da50 con 0x7f7c8c0739b0 2024-01-11T17:50:55.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.761+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f7c7c003ba0 con 0x7f7c8c0739b0 2024-01-11T17:50:55.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.761+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c062c40 con 0x7f7c8c0739b0 2024-01-11T17:50:55.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.762+0000 7f7c73fff700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7c74038eb0 0x7f7c7403b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.762+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c7c04cd70 con 0x7f7c8c0739b0 2024-01-11T17:50:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.762+0000 7f7c8a59c700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7c74038eb0 0x7f7c7403b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.762+0000 7f7c8a59c700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7c74038eb0 0x7f7c7403b370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c80006fd0 tx=0x7f7c80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:55.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.767+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f7c7c00bc10 con 0x7f7c8c0739b0 2024-01-11T17:50:55.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:55.985+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7c8c1119e0 con 0x7f7c8c0739b0 2024-01-11T17:50:56.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.226+0000 7f7c73fff700 1 -- 172.21.15.7:0/250824849 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7c7c026070 con 0x7f7c8c0739b0 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7c74038eb0 msgr2=0x7f7c7403b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7c74038eb0 0x7f7c7403b370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c80006fd0 tx=0x7f7c80006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 msgr2=0x7f7c8c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c19ce30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c004f40 tx=0x7f7c7c005020 comp rx=0 tx=0).stop 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 shutdown_connections 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7c74038eb0 0x7f7c7403b370 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 --2- 172.21.15.7:0/250824849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c8c0739b0 0x7f7c8c19ce30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 >> 172.21.15.7:0/250824849 conn(0x7f7c8c0fcaf0 msgr2=0x7f7c8c0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 shutdown_connections 2024-01-11T17:50:56.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:50:56.228+0000 7f7c91131700 1 -- 172.21.15.7:0/250824849 wait complete. 2024-01-11T17:50:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/250824849' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:50:56.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:50:56.571 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-01-11T17:50:56.571 INFO:tasks.cephadm:Adding mon.b on smithi016 2024-01-11T17:50:56.571 INFO:tasks.cephadm:Adding mon.c on smithi038 2024-01-11T17:50:56.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c' 2024-01-11T17:50:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:50:57.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: Updating smithi038:/etc/ceph/ceph.conf 2024-01-11T17:50:57.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/250824849' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-01-11T17:50:57.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:50:57.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:50:57.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:50:57.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:50:57.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:57.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:57.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:57 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 -- 172.21.15.38:0/859612686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 msgr2=0x7f081c101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 --2- 172.21.15.38:0/859612686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c101630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f080c009910 tx=0x7f080c009c20 comp rx=0 tx=0).stop 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 -- 172.21.15.38:0/859612686 shutdown_connections 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 --2- 172.21.15.38:0/859612686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c101630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 -- 172.21.15.38:0/859612686 >> 172.21.15.38:0/859612686 conn(0x7f081c0fcb00 msgr2=0x7f081c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 -- 172.21.15.38:0/859612686 shutdown_connections 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.877+0000 7f08213a6700 1 -- 172.21.15.38:0/859612686 wait complete. 2024-01-11T17:50:57.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f08213a6700 1 Processor -- start 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f08213a6700 1 -- start start 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f08213a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f08213a6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c19cb60 con 0x7f081c101250 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f081affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f081affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.38:60606/0 (socket says 172.21.15.38:60606) 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.878+0000 7f081affd700 1 -- 172.21.15.38:0/3892019067 learned_addr learned my addr 172.21.15.38:0/3892019067 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f081affd700 1 -- 172.21.15.38:0/3892019067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c0095f0 con 0x7f081c101250 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f081affd700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c19c620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f080c004f40 tx=0x7f080c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f080c01d070 con 0x7f081c101250 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c00bc10 con 0x7f081c101250 2024-01-11T17:50:57.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c19cd60 con 0x7f081c101250 2024-01-11T17:50:57.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f080c0037d0 con 0x7f081c101250 2024-01-11T17:50:57.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.879+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c1966f0 con 0x7f081c101250 2024-01-11T17:50:57.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.880+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f080c003a90 con 0x7f081c101250 2024-01-11T17:50:57.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.880+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c19d180 con 0x7f081c101250 2024-01-11T17:50:57.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.880+0000 7f0818ff9700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0808038f10 0x7f080803b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:50:57.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.881+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f080c04ce60 con 0x7f081c101250 2024-01-11T17:50:57.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.881+0000 7f08127fc700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0808038f10 0x7f080803b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:50:57.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.881+0000 7f08127fc700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0808038f10 0x7f080803b3d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0804006fd0 tx=0x7f0804006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:50:57.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:57.886+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f080c029770 con 0x7f081c101250 2024-01-11T17:50:58.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.107+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f081c10b340 con 0x7f0808038f10 2024-01-11T17:50:58.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.115+0000 7f0818ff9700 1 -- 172.21.15.38:0/3892019067 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f081c10b340 con 0x7f0808038f10 2024-01-11T17:50:58.115 INFO:teuthology.orchestra.run.smithi038.stdout:Scheduled mon update... 2024-01-11T17:50:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0808038f10 msgr2=0x7f080803b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0808038f10 0x7f080803b3d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0804006fd0 tx=0x7f0804006e40 comp rx=0 tx=0).stop 2024-01-11T17:50:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 msgr2=0x7f081c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:50:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c19c620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f080c004f40 tx=0x7f080c005020 comp rx=0 tx=0).stop 2024-01-11T17:50:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 shutdown_connections 2024-01-11T17:50:58.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0808038f10 0x7f080803b3d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 --2- 172.21.15.38:0/3892019067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c101250 0x7f081c19c620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:50:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.117+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 >> 172.21.15.38:0/3892019067 conn(0x7f081c0fcb00 msgr2=0x7f081c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:50:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.118+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 shutdown_connections 2024-01-11T17:50:58.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:50:58.118+0000 7f08213a6700 1 -- 172.21.15.38:0/3892019067 wait complete. 2024-01-11T17:50:58.531 DEBUG:teuthology.orchestra.run.smithi016:mon.b> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.b.service 2024-01-11T17:50:58.534 DEBUG:teuthology.orchestra.run.smithi038:mon.c> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.c.service 2024-01-11T17:50:58.536 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-11T17:50:58.536 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph mon dump -f json 2024-01-11T17:50:58.570 INFO:journalctl@ceph.mon.c.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.client.admin.keyring 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:50:58.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:50:58.574 INFO:journalctl@ceph.mon.b.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:50:59.175 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:50:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:59 smithi007 ceph-mon[33361]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:50:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:59 smithi007 ceph-mon[33361]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c;count:3 2024-01-11T17:50:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:59 smithi007 ceph-mon[33361]: Deploying daemon mon.c on smithi038 2024-01-11T17:50:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:50:59 smithi007 ceph-mon[33361]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 -- 172.21.15.38:0/3974817358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 msgr2=0x7fd7e8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 --2- 172.21.15.38:0/3974817358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e8100530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009910 tx=0x7fd7d8009c20 comp rx=0 tx=0).stop 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 -- 172.21.15.38:0/3974817358 shutdown_connections 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 --2- 172.21.15.38:0/3974817358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e8100530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 -- 172.21.15.38:0/3974817358 >> 172.21.15.38:0/3974817358 conn(0x7fd7e80fb9e0 msgr2=0x7fd7e80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 -- 172.21.15.38:0/3974817358 shutdown_connections 2024-01-11T17:51:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.917+0000 7fd7ed927700 1 -- 172.21.15.38:0/3974817358 wait complete. 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.918+0000 7fd7ed927700 1 Processor -- start 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.918+0000 7fd7ed927700 1 -- start start 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.918+0000 7fd7ed927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.918+0000 7fd7ed927700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e819cb10 con 0x7fd7e8100150 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.919+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.919+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e819c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.38:60624/0 (socket says 172.21.15.38:60624) 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.919+0000 7fd7e6ffd700 1 -- 172.21.15.38:0/3361353751 learned_addr learned my addr 172.21.15.38:0/3361353751 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:51:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.919+0000 7fd7e6ffd700 1 -- 172.21.15.38:0/3361353751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80095f0 con 0x7fd7e8100150 2024-01-11T17:51:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.919+0000 7fd7e6ffd700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e819c5d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004f40 tx=0x7fd7d8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.919+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7d801d070 con 0x7fd7e8100150 2024-01-11T17:51:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.920+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e819cd10 con 0x7fd7e8100150 2024-01-11T17:51:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.920+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d800bc10 con 0x7fd7e8100150 2024-01-11T17:51:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.920+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7d80037d0 con 0x7fd7e8100150 2024-01-11T17:51:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.920+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e81966a0 con 0x7fd7e8100150 2024-01-11T17:51:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.921+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7fd7d800b6f0 con 0x7fd7e8100150 2024-01-11T17:51:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.921+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e819d130 con 0x7fd7e8100150 2024-01-11T17:51:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.921+0000 7fd7ec925700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d0038f10 0x7fd7d003b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.921+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd7d804cce0 con 0x7fd7e8100150 2024-01-11T17:51:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.921+0000 7fd7e67fc700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d0038f10 0x7fd7d003b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.922+0000 7fd7e67fc700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d0038f10 0x7fd7d003b3d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc006fd0 tx=0x7fd7dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:00.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:00.927+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd7d802ad40 con 0x7fd7e8100150 2024-01-11T17:51:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.250+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd7e8197c60 con 0x7fd7e8100150 2024-01-11T17:51:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.251+0000 7fd7ec925700 1 -- 172.21.15.38:0/3361353751 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+741 (secure 0 0 0) 0x7fd7d8026070 con 0x7fd7e8100150 2024-01-11T17:51:01.252 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:51:01.252 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":1,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","modified":"2024-01-11T17:49:21.415570Z","created":"2024-01-11T17:49:21.415570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-01-11T17:51:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d0038f10 msgr2=0x7fd7d003b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d0038f10 0x7fd7d003b3d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc006fd0 tx=0x7fd7dc006e40 comp rx=0 tx=0).stop 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 msgr2=0x7fd7e819c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e819c5d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8004f40 tx=0x7fd7d8005020 comp rx=0 tx=0).stop 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 shutdown_connections 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d0038f10 0x7fd7d003b3d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 --2- 172.21.15.38:0/3361353751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8100150 0x7fd7e819c5d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 >> 172.21.15.38:0/3361353751 conn(0x7fd7e80fb9e0 msgr2=0x7fd7e8069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.253+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 shutdown_connections 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:01.254+0000 7fd7ed927700 1 -- 172.21.15.38:0/3361353751 wait complete. 2024-01-11T17:51:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:dumped monmap epoch 1 2024-01-11T17:51:01.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:01 smithi007 ceph-mon[33361]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:01.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:01 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/3361353751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 systemd[1]: Started Ceph mon.c for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: load: jerasure load: lrc 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: RocksDB version: 7.9.2 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Git sha 0 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Compile date 2024-01-10 10:08:18 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: DB SUMMARY 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: DB Session ID: 4P9DDLVTX8TEU5E5PO8P 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: CURRENT file: CURRENT 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: IDENTITY file: IDENTITY 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-01-11T17:51:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.error_if_exists: 0 2024-01-11T17:51:02.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.create_if_missing: 0 2024-01-11T17:51:02.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.paranoid_checks: 1 2024-01-11T17:51:02.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.flush_verify_memtable_count: 1 2024-01-11T17:51:02.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.env: 0x55cf02c90a60 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.fs: PosixFileSystem 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.info_log: 0x55cf03dc9760 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_file_opening_threads: 16 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.statistics: (nil) 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.use_fsync: 0 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_log_file_size: 0 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.log_file_time_to_roll: 0 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.keep_log_file_num: 1000 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.recycle_log_file_num: 0 2024-01-11T17:51:02.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.allow_fallocate: 1 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.allow_mmap_reads: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.allow_mmap_writes: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.use_direct_reads: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.create_missing_column_families: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.db_log_dir: 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.wal_dir: 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.table_cache_numshardbits: 6 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.WAL_ttl_seconds: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.WAL_size_limit_MB: 0 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.is_fd_close_on_exec: 1 2024-01-11T17:51:02.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.advise_random_on_open: 1 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.db_write_buffer_size: 0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.write_buffer_manager: 0x55cf03d7f4a0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.use_adaptive_mutex: 0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.rate_limiter: (nil) 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.wal_recovery_mode: 2 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.enable_thread_tracking: 0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.enable_pipelined_write: 0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.unordered_write: 0 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-01-11T17:51:02.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.row_cache: None 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.wal_filter: None 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.allow_ingest_behind: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.two_write_queues: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.manual_wal_flush: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.wal_compression: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.atomic_flush: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.persist_stats_to_disk: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.write_dbid_to_manifest: 0 2024-01-11T17:51:02.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.log_readahead_size: 0 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.best_efforts_recovery: 0 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.allow_data_in_errors: 0 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.db_host_id: __hostname__ 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.enforce_single_del_contracts: true 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_background_jobs: 2 2024-01-11T17:51:02.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_background_compactions: -1 2024-01-11T17:51:02.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_subcompactions: 1 2024-01-11T17:51:02.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-01-11T17:51:02.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-01-11T17:51:02.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.delayed_write_rate : 16777216 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_total_wal_size: 0 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.stats_dump_period_sec: 600 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.stats_persist_period_sec: 600 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_open_files: -1 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bytes_per_sync: 0 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.wal_bytes_per_sync: 0 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.strict_bytes_per_sync: 0 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_readahead_size: 0 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_background_flushes: -1 2024-01-11T17:51:02.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Compression algorithms supported: 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kZSTD supported: 0 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kXpressCompression supported: 0 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kLZ4HCCompression supported: 1 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kZlibCompression supported: 1 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kSnappyCompression supported: 1 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kLZ4Compression supported: 1 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: kBZip2Compression supported: 0 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Fast CRC32 supported: Supported on x86 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: DMutex implementation: pthread_mutex_t 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-01-11T17:51:02.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.merge_operator: 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_filter: None 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_filter_factory: None 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.sst_partitioner_factory: None 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.memtable_factory: SkipListFactory 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.table_factory: BlockBasedTable 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cf03dc9880) 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: index_type: 0 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: data_block_index_type: 0 2024-01-11T17:51:02.040 INFO:journalctl@ceph.mon.c.smithi038.stdout: index_shortening: 1 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: checksum: 4 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: no_block_cache: 0 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache: 0x55cf03e1f090 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache_options: 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: capacity : 536870912 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: num_shard_bits : 4 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: strict_capacity_limit : 0 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache_compressed: (nil) 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: persistent_cache: (nil) 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_size: 4096 2024-01-11T17:51:02.041 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_size_deviation: 10 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_restart_interval: 16 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: index_block_restart_interval: 1 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: metadata_block_size: 4096 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: partition_filters: 0 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: use_delta_encoding: 1 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: filter_policy: bloomfilter 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: whole_key_filtering: 1 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: verify_compression: 0 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: format_version: 5 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: enable_index_compression: 1 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_align: 0 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: max_auto_readahead_size: 262144 2024-01-11T17:51:02.042 INFO:journalctl@ceph.mon.c.smithi038.stdout: prepopulate_block_cache: 0 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout: initial_auto_readahead_size: 8192 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.write_buffer_size: 33554432 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_write_buffer_number: 2 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression: NoCompression 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression: Disabled 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.prefix_extractor: nullptr 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.num_levels: 7 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-01-11T17:51:02.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.window_bits: -14 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.level: 32767 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.strategy: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-01-11T17:51:02.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.enabled: false 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.target_file_size_base: 67108864 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.target_file_size_multiplier: 1 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-01-11T17:51:02.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.arena_block_size: 1048576 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.disable_auto_compactions: 0 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-01-11T17:51:02.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.table_properties_collectors: 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.inplace_update_support: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.inplace_update_num_locks: 10000 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.memtable_huge_page_size: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.bloom_locality: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.max_successive_merges: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.optimize_filters_for_hits: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.paranoid_file_checks: 0 2024-01-11T17:51:02.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.force_consistency_checks: 1 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.report_bg_io_stats: 0 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.ttl: 2592000 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.periodic_compaction_seconds: 0 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.enable_blob_files: false 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.min_blob_size: 0 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.blob_file_size: 268435456 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.blob_compression_type: NoCompression 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.enable_blob_garbage_collection: false 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-01-11T17:51:02.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.blob_file_starting_level: 0 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 648ea93a-7011-4b88-b561-9c4a5eea7cdd 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995461783852, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995461784610, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1704995461, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "648ea93a-7011-4b88-b561-9c4a5eea7cdd", "db_session_id": "4P9DDLVTX8TEU5E5PO8P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995461784732, "job": 1, "event": "recovery_finished"} 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cf03eba000 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: DB pointer 0x55cf03ea8000 2024-01-11T17:51:02.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** DB Stats ** 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** Compaction Stats [default] ** 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-01-11T17:51:02.050 INFO:journalctl@ceph.mon.c.smithi038.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** Compaction Stats [default] ** 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-01-11T17:51:02.051 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: Block cache BinnedLRUCache@0x55cf03e1f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: using public_addr v2:172.21.15.38:0/0 -> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: starting mon.c rank -1 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(???) e0 preinit fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).mds e1 new map 2024-01-11T17:51:02.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).mds e1 print_map 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout: e1 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout: legacy client fscid: -1 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout: No filesystems configured 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-01-11T17:51:02.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Added host smithi038 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/250824849' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Updating smithi038:/etc/ceph/ceph.conf 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/250824849' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:02.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.client.admin.keyring 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:02.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:02.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c;count:3 2024-01-11T17:51:02.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: Deploying daemon mon.c on smithi038 2024-01-11T17:51:02.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:02.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:02.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/3361353751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:02.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:01 smithi038 ceph-mon[43582]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-01-11T17:51:02.614 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-11T17:51:02.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph mon dump -f json 2024-01-11T17:51:02.916 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 -- 172.21.15.38:0/3295753481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe810102320 msgr2=0x7fe7f8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 --2- 172.21.15.38:0/3295753481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe810102320 0x7fe7f8005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe800005fa0 tx=0x7fe80000ffc0 comp rx=0 tx=0).stop 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 -- 172.21.15.38:0/3295753481 shutdown_connections 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 --2- 172.21.15.38:0/3295753481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe810102320 0x7fe7f8005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 -- 172.21.15.38:0/3295753481 >> 172.21.15.38:0/3295753481 conn(0x7fe8100fbab0 msgr2=0x7fe8100fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 -- 172.21.15.38:0/3295753481 shutdown_connections 2024-01-11T17:51:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.939+0000 7fe81506b700 1 -- 172.21.15.38:0/3295753481 wait complete. 2024-01-11T17:51:06.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.940+0000 7fe81506b700 1 Processor -- start 2024-01-11T17:51:06.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe81506b700 1 -- start start 2024-01-11T17:51:06.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe81506b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:06.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe81506b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe81019ed80 0x7fe810198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:06.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe81506b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019f310 con 0x7fe81019ed80 2024-01-11T17:51:06.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe81506b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019f480 con 0x7fe810102320 2024-01-11T17:51:06.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37242/0 (socket says 172.21.15.38:37242) 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 -- 172.21.15.38:0/1710947821 learned_addr learned my addr 172.21.15.38:0/1710947821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80e59c700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe81019ed80 0x7fe810198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 -- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 msgr2=0x7fe81019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 -- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 msgr2=0x7fe81019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.941+0000 7fe80ed9d700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe80e59c700 1 -- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 msgr2=0x7fe81019e840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe80e59c700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe80e59c700 1 -- 172.21.15.38:0/1710947821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80000f9c0 con 0x7fe81019ed80 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe80e59c700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe81019ed80 0x7fe810198a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b4d0 tx=0x7fe80400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe804011630 con 0x7fe81019ed80 2024-01-11T17:51:06.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe804011c50 con 0x7fe81019ed80 2024-01-11T17:51:06.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe810199000 con 0x7fe81019ed80 2024-01-11T17:51:06.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe804010d90 con 0x7fe81019ed80 2024-01-11T17:51:06.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.942+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe810199550 con 0x7fe81019ed80 2024-01-11T17:51:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.943+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7fe80401a400 con 0x7fe81019ed80 2024-01-11T17:51:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.944+0000 7fe7f7fff700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe7fc038ca0 0x7fe7fc03b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.944+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe80404c5e0 con 0x7fe81019ed80 2024-01-11T17:51:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.944+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe810106880 con 0x7fe81019ed80 2024-01-11T17:51:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.944+0000 7fe80ed9d700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe7fc038ca0 0x7fe7fc03b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.945+0000 7fe80ed9d700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe7fc038ca0 0x7fe7fc03b160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe800005b40 tx=0x7fe80000f400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:06.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:06.950+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe8040193b0 con 0x7fe81019ed80 2024-01-11T17:51:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.251+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe810062ca0 con 0x7fe81019ed80 2024-01-11T17:51:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.252+0000 7fe7f7fff700 1 -- 172.21.15.38:0/1710947821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1007 (secure 0 0 0) 0x7fe80403f090 con 0x7fe81019ed80 2024-01-11T17:51:07.252 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:51:07.252 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","modified":"2024-01-11T17:51:01.816730Z","created":"2024-01-11T17:49:21.415570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-01-11T17:51:07.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe7fc038ca0 msgr2=0x7fe7fc03b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe7fc038ca0 0x7fe7fc03b160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe800005b40 tx=0x7fe80000f400 comp rx=0 tx=0).stop 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe81019ed80 msgr2=0x7fe810198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe81019ed80 0x7fe810198a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b4d0 tx=0x7fe80400b890 comp rx=0 tx=0).stop 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 shutdown_connections 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe7fc038ca0 0x7fe7fc03b160 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe810102320 0x7fe81019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 --2- 172.21.15.38:0/1710947821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe81019ed80 0x7fe810198a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 >> 172.21.15.38:0/1710947821 conn(0x7fe8100fbab0 msgr2=0x7fe8100fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 shutdown_connections 2024-01-11T17:51:07.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:07.254+0000 7fe81506b700 1 -- 172.21.15.38:0/1710947821 wait complete. 2024-01-11T17:51:07.256 INFO:teuthology.orchestra.run.smithi038.stderr:dumped monmap epoch 2 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: mon.a calling monitor election 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: mon.c calling monitor election 2024-01-11T17:51:07.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: monmap e2: 2 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],c=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: fsmap 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: mgrmap e13: x(active, since 42s) 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: overall HEALTH_OK 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:07.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: mon.a calling monitor election 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: mon.c calling monitor election 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: monmap e2: 2 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],c=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: fsmap 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: mgrmap e13: x(active, since 42s) 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: overall HEALTH_OK 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:07.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:08.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:08.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:08.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:08.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:07 smithi038 ceph-mon[43582]: Deploying daemon mon.b on smithi016 2024-01-11T17:51:08.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/1710947821' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:08.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:07 smithi007 ceph-mon[33361]: Deploying daemon mon.b on smithi016 2024-01-11T17:51:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/1710947821' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:08.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:08.677 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-11T17:51:08.677 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph mon dump -f json 2024-01-11T17:51:08.977 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:51:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:08 smithi038 ceph-mon[43582]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:08 smithi007 ceph-mon[33361]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:09.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:51:08 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:51:08.815+0000 7fdc355ec700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-11T17:51:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 systemd[1]: Started Ceph mon.b for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:51:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: set uid:gid to 167:167 (ceph:ceph) 2024-01-11T17:51:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev), process ceph-mon, pid 7 2024-01-11T17:51:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pidfile_write: ignore empty --pid-file 2024-01-11T17:51:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: load: jerasure load: lrc 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: RocksDB version: 7.9.2 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Git sha 0 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Compile date 2024-01-10 10:08:18 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: DB SUMMARY 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: DB Session ID: RDDZY0KVOYP5K4VVW3J2 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: CURRENT file: CURRENT 2024-01-11T17:51:09.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: IDENTITY file: IDENTITY 2024-01-11T17:51:09.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-01-11T17:51:09.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-01-11T17:51:09.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-01-11T17:51:09.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.error_if_exists: 0 2024-01-11T17:51:09.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.create_if_missing: 0 2024-01-11T17:51:09.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.paranoid_checks: 1 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.flush_verify_memtable_count: 1 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.env: 0x563657ae9a60 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.fs: PosixFileSystem 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.info_log: 0x563658a39760 2024-01-11T17:51:10.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_file_opening_threads: 16 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.statistics: (nil) 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.use_fsync: 0 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_log_file_size: 0 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.log_file_time_to_roll: 0 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.keep_log_file_num: 1000 2024-01-11T17:51:10.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.recycle_log_file_num: 0 2024-01-11T17:51:10.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.allow_fallocate: 1 2024-01-11T17:51:10.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.allow_mmap_reads: 0 2024-01-11T17:51:10.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.allow_mmap_writes: 0 2024-01-11T17:51:10.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.use_direct_reads: 0 2024-01-11T17:51:10.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-01-11T17:51:10.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.create_missing_column_families: 0 2024-01-11T17:51:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.db_log_dir: 2024-01-11T17:51:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.wal_dir: 2024-01-11T17:51:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.table_cache_numshardbits: 6 2024-01-11T17:51:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.WAL_ttl_seconds: 0 2024-01-11T17:51:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.WAL_size_limit_MB: 0 2024-01-11T17:51:10.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-01-11T17:51:10.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-01-11T17:51:10.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.is_fd_close_on_exec: 1 2024-01-11T17:51:10.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.advise_random_on_open: 1 2024-01-11T17:51:10.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.db_write_buffer_size: 0 2024-01-11T17:51:10.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.write_buffer_manager: 0x5636589ef4a0 2024-01-11T17:51:10.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-01-11T17:51:10.005 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-01-11T17:51:10.005 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.use_adaptive_mutex: 0 2024-01-11T17:51:10.005 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.rate_limiter: (nil) 2024-01-11T17:51:10.005 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-01-11T17:51:10.006 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.wal_recovery_mode: 2 2024-01-11T17:51:10.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.enable_thread_tracking: 0 2024-01-11T17:51:10.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.enable_pipelined_write: 0 2024-01-11T17:51:10.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.unordered_write: 0 2024-01-11T17:51:10.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-01-11T17:51:10.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-01-11T17:51:10.007 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-01-11T17:51:10.008 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-01-11T17:51:10.008 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.row_cache: None 2024-01-11T17:51:10.008 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.wal_filter: None 2024-01-11T17:51:10.008 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-01-11T17:51:10.008 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.allow_ingest_behind: 0 2024-01-11T17:51:10.008 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.two_write_queues: 0 2024-01-11T17:51:10.009 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.manual_wal_flush: 0 2024-01-11T17:51:10.009 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.wal_compression: 0 2024-01-11T17:51:10.009 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.atomic_flush: 0 2024-01-11T17:51:10.009 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-01-11T17:51:10.009 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.persist_stats_to_disk: 0 2024-01-11T17:51:10.009 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.write_dbid_to_manifest: 0 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.log_readahead_size: 0 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.best_efforts_recovery: 0 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.allow_data_in_errors: 0 2024-01-11T17:51:10.010 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.db_host_id: __hostname__ 2024-01-11T17:51:10.013 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.enforce_single_del_contracts: true 2024-01-11T17:51:10.014 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_background_jobs: 2 2024-01-11T17:51:10.014 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_background_compactions: -1 2024-01-11T17:51:10.014 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_subcompactions: 1 2024-01-11T17:51:10.014 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-01-11T17:51:10.014 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-01-11T17:51:10.014 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.delayed_write_rate : 16777216 2024-01-11T17:51:10.015 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_total_wal_size: 0 2024-01-11T17:51:10.015 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-01-11T17:51:10.015 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.stats_dump_period_sec: 600 2024-01-11T17:51:10.015 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.stats_persist_period_sec: 600 2024-01-11T17:51:10.015 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-01-11T17:51:10.015 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_open_files: -1 2024-01-11T17:51:10.016 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bytes_per_sync: 0 2024-01-11T17:51:10.016 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.wal_bytes_per_sync: 0 2024-01-11T17:51:10.016 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.strict_bytes_per_sync: 0 2024-01-11T17:51:10.016 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_readahead_size: 0 2024-01-11T17:51:10.016 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_background_flushes: -1 2024-01-11T17:51:10.016 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Compression algorithms supported: 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kZSTD supported: 0 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kXpressCompression supported: 0 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kLZ4HCCompression supported: 1 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kZlibCompression supported: 1 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kSnappyCompression supported: 1 2024-01-11T17:51:10.017 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kLZ4Compression supported: 1 2024-01-11T17:51:10.018 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: kBZip2Compression supported: 0 2024-01-11T17:51:10.018 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Fast CRC32 supported: Supported on x86 2024-01-11T17:51:10.018 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: DMutex implementation: pthread_mutex_t 2024-01-11T17:51:10.018 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-01-11T17:51:10.018 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-01-11T17:51:10.018 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-01-11T17:51:10.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.merge_operator: 2024-01-11T17:51:10.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_filter: None 2024-01-11T17:51:10.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_filter_factory: None 2024-01-11T17:51:10.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.sst_partitioner_factory: None 2024-01-11T17:51:10.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.memtable_factory: SkipListFactory 2024-01-11T17:51:10.020 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.table_factory: BlockBasedTable 2024-01-11T17:51:10.020 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563658a39880) 2024-01-11T17:51:10.020 INFO:journalctl@ceph.mon.b.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-01-11T17:51:10.020 INFO:journalctl@ceph.mon.b.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-01-11T17:51:10.020 INFO:journalctl@ceph.mon.b.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-01-11T17:51:10.020 INFO:journalctl@ceph.mon.b.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-01-11T17:51:10.021 INFO:journalctl@ceph.mon.b.smithi016.stdout: index_type: 0 2024-01-11T17:51:10.021 INFO:journalctl@ceph.mon.b.smithi016.stdout: data_block_index_type: 0 2024-01-11T17:51:10.021 INFO:journalctl@ceph.mon.b.smithi016.stdout: index_shortening: 1 2024-01-11T17:51:10.021 INFO:journalctl@ceph.mon.b.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-01-11T17:51:10.021 INFO:journalctl@ceph.mon.b.smithi016.stdout: checksum: 4 2024-01-11T17:51:10.021 INFO:journalctl@ceph.mon.b.smithi016.stdout: no_block_cache: 0 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache: 0x563658a8f090 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache_options: 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: capacity : 536870912 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: num_shard_bits : 4 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: strict_capacity_limit : 0 2024-01-11T17:51:10.022 INFO:journalctl@ceph.mon.b.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache_compressed: (nil) 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: persistent_cache: (nil) 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_size: 4096 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_size_deviation: 10 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_restart_interval: 16 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: index_block_restart_interval: 1 2024-01-11T17:51:10.023 INFO:journalctl@ceph.mon.b.smithi016.stdout: metadata_block_size: 4096 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: partition_filters: 0 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: use_delta_encoding: 1 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: filter_policy: bloomfilter 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: whole_key_filtering: 1 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: verify_compression: 0 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: format_version: 5 2024-01-11T17:51:10.024 INFO:journalctl@ceph.mon.b.smithi016.stdout: enable_index_compression: 1 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_align: 0 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout: max_auto_readahead_size: 262144 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout: prepopulate_block_cache: 0 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout: initial_auto_readahead_size: 8192 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.write_buffer_size: 33554432 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_write_buffer_number: 2 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression: NoCompression 2024-01-11T17:51:10.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression: Disabled 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.prefix_extractor: nullptr 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.num_levels: 7 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-01-11T17:51:10.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.window_bits: -14 2024-01-11T17:51:10.027 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.level: 32767 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.strategy: 0 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.enabled: false 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.target_file_size_base: 67108864 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.target_file_size_multiplier: 1 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-01-11T17:51:10.028 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.arena_block_size: 1048576 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-01-11T17:51:10.029 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.disable_auto_compactions: 0 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.table_properties_collectors: 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.inplace_update_support: 0 2024-01-11T17:51:10.030 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.inplace_update_num_locks: 10000 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.memtable_huge_page_size: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.bloom_locality: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.max_successive_merges: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.optimize_filters_for_hits: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.paranoid_file_checks: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.force_consistency_checks: 1 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.report_bg_io_stats: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.ttl: 2592000 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.periodic_compaction_seconds: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.enable_blob_files: false 2024-01-11T17:51:10.031 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.min_blob_size: 0 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.blob_file_size: 268435456 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.blob_compression_type: NoCompression 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.enable_blob_garbage_collection: false 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.blob_file_starting_level: 0 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8a37e86b-c2c8-4bb2-adbf-f52126b2b8fe 2024-01-11T17:51:10.032 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995469645064, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995469645556, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1704995469, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8a37e86b-c2c8-4bb2-adbf-f52126b2b8fe", "db_session_id": "RDDZY0KVOYP5K4VVW3J2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1704995469645641, "job": 1, "event": "recovery_finished"} 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563658b2a000 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: DB pointer 0x563658b18000 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** DB Stats ** 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-11T17:51:10.033 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** Compaction Stats [default] ** 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** Compaction Stats [default] ** 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-11T17:51:10.034 INFO:journalctl@ceph.mon.b.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Block cache BinnedLRUCache@0x563658a8f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-01-11T17:51:10.035 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: using public_addr v2:172.21.15.16:0/0 -> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: starting mon.b rank -1 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(???) e0 preinit fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).mds e1 new map 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).mds e1 print_map 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout: e1 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-11T17:51:10.036 INFO:journalctl@ceph.mon.b.smithi016.stdout: legacy client fscid: -1 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout: No filesystems configured 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-01-11T17:51:10.037 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Added host smithi038 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/250824849' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-01-11T17:51:10.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Updating smithi038:/etc/ceph/ceph.conf 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/250824849' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.039 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.client.admin.keyring 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c;count:3 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Deploying daemon mon.c on smithi038 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.040 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/3361353751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.a calling monitor election 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.c calling monitor election 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: monmap e2: 2 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],c=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:51:10.041 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: fsmap 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mgrmap e13: x(active, since 42s) 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: overall HEALTH_OK 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: Deploying daemon mon.b on smithi016 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/1710947821' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:10.042 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:09 smithi016 ceph-mon[43195]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-01-11T17:51:15.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:51:15.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: mon.a calling monitor election 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: mon.c calling monitor election 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: mon.b calling monitor election 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: fsmap 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: mgrmap e13: x(active, since 50s) 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: overall HEALTH_OK 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:14 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:51:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: mon.a calling monitor election 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: mon.c calling monitor election 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: mon.b calling monitor election 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: fsmap 2024-01-11T17:51:15.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:15.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: mgrmap e13: x(active, since 50s) 2024-01-11T17:51:15.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: overall HEALTH_OK 2024-01-11T17:51:15.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:14 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: mon.a calling monitor election 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: mon.c calling monitor election 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: mon.b calling monitor election 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: fsmap 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: osdmap e4: 0 total, 0 up, 0 in 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: mgrmap e13: x(active, since 50s) 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: overall HEALTH_OK 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:14 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 -- 172.21.15.38:0/595995552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278006230 msgr2=0x7f12780086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 --2- 172.21.15.38:0/595995552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278006230 0x7f12780086a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f128c0699e0 tx=0x7f127c00fe90 comp rx=0 tx=0).stop 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 -- 172.21.15.38:0/595995552 shutdown_connections 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 --2- 172.21.15.38:0/595995552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278006230 0x7f12780086a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 -- 172.21.15.38:0/595995552 >> 172.21.15.38:0/595995552 conn(0x7f128c0fbb30 msgr2=0x7f128c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 -- 172.21.15.38:0/595995552 shutdown_connections 2024-01-11T17:51:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.931+0000 7f12923b2700 1 -- 172.21.15.38:0/595995552 wait complete. 2024-01-11T17:51:15.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.933+0000 7f12923b2700 1 Processor -- start 2024-01-11T17:51:15.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.933+0000 7f12923b2700 1 -- start start 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f12923b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f12923b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f128c19cda0 0x7f128c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f12923b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 0x7f128c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f12923b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19d460 con 0x7f128c19cda0 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f12923b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c1a3730 con 0x7f128c196f60 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f12923b2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c1a3870 con 0x7f1278006230 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48982/0 (socket says 172.21.15.38:48982) 2024-01-11T17:51:15.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 -- 172.21.15.38:0/2829825256 learned_addr learned my addr 172.21.15.38:0/2829825256 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128b7fe700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f128c19cda0 0x7f128c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f129094f700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 0x7f128c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 msgr2=0x7f128c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 msgr2=0x7f128c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128bfff700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f129094f700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 msgr2=0x7f128c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f129094f700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 msgr2=0x7f128c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f129094f700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 0x7f128c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f129094f700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 0x7f128c197410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T17:51:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128b7fe700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 msgr2=0x7f128c19c860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128b7fe700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128b7fe700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 msgr2=0x7f128c197410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128b7fe700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 0x7f128c197410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.934+0000 7f128b7fe700 1 -- 172.21.15.38:0/2829825256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c00f8e0 con 0x7f128c19cda0 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.935+0000 7f128b7fe700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f128c19cda0 0x7f128c196a20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f127400b550 tx=0x7f127400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.935+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274004230 con 0x7f128c19cda0 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.935+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c1a3b50 con 0x7f128c19cda0 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.935+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1274004390 con 0x7f128c19cda0 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.935+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274011af0 con 0x7f128c19cda0 2024-01-11T17:51:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.935+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c1a40a0 con 0x7f128c19cda0 2024-01-11T17:51:15.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.936+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f1274011c50 con 0x7f128c19cda0 2024-01-11T17:51:15.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.937+0000 7f12897fa700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f126c039340 0x7f126c03b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:15.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.937+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c062c40 con 0x7f128c19cda0 2024-01-11T17:51:15.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.937+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f127404b570 con 0x7f128c19cda0 2024-01-11T17:51:15.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.937+0000 7f128bfff700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f126c039340 0x7f126c03b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:15.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.938+0000 7f128bfff700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f126c039340 0x7f126c03b800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f127c005b40 tx=0x7f127c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:15.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:15.943+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f127401a400 con 0x7f128c19cda0 2024-01-11T17:51:16.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:15 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:16.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:15 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:16.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:15 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:16.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:15 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:16.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:15 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:16.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:15 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:16.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:15 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:16.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:15 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:16.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:15 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:51:16.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.249+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f128c108e70 con 0x7f128c19cda0 2024-01-11T17:51:16.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.249+0000 7f12897fa700 1 -- 172.21.15.38:0/2829825256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1273 (secure 0 0 0) 0x7f127400f2e0 con 0x7f128c19cda0 2024-01-11T17:51:16.250 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-01-11T17:51:16.250 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","modified":"2024-01-11T17:51:09.672336Z","created":"2024-01-11T17:49:21.415570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-01-11T17:51:16.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f126c039340 msgr2=0x7f126c03b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:16.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f126c039340 0x7f126c03b800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f127c005b40 tx=0x7f127c005a90 comp rx=0 tx=0).stop 2024-01-11T17:51:16.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f128c19cda0 msgr2=0x7f128c196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f128c19cda0 0x7f128c196a20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f127400b550 tx=0x7f127400b910 comp rx=0 tx=0).stop 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 shutdown_connections 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f126c039340 0x7f126c03b800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1278006230 0x7f128c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c196f60 0x7f128c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 --2- 172.21.15.38:0/2829825256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f128c19cda0 0x7f128c196a20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 >> 172.21.15.38:0/2829825256 conn(0x7f128c0fbb30 msgr2=0x7f128c1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 shutdown_connections 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:16.252+0000 7f12923b2700 1 -- 172.21.15.38:0/2829825256 wait complete. 2024-01-11T17:51:16.253 INFO:teuthology.orchestra.run.smithi038.stderr:dumped monmap epoch 3 2024-01-11T17:51:16.703 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-01-11T17:51:16.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph config generate-minimal-conf 2024-01-11T17:51:17.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: Updating smithi007:/etc/ceph/ceph.conf 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: Updating smithi016:/etc/ceph/ceph.conf 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: Updating smithi038:/etc/ceph/ceph.conf 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: Updating smithi016:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/2829825256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: Updating smithi007:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:51:16.671+0000 7fdc355ec700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: Updating smithi007:/etc/ceph/ceph.conf 2024-01-11T17:51:17.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: Updating smithi016:/etc/ceph/ceph.conf 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: Updating smithi038:/etc/ceph/ceph.conf 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: Updating smithi016:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/2829825256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: Updating smithi007:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: Updating smithi007:/etc/ceph/ceph.conf 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: Updating smithi016:/etc/ceph/ceph.conf 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: Updating smithi038:/etc/ceph/ceph.conf 2024-01-11T17:51:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: Updating smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: Updating smithi016:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/2829825256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: Updating smithi007:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/config/ceph.conf 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:17.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: Reconfiguring mon.a (unknown last config time)... 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:18.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:18 smithi007 ceph-mon[33361]: Reconfiguring daemon mon.a on smithi007 2024-01-11T17:51:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 -- 172.21.15.7:0/3368566824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d80739b0 msgr2=0x7f95d8073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 --2- 172.21.15.7:0/3368566824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d80739b0 0x7f95d8073d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009910 tx=0x7f95c8009c20 comp rx=0 tx=0).stop 2024-01-11T17:51:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 -- 172.21.15.7:0/3368566824 shutdown_connections 2024-01-11T17:51:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 --2- 172.21.15.7:0/3368566824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d80739b0 0x7f95d8073d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 -- 172.21.15.7:0/3368566824 >> 172.21.15.7:0/3368566824 conn(0x7f95d80fcaf0 msgr2=0x7f95d80fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 -- 172.21.15.7:0/3368566824 shutdown_connections 2024-01-11T17:51:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.342+0000 7f95de110700 1 -- 172.21.15.7:0/3368566824 wait complete. 2024-01-11T17:51:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.343+0000 7f95de110700 1 Processor -- start 2024-01-11T17:51:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 -- start start 2024-01-11T17:51:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d80739b0 0x7f95d819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95d819d560 0x7f95d81a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 0x7f95d81a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819dc50 con 0x7f95d81a1e30 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d81a9ba0 con 0x7f95d80739b0 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.344+0000 7f95de110700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d81a9ce0 con 0x7f95d819d560 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 0x7f95d81a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 0x7f95d81a22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56244/0 (socket says 172.21.15.7:56244) 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 -- 172.21.15.7:0/2302878243 learned_addr learned my addr 172.21.15.7:0/2302878243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d6ffd700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95d819d560 0x7f95d81a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d77fe700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d80739b0 0x7f95d819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 -- 172.21.15.7:0/2302878243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95d819d560 msgr2=0x7f95d81a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95d819d560 0x7f95d81a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 -- 172.21.15.7:0/2302878243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d80739b0 msgr2=0x7f95d819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d80739b0 0x7f95d819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.345+0000 7f95d7fff700 1 -- 172.21.15.7:0/2302878243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80095f0 con 0x7f95d81a1e30 2024-01-11T17:51:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.346+0000 7f95d7fff700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 0x7f95d81a22e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95d000ba50 tx=0x7f95d000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.346+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000c930 con 0x7f95d81a1e30 2024-01-11T17:51:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.346+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d0014440 con 0x7f95d81a1e30 2024-01-11T17:51:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.346+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d81a9fc0 con 0x7f95d81a1e30 2024-01-11T17:51:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.346+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d00075b0 con 0x7f95d81a1e30 2024-01-11T17:51:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.346+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d81aa480 con 0x7f95d81a1e30 2024-01-11T17:51:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.347+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d8062c40 con 0x7f95d81a1e30 2024-01-11T17:51:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.347+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f95d000ca90 con 0x7f95d81a1e30 2024-01-11T17:51:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.348+0000 7f95d4ff9700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95c0039340 0x7f95c003b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.348+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f95d004b210 con 0x7f95d81a1e30 2024-01-11T17:51:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.348+0000 7f95d77fe700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95c0039340 0x7f95c003b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.349+0000 7f95d77fe700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95c0039340 0x7f95c003b800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95c8004690 tx=0x7f95c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.353+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f95d00148b0 con 0x7f95d81a1e30 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: Reconfiguring mon.a (unknown last config time)... 2024-01-11T17:51:18.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:18.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:18.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:18.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:18 smithi016 ceph-mon[43195]: Reconfiguring daemon mon.a on smithi007 2024-01-11T17:51:18.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: Reconfiguring mon.a (unknown last config time)... 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:18.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:18 smithi038 ceph-mon[43582]: Reconfiguring daemon mon.a on smithi007 2024-01-11T17:51:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.563+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f95d8111a80 con 0x7f95d81a1e30 2024-01-11T17:51:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.563+0000 7f95d4ff9700 1 -- 172.21.15.7:0/2302878243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+269 (secure 0 0 0) 0x7f95d0012340 con 0x7f95d81a1e30 2024-01-11T17:51:18.564 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:51:18.564 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-01-11T17:51:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:51:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] 2024-01-11T17:51:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95c0039340 msgr2=0x7f95c003b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95c0039340 0x7f95c003b800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95c8004690 tx=0x7f95c8005fb0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 msgr2=0x7f95d81a22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 0x7f95d81a22e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95d000ba50 tx=0x7f95d000bd60 comp rx=0 tx=0).stop 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 shutdown_connections 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95c0039340 0x7f95c003b800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d81a1e30 0x7f95d81a22e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d80739b0 0x7f95d819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 --2- 172.21.15.7:0/2302878243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95d819d560 0x7f95d81a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 >> 172.21.15.7:0/2302878243 conn(0x7f95d80fcaf0 msgr2=0x7f95d80fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.565+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 shutdown_connections 2024-01-11T17:51:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:18.566+0000 7f95de110700 1 -- 172.21.15.7:0/2302878243 wait complete. 2024-01-11T17:51:18.942 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-01-11T17:51:18.942 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:51:18.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-01-11T17:51:19.015 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:51:19.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:51:19.126 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:51:19.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-01-11T17:51:19.165 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:51:19.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:51:19.238 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:51:19.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-01-11T17:51:19.277 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:51:19.277 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2302878243' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:19.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:19.348 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:19.349 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2302878243' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:19.349 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:19.349 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:19.349 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:19.349 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:19.349 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:19.353 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-01-11T17:51:19.353 INFO:tasks.cephadm:Adding mgr.y on smithi016 2024-01-11T17:51:19.353 INFO:tasks.cephadm:Adding mgr.z on smithi038 2024-01-11T17:51:19.353 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch apply mgr '3;smithi007=x;smithi016=y;smithi038=z' 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2302878243' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:19.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:19.686 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:51:20.745 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: Reconfiguring mon.b (monmap changed)... 2024-01-11T17:51:20.745 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: Reconfiguring daemon mon.b on smithi016 2024-01-11T17:51:20.746 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:20.746 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:20.746 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:20.746 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:20.746 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:20.746 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: Reconfiguring mon.b (monmap changed)... 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: Reconfiguring daemon mon.b on smithi016 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:20.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.036+0000 7f5b229aa700 1 -- 172.21.15.38:0/3855224695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c100080 msgr2=0x7f5b1c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.036+0000 7f5b229aa700 1 --2- 172.21.15.38:0/3855224695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c100080 0x7f5b1c109c50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400b0a0 tx=0x7f5b1400b3b0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 -- 172.21.15.38:0/3855224695 shutdown_connections 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 --2- 172.21.15.38:0/3855224695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c100080 0x7f5b1c109c50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 --2- 172.21.15.38:0/3855224695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c102c60 0x7f5b1c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 --2- 172.21.15.38:0/3855224695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c0ff6e0 0x7f5b1c0ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 -- 172.21.15.38:0/3855224695 >> 172.21.15.38:0/3855224695 conn(0x7f5b1c0fb280 msgr2=0x7f5b1c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 -- 172.21.15.38:0/3855224695 shutdown_connections 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 -- 172.21.15.38:0/3855224695 wait complete. 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 Processor -- start 2024-01-11T17:51:21.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.037+0000 7f5b229aa700 1 -- start start 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b229aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c0ff6e0 0x7f5b1c102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b229aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c100080 0x7f5b1c1009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b229aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 0x7f5b1c100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b229aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c101670 con 0x7f5b1c0ff6e0 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b229aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c1017e0 con 0x7f5b1c100080 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b229aa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c1a15f0 con 0x7f5b1c102c60 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 0x7f5b1c100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 0x7f5b1c100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49008/0 (socket says 172.21.15.38:49008) 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 -- 172.21.15.38:0/1096261302 learned_addr learned my addr 172.21.15.38:0/1096261302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b1b7fe700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c100080 0x7f5b1c1009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b1bfff700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c0ff6e0 0x7f5b1c102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 -- 172.21.15.38:0/1096261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c100080 msgr2=0x7f5b1c1009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:21.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c100080 0x7f5b1c1009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 -- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c0ff6e0 msgr2=0x7f5b1c102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c0ff6e0 0x7f5b1c102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b20f47700 1 -- 172.21.15.38:0/1096261302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14009d00 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.038+0000 7f5b1b7fe700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c100080 0x7f5b1c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.039+0000 7f5b20f47700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 0x7f5b1c100ef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009030 tx=0x7f5b1400bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.039+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400e040 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.039+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b14004e60 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.039+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14005370 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.039+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c1a1870 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.039+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c1a1ce0 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.040+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7f5b140054d0 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.040+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c108c90 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.040+0000 7f5b197fa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5b040390f0 0x7f5b0403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.040+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5b1404d1b0 con 0x7f5b1c102c60 2024-01-11T17:51:21.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.040+0000 7f5b1bfff700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5b040390f0 0x7f5b0403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:21.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.041+0000 7f5b1bfff700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5b040390f0 0x7f5b0403b5b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c068fa0 tx=0x7f5b0c00bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:21.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.045+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f5b14011390 con 0x7f5b1c102c60 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: Reconfiguring mon.b (monmap changed)... 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: Reconfiguring daemon mon.b on smithi016 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-11T17:51:21.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.271+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi038=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b1c02d3c0 con 0x7f5b040390f0 2024-01-11T17:51:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.277+0000 7f5b197fa700 1 -- 172.21.15.38:0/1096261302 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5b1c02d3c0 con 0x7f5b040390f0 2024-01-11T17:51:21.278 INFO:teuthology.orchestra.run.smithi038.stdout:Scheduled mgr update... 2024-01-11T17:51:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5b040390f0 msgr2=0x7f5b0403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5b040390f0 0x7f5b0403b5b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c068fa0 tx=0x7f5b0c00bef0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 msgr2=0x7f5b1c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 0x7f5b1c100ef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009030 tx=0x7f5b1400bb60 comp rx=0 tx=0).stop 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 shutdown_connections 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5b040390f0 0x7f5b0403b5b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102c60 0x7f5b1c100ef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b1c0ff6e0 0x7f5b1c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 --2- 172.21.15.38:0/1096261302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c100080 0x7f5b1c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 >> 172.21.15.38:0/1096261302 conn(0x7f5b1c0fb280 msgr2=0x7f5b1c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 shutdown_connections 2024-01-11T17:51:21.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:51:21.280+0000 7f5b229aa700 1 -- 172.21.15.38:0/1096261302 wait complete. 2024-01-11T17:51:21.578 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: Reconfiguring mon.c (monmap changed)... 2024-01-11T17:51:21.578 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: Reconfiguring daemon mon.c on smithi038 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.579 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.709 DEBUG:teuthology.orchestra.run.smithi016:mgr.y> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.y.service 2024-01-11T17:51:21.712 DEBUG:teuthology.orchestra.run.smithi038:mgr.z> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.z.service 2024-01-11T17:51:21.714 INFO:tasks.cephadm:Deploying OSDs... 2024-01-11T17:51:21.715 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:51:21.715 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-01-11T17:51:21.748 INFO:journalctl@ceph.mgr.z.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:51:21.748 INFO:journalctl@ceph.mgr.y.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:51:21.764 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-11T17:51:21.764 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 17:50:33.191513536 +0000 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 17:44:22.249506356 +0000 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 17:44:22.249506356 +0000 2024-01-11T17:51:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-01-11 17:44:22.226506688 +0000 2024-01-11T17:51:21.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: Reconfiguring mon.c (monmap changed)... 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: Reconfiguring daemon mon.c on smithi038 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:21.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T17:51:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T17:51:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000188801 s, 2.7 MB/s 2024-01-11T17:51:21.882 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-11T17:51:21.921 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:21.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.927 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: Reconfiguring mon.c (monmap changed)... 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: Reconfiguring daemon mon.c on smithi038 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-11T17:51:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 17:50:33.260512530 +0000 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 17:44:22.742499243 +0000 2024-01-11T17:51:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 17:44:22.742499243 +0000 2024-01-11T17:51:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-01-11 17:44:22.719499575 +0000 2024-01-11T17:51:22.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-11T17:51:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T17:51:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T17:51:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000190564 s, 2.7 MB/s 2024-01-11T17:51:22.120 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-11T17:51:22.165 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 17:50:33.320511656 +0000 2024-01-11T17:51:22.250 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 17:44:23.241492043 +0000 2024-01-11T17:51:22.251 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 17:44:23.241492043 +0000 2024-01-11T17:51:22.251 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-01-11 17:44:23.221492332 +0000 2024-01-11T17:51:22.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-11T17:51:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T17:51:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T17:51:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000191893 s, 2.7 MB/s 2024-01-11T17:51:22.360 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-11T17:51:22.405 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 17:50:33.380510782 +0000 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 17:44:23.723485088 +0000 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 17:44:23.723485088 +0000 2024-01-11T17:51:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-01-11 17:44:23.701485406 +0000 2024-01-11T17:51:22.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-11T17:51:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T17:51:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T17:51:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000228628 s, 2.2 MB/s 2024-01-11T17:51:22.599 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-11T17:51:22.645 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-01-11T17:51:22.645 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-01-11T17:51:22.667 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-11T17:51:22.667 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 171211 Links: 1 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-01-11 17:51:09.197501504 +0000 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-01-11 17:43:41.687481429 +0000 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-01-11 17:43:41.687481429 +0000 2024-01-11T17:51:22.730 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-01-11T17:51:22.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-11T17:51:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:22 smithi038 ceph-mon[43582]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi038=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:22 smithi038 ceph-mon[43582]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi038=z;count:3 2024-01-11T17:51:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:22 smithi038 ceph-mon[43582]: Deploying daemon mgr.z on smithi038 2024-01-11T17:51:22.776 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:22 smithi038 systemd[1]: Starting Ceph mgr.z for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:51:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-01-11T17:51:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-01-11T17:51:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000144784 s, 3.5 MB/s 2024-01-11T17:51:22.804 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-11T17:51:22.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:22 smithi007 ceph-mon[33361]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi038=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:22.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:22 smithi007 ceph-mon[33361]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi038=z;count:3 2024-01-11T17:51:22.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:22 smithi007 ceph-mon[33361]: Deploying daemon mgr.z on smithi038 2024-01-11T17:51:22.867 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-01-11T17:51:22.929 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-11T17:51:22.929 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:22.929 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 172246 Links: 1 2024-01-11T17:51:22.929 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:22.929 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:22.930 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-01-11 17:51:09.197501504 +0000 2024-01-11T17:51:22.930 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-01-11 17:43:42.084475340 +0000 2024-01-11T17:51:22.930 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-01-11 17:43:42.084475340 +0000 2024-01-11T17:51:22.930 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-01-11T17:51:22.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-11T17:51:22.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:22 smithi016 ceph-mon[43195]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi038=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:22 smithi016 ceph-mon[43195]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi038=z;count:3 2024-01-11T17:51:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:22 smithi016 ceph-mon[43195]: Deploying daemon mgr.z on smithi038 2024-01-11T17:51:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-01-11T17:51:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-01-11T17:51:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000142184 s, 3.6 MB/s 2024-01-11T17:51:23.004 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-11T17:51:23.067 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 167323 Links: 1 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-01-11 17:51:09.197501504 +0000 2024-01-11T17:51:23.129 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-01-11 17:43:42.466469482 +0000 2024-01-11T17:51:23.130 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-01-11 17:43:42.466469482 +0000 2024-01-11T17:51:23.130 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-01-11T17:51:23.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-11T17:51:23.203 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-01-11T17:51:23.203 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-01-11T17:51:23.204 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000117742 s, 4.3 MB/s 2024-01-11T17:51:23.205 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-11T17:51:23.268 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-01-11T17:51:23.328 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 172439 Links: 1 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-01-11 17:51:09.197501504 +0000 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-01-11 17:43:42.834463839 +0000 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-01-11 17:43:42.834463839 +0000 2024-01-11T17:51:23.329 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-01-11T17:51:23.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-11T17:51:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-01-11T17:51:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-01-11T17:51:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000117299 s, 4.4 MB/s 2024-01-11T17:51:23.399 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-11T17:51:23.460 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-01-11T17:51:23.460 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-01-11T17:51:23.480 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-11T17:51:23.480 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-01-11T17:51:23.514 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mgr[45460]: -- 172.21.15.38:0/3948205696 <== mon.1 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f32a17520 con 0x564f329ec800 2024-01-11T17:51:23.514 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 bash[45422]: 673ca8eaafccc8245531a4f8c08c51a71548ea0c58ede7285473c1f676517a5a 2024-01-11T17:51:23.514 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 systemd[1]: Started Ceph mgr.z for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 169364 Links: 1 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-01-11 17:51:22.575184968 +0000 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-01-11 17:43:46.823344609 +0000 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-01-11 17:43:46.823344609 +0000 2024-01-11T17:51:23.534 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-01-11T17:51:23.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-11T17:51:23.579 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:23.513+0000 7f3a41497200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T17:51:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-01-11T17:51:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-01-11T17:51:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000320983 s, 1.6 MB/s 2024-01-11T17:51:23.610 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-11T17:51:23.670 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:23.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 173090 Links: 1 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-01-11 17:51:22.575184968 +0000 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-01-11 17:43:47.217337099 +0000 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-01-11 17:43:47.217337099 +0000 2024-01-11T17:51:23.730 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-01-11T17:51:23.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-11T17:51:23.775 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:23.623+0000 7f3a41497200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:23.721+0000 7f3a41497200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:23.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:23.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-01-11T17:51:23.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-01-11T17:51:23.799 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000298144 s, 1.7 MB/s 2024-01-11T17:51:23.800 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-11T17:51:23.861 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-01-11T17:51:23.920 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-11T17:51:23.920 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:23.920 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 171443 Links: 1 2024-01-11T17:51:23.920 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:23.920 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:23.921 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-01-11 17:51:22.575184968 +0000 2024-01-11T17:51:23.921 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-01-11 17:43:47.602329760 +0000 2024-01-11T17:51:23.921 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-01-11 17:43:47.602329760 +0000 2024-01-11T17:51:23.921 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-01-11T17:51:23.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-11T17:51:23.986 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:23.819+0000 7f3a41497200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T17:51:24.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-01-11T17:51:24.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-01-11T17:51:24.042 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000306799 s, 1.7 MB/s 2024-01-11T17:51:24.042 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:24.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:24.102 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 170504 Links: 1 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-01-11 17:51:22.575184968 +0000 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-01-11 17:43:47.966322822 +0000 2024-01-11T17:51:24.162 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-01-11 17:43:47.966322822 +0000 2024-01-11T17:51:24.163 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-01-11T17:51:24.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-11T17:51:24.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-01-11T17:51:24.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-01-11T17:51:24.231 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000370025 s, 1.4 MB/s 2024-01-11T17:51:24.233 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-11T17:51:24.305 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-01-11T17:51:24.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-11T17:51:24.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:24 smithi016 ceph-mon[43195]: Deploying daemon mgr.y on smithi016 2024-01-11T17:51:24.721 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:51:24.808 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:24 smithi038 ceph-mon[43582]: Deploying daemon mgr.y on smithi016 2024-01-11T17:51:24.808 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:24.540+0000 7f3a41497200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:51:24.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:24 smithi007 ceph-mon[33361]: Deploying daemon mgr.y on smithi016 2024-01-11T17:51:24.991 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:24 smithi016 systemd[1]: Starting Ceph mgr.y for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:51:25.204 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:24.808+0000 7f3a41497200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T17:51:25.204 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:24.903+0000 7f3a41497200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T17:51:25.204 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:25.024+0000 7f3a41497200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T17:51:25.204 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:25.203+0000 7f3a41497200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T17:51:25.496 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mgr[44692]: -- 172.21.15.16:0/22500511 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b292929520 con 0x55b2928fec00 2024-01-11T17:51:25.496 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 bash[44654]: 35ddcb941cb8a4d04efc6c1c1ed294c81542f27eb75965be6c7f75c2e02af72d 2024-01-11T17:51:25.496 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 systemd[1]: Started Ceph mgr.y for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:51:25.526 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:25.350+0000 7f3a41497200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T17:51:25.526 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:25.446+0000 7f3a41497200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T17:51:25.705 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:25.597+0000 7f4e5cc5d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T17:51:25.706 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mon[43195]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:25.706 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.706 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.706 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.706 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.706 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:25.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-mon[43582]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:25.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:25.881 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:25.919 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:25.704+0000 7f4e5cc5d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T17:51:25.920 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:25.805+0000 7f4e5cc5d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T17:51:25.920 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:25.919+0000 7f4e5cc5d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T17:51:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:25 smithi007 ceph-mon[33361]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:26.211 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:25.879+0000 7f3a41497200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T17:51:26.526 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:26.209+0000 7f3a41497200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T17:51:26.548 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-11T17:51:26.549 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-11T17:51:26.549 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-01-11T17:51:26.549 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-01-11T17:51:26.549 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0480923 s, 218 MB/s 2024-01-11T17:51:26.549 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 2024-01-11T17:51:26.549 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-01-11T17:51:26.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-01-11T17:51:26.951 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:26.678+0000 7f4e5cc5d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:51:27.026 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:26.546+0000 7f3a41497200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T17:51:27.026 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:26.651+0000 7f3a41497200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T17:51:27.245 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:26.951+0000 7f4e5cc5d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T17:51:27.246 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:27.046+0000 7f4e5cc5d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T17:51:27.246 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:27.168+0000 7f4e5cc5d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T17:51:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:51:27.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:27.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:27.745 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:27.348+0000 7f4e5cc5d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:27.493+0000 7f4e5cc5d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:27.590+0000 7f4e5cc5d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:27.513+0000 7f3a41497200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:27.619+0000 7f3a41497200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-11T17:51:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-11T17:51:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:28.276 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:27.787+0000 7f3a41497200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T17:51:28.315 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:28.015+0000 7f4e5cc5d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T17:51:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:28 smithi007 ceph-mon[33361]: Reconfiguring mgr.x (unknown last config time)... 2024-01-11T17:51:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:28 smithi007 ceph-mon[33361]: Reconfiguring daemon mgr.x on smithi007 2024-01-11T17:51:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:28.639 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:28.315+0000 7f4e5cc5d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T17:51:28.640 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-mon[43195]: Reconfiguring mgr.x (unknown last config time)... 2024-01-11T17:51:28.640 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-mon[43195]: Reconfiguring daemon mgr.x on smithi007 2024-01-11T17:51:28.640 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:28.640 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:28.640 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:28.714 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:28.509+0000 7f3a41497200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T17:51:28.714 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-mon[43582]: Reconfiguring mgr.x (unknown last config time)... 2024-01-11T17:51:28.714 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-mon[43582]: Reconfiguring daemon mgr.x on smithi007 2024-01-11T17:51:28.714 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:28.714 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:28.714 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:28.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 -- 172.21.15.7:0/3433104159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea901020d0 msgr2=0x7fea901044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:28.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 --2- 172.21.15.7:0/3433104159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea901020d0 0x7fea901044c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009990 tx=0x7fea8c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:51:28.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 -- 172.21.15.7:0/3433104159 shutdown_connections 2024-01-11T17:51:28.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 --2- 172.21.15.7:0/3433104159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea90107330 0x7fea901097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:28.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 --2- 172.21.15.7:0/3433104159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea901020d0 0x7fea901044c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:28.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 --2- 172.21.15.7:0/3433104159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90104a00 0x7fea90106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:28.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 -- 172.21.15.7:0/3433104159 >> 172.21.15.7:0/3433104159 conn(0x7fea900fbae0 msgr2=0x7fea900fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 -- 172.21.15.7:0/3433104159 shutdown_connections 2024-01-11T17:51:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.990+0000 7fea97aec700 1 -- 172.21.15.7:0/3433104159 wait complete. 2024-01-11T17:51:28.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 Processor -- start 2024-01-11T17:51:28.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 -- start start 2024-01-11T17:51:28.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 0x7fea90101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:28.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90104a00 0x7fea90100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:28.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90107330 0x7fea901005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90100d40 con 0x7fea90104a00 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90100eb0 con 0x7fea90107330 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea97aec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901a15c0 con 0x7fea901020d0 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea95888700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 0x7fea90101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea95888700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 0x7fea90101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:60922/0 (socket says 172.21.15.7:60922) 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.991+0000 7fea95888700 1 -- 172.21.15.7:0/1983174565 learned_addr learned my addr 172.21.15.7:0/1983174565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95087700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90104a00 0x7fea90100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea96089700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90107330 0x7fea901005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95888700 1 -- 172.21.15.7:0/1983174565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90107330 msgr2=0x7fea901005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95888700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90107330 0x7fea901005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95888700 1 -- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90104a00 msgr2=0x7fea90100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95888700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90104a00 0x7fea90100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95888700 1 -- 172.21.15.7:0/1983174565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c0095f0 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea95888700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 0x7fea90101a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea8c000c00 tx=0x7fea8c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c01d070 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea8c021410 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea901a1840 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.992+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c017650 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.993+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea901a1cb0 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.993+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 45934+0+0 (secure 0 0 0) 0x7fea8c00fb40 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.993+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea900fd660 con 0x7fea901020d0 2024-01-11T17:51:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.994+0000 7fea86ffd700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fea7c0390f0 0x7fea7c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.994+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea8c04ce70 con 0x7fea901020d0 2024-01-11T17:51:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.994+0000 7fea95087700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fea7c0390f0 0x7fea7c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.995+0000 7fea95087700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fea7c0390f0 0x7fea7c03b5b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea80009c60 tx=0x7fea80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:28.999 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:28.639+0000 7f4e5cc5d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T17:51:29.000 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:28.744+0000 7f4e5cc5d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T17:51:29.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:28.999+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fea8c00fdf0 con 0x7fea901020d0 2024-01-11T17:51:29.026 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:28.713+0000 7f3a41497200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T17:51:29.026 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:28.930+0000 7f3a41497200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T17:51:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:29.219+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fea9002d460 con 0x7fea7c0390f0 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:29.623+0000 7f4e5cc5d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:51:29.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:51:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:51:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:29.996 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:29.729+0000 7f4e5cc5d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T17:51:29.996 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:29.898+0000 7f4e5cc5d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T17:51:30.276 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:29 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:29.974+0000 7f3a41497200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T17:51:30.584 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:30.278+0000 7f3a41497200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:51:30.584 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:30.377+0000 7f3a41497200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T17:51:30.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:30 smithi016 ceph-mon[43195]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:30.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:30 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:30.996 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:30.619+0000 7f4e5cc5d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T17:51:30.996 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:30.822+0000 7f4e5cc5d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T17:51:31.026 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 17:51:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z[45437]: 2024-01-11T17:51:30.594+0000 7f3a41497200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T17:51:31.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:30 smithi038 ceph-mon[43582]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:31.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:30 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:31.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:30 smithi007 ceph-mon[33361]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:31.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:30 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:31.495 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:31.039+0000 7f4e5cc5d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T17:51:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:31.591+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 14) v1 ==== 91251+0+0 (secure 0 0 0) 0x7fea8c02b440 con 0x7fea901020d0 2024-01-11T17:51:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-mon[43195]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-mon[43195]: Standby manager daemon z started 2024-01-11T17:51:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-11T17:51:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-11T17:51:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-11T17:51:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:31 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-11T17:51:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:31 smithi038 ceph-mon[43582]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:31 smithi038 ceph-mon[43582]: Standby manager daemon z started 2024-01-11T17:51:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:31 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-11T17:51:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:31 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-11T17:51:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:31 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-11T17:51:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:31 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-11T17:51:32.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:31 smithi007 ceph-mon[33361]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:32.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:31 smithi007 ceph-mon[33361]: Standby manager daemon z started 2024-01-11T17:51:32.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:31 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-11T17:51:32.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:31 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-11T17:51:32.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:31 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-11T17:51:32.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:31 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.38:0/3665082148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-11T17:51:32.382 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:32.079+0000 7f4e5cc5d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:32.381+0000 7f4e5cc5d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:32.481+0000 7f4e5cc5d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: mgrmap e14: x(active, since 67s), standbys: z 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/816184167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]: dispatch 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]: dispatch 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]': finished 2024-01-11T17:51:32.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: osdmap e5: 1 total, 0 up, 1 in 2024-01-11T17:51:32.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:32.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/741485758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: mgrmap e14: x(active, since 67s), standbys: z 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/816184167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]: dispatch 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]: dispatch 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]': finished 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: osdmap e5: 1 total, 0 up, 1 in 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/741485758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: mgrmap e14: x(active, since 67s), standbys: z 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/816184167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]: dispatch 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]: dispatch 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e34d29ca-d1a5-4771-a752-5acca1c7d041"}]': finished 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: osdmap e5: 1 total, 0 up, 1 in 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:33.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/741485758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:51:33.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:33.604+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mon.1 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 15) v1 ==== 136568+0+0 (secure 0 0 0) 0x7fea8c016ca0 con 0x7fea901020d0 2024-01-11T17:51:33.996 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 17:51:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y[44669]: 2024-01-11T17:51:32.695+0000 7f4e5cc5d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T17:51:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:33 smithi016 ceph-mon[43195]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:33 smithi016 ceph-mon[43195]: Standby manager daemon y started 2024-01-11T17:51:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:33 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-11T17:51:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:33 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-11T17:51:33.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:33 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-11T17:51:33.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:33 smithi016 ceph-mon[43195]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-11T17:51:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:33 smithi038 ceph-mon[43582]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:33 smithi038 ceph-mon[43582]: Standby manager daemon y started 2024-01-11T17:51:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:33 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-11T17:51:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:33 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-11T17:51:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:33 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-11T17:51:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:33 smithi038 ceph-mon[43582]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-11T17:51:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:33 smithi007 ceph-mon[33361]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:33 smithi007 ceph-mon[33361]: Standby manager daemon y started 2024-01-11T17:51:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:33 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-11T17:51:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:33 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-11T17:51:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:33 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-11T17:51:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:33 smithi007 ceph-mon[33361]: from='mgr.? 172.21.15.16:0/87700452' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-11T17:51:34.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:34 smithi016 ceph-mon[43195]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-01-11T17:51:34.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:34 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-11T17:51:35.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:34 smithi038 ceph-mon[43582]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-01-11T17:51:35.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:34 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-11T17:51:35.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:34 smithi007 ceph-mon[33361]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-01-11T17:51:35.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:34 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-11T17:51:35.905 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:35 smithi007 ceph-mon[33361]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:35.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:35 smithi016 ceph-mon[43195]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:36.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:35 smithi038 ceph-mon[43582]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:37.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:37 smithi007 ceph-mon[33361]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:37 smithi016 ceph-mon[43195]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:37 smithi038 ceph-mon[43582]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:38.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-11T17:51:38.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:38.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-11T17:51:38.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-11T17:51:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:39 smithi007 ceph-mon[33361]: Deploying daemon osd.0 on smithi007 2024-01-11T17:51:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:39 smithi007 ceph-mon[33361]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:39 smithi016 ceph-mon[43195]: Deploying daemon osd.0 on smithi007 2024-01-11T17:51:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:39 smithi016 ceph-mon[43195]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:39 smithi038 ceph-mon[43582]: Deploying daemon osd.0 on smithi007 2024-01-11T17:51:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:39 smithi038 ceph-mon[43582]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:41 smithi016 ceph-mon[43195]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:42.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:41 smithi038 ceph-mon[43582]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:41 smithi007 ceph-mon[33361]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:43.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:43 smithi007 ceph-mon[33361]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:43.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:43.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:43.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:43 smithi016 ceph-mon[43195]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:43 smithi038 ceph-mon[43582]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:45 smithi016 ceph-mon[43195]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:45.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:45.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:45.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:45 smithi038 ceph-mon[43582]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:45.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:45.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:45.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:45 smithi007 ceph-mon[33361]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:45.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:45.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:45.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:45.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.626+0000 7fea86ffd700 1 -- 172.21.15.7:0/1983174565 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea9002d460 con 0x7fea7c0390f0 2024-01-11T17:51:46.627 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-01-11T17:51:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fea7c0390f0 msgr2=0x7fea7c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fea7c0390f0 0x7fea7c03b5b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea80009c60 tx=0x7fea80009260 comp rx=0 tx=0).stop 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 msgr2=0x7fea90101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 0x7fea90101a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea8c000c00 tx=0x7fea8c005720 comp rx=0 tx=0).stop 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 shutdown_connections 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fea7c0390f0 0x7fea7c03b5b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea901020d0 0x7fea90101a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90107330 0x7fea901005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 --2- 172.21.15.7:0/1983174565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90104a00 0x7fea90100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 >> 172.21.15.7:0/1983174565 conn(0x7fea900fbae0 msgr2=0x7fea90102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.629+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 shutdown_connections 2024-01-11T17:51:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:51:46.630+0000 7fea97aec700 1 -- 172.21.15.7:0/1983174565 wait complete. 2024-01-11T17:51:46.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:46.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:46.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:46.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:46.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:46.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:47.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:51:47.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:47.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:47.049 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.0.service 2024-01-11T17:51:47.053 INFO:tasks.cephadm:Deploying osd.1 on smithi016 with /dev/vg_nvme/lv_4... 2024-01-11T17:51:47.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-11T17:51:47.134 INFO:journalctl@ceph.osd.0.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:51:47.343 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:51:47.432 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:51:47 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0[48084]: 2024-01-11T17:51:47.289+0000 7f7d9492f640 -1 osd.0 0 log_to_monitors true 2024-01-11T17:51:47.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:47 smithi007 ceph-mon[33361]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:47.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:47 smithi007 ceph-mon[33361]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-11T17:51:47.822 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:47 smithi016 ceph-mon[43195]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:47.822 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:47 smithi016 ceph-mon[43195]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-11T17:51:48.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:47 smithi038 ceph-mon[43582]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:48.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:47 smithi038 ceph-mon[43582]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-11T17:51:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-11T17:51:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-11T17:51:48.736 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-01-11T17:51:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-01-11T17:51:48.737 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00922904 s, 1.1 GB/s 2024-01-11T17:51:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: osdmap e6: 1 total, 0 up, 1 in 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: osdmap e7: 1 total, 0 up, 1 in 2024-01-11T17:51:48.897 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-11T17:51:49.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: osdmap e6: 1 total, 0 up, 1 in 2024-01-11T17:51:49.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:51:49.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:51:49.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: osdmap e7: 1 total, 0 up, 1 in 2024-01-11T17:51:49.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.071 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0[48084]: 2024-01-11T17:51:48.639+0000 7f7d89f95700 -1 osd.0 0 waiting for initial osdmap 2024-01-11T17:51:49.071 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0[48084]: 2024-01-11T17:51:48.642+0000 7f7d85d8f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:51:49.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-11T17:51:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: osdmap e6: 1 total, 0 up, 1 in 2024-01-11T17:51:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:51:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: from='osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:51:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: osdmap e7: 1 total, 0 up, 1 in 2024-01-11T17:51:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-01-11T17:51:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:51:49.869 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:49 smithi007 ceph-mon[33361]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:49.869 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.869 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:49 smithi016 ceph-mon[43195]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:50.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:49 smithi038 ceph-mon[43582]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:50.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:50.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 -- 172.21.15.16:0/959846302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac101740 msgr2=0x7f94ac101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 --2- 172.21.15.16:0/959846302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac101740 0x7f94ac101ba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f94980099c0 tx=0x7f9498009cd0 comp rx=0 tx=0).stop 2024-01-11T17:51:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 -- 172.21.15.16:0/959846302 shutdown_connections 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 --2- 172.21.15.16:0/959846302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac1020e0 0x7f94ac108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 --2- 172.21.15.16:0/959846302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ac100540 0x7f94ac100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 --2- 172.21.15.16:0/959846302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac101740 0x7f94ac101ba0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 -- 172.21.15.16:0/959846302 >> 172.21.15.16:0/959846302 conn(0x7f94ac0fbae0 msgr2=0x7f94ac0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 -- 172.21.15.16:0/959846302 shutdown_connections 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.566+0000 7f94b303d700 1 -- 172.21.15.16:0/959846302 wait complete. 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.567+0000 7f94b303d700 1 Processor -- start 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.567+0000 7f94b303d700 1 -- start start 2024-01-11T17:51:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.567+0000 7f94b303d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ac100540 0x7f94ac104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.567+0000 7f94b303d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac101740 0x7f94ac102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.567+0000 7f94b303d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 0x7f94ac103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b303d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac103800 con 0x7f94ac100540 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b303d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac103970 con 0x7f94ac1020e0 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b303d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1a1690 con 0x7f94ac101740 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 0x7f94ac103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 0x7f94ac103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34628/0 (socket says 172.21.15.16:34628) 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 -- 172.21.15.16:0/3799350679 learned_addr learned my addr 172.21.15.16:0/3799350679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94abfff700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac101740 0x7f94ac102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b0dd9700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ac100540 0x7f94ac104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 -- 172.21.15.16:0/3799350679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac101740 msgr2=0x7f94ac102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac101740 0x7f94ac102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 -- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ac100540 msgr2=0x7f94ac104480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ac100540 0x7f94ac104480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:51:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 -- 172.21.15.16:0/3799350679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980095f0 con 0x7f94ac1020e0 2024-01-11T17:51:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b15da700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 0x7f94ac103060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007ee0 tx=0x7f94a400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4017070 con 0x7f94ac1020e0 2024-01-11T17:51:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94a400f040 con 0x7f94ac1020e0 2024-01-11T17:51:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac1a1920 con 0x7f94ac1020e0 2024-01-11T17:51:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.568+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4005580 con 0x7f94ac1020e0 2024-01-11T17:51:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.569+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac1a1e20 con 0x7f94ac1020e0 2024-01-11T17:51:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.570+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0682d0 con 0x7f94ac1020e0 2024-01-11T17:51:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.571+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f94a400db20 con 0x7f94ac1020e0 2024-01-11T17:51:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.571+0000 7f94a9ffb700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f949c0a1ed0 0x7f949c0a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:51:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.571+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f94a40cb5b0 con 0x7f94ac1020e0 2024-01-11T17:51:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.572+0000 7f94b0dd9700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f949c0a1ed0 0x7f949c0a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:51:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.572+0000 7f94b0dd9700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f949c0a1ed0 0x7f949c0a4390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94ac1015a0 tx=0x7f94a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:51:50.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.576+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f94a40960c0 con 0x7f94ac1020e0 2024-01-11T17:51:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:51:50.793+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f94ac069a40 con 0x7f949c0a1ed0 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] boot 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: osdmap e8: 1 total, 1 up, 1 in 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:50.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] boot 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: osdmap e8: 1 total, 1 up, 1 in 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:51:51.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:51.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:51.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:51:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:51:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: osd.0 [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] boot 2024-01-11T17:51:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: osdmap e8: 1 total, 1 up, 1 in 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:51:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi007 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi007 to 17276M 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:51 smithi016 ceph-mon[43195]: osdmap e9: 1 total, 1 up, 1 in 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi007 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi007 to 17276M 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:51 smithi038 ceph-mon[43582]: osdmap e9: 1 total, 1 up, 1 in 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi007 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi007 to 17276M 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:51:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:51 smithi007 ceph-mon[33361]: osdmap e9: 1 total, 1 up, 1 in 2024-01-11T17:51:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:52 smithi016 ceph-mon[43195]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:53.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:52 smithi038 ceph-mon[43582]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:53.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:52 smithi007 ceph-mon[33361]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:51:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/4164011783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]: dispatch 2024-01-11T17:51:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]: dispatch 2024-01-11T17:51:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]': finished 2024-01-11T17:51:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: osdmap e10: 2 total, 1 up, 2 in 2024-01-11T17:51:53.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:51:53.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/459868884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/4164011783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]: dispatch 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]: dispatch 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]': finished 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: osdmap e10: 2 total, 1 up, 2 in 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:51:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/459868884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:51:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/4164011783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]: dispatch 2024-01-11T17:51:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]: dispatch 2024-01-11T17:51:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b918edd-6f35-4186-83f6-8dea129fa4d8"}]': finished 2024-01-11T17:51:54.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: osdmap e10: 2 total, 1 up, 2 in 2024-01-11T17:51:54.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:51:54.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/459868884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:51:55.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:55 smithi016 ceph-mon[43195]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:55.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:55 smithi038 ceph-mon[43582]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:55.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:55 smithi007 ceph-mon[33361]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:57.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:57 smithi016 ceph-mon[43195]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:57.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:57 smithi007 ceph-mon[33361]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:58.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:57 smithi038 ceph-mon[43582]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:59.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:59 smithi016 ceph-mon[43195]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:51:59.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-11T17:51:59.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:51:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:59 smithi038 ceph-mon[43582]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-11T17:52:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:51:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:59 smithi007 ceph-mon[33361]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-11T17:52:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:51:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:00.908 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:00 smithi016 ceph-mon[43195]: Deploying daemon osd.1 on smithi016 2024-01-11T17:52:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:00 smithi038 ceph-mon[43582]: Deploying daemon osd.1 on smithi016 2024-01-11T17:52:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:00 smithi007 ceph-mon[33361]: Deploying daemon osd.1 on smithi016 2024-01-11T17:52:01.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:01 smithi016 ceph-mon[43195]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:02.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:01 smithi038 ceph-mon[43582]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:01 smithi007 ceph-mon[33361]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:03.859 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:03 smithi016 ceph-mon[43195]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:03.860 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:03.860 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:03.860 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:03 smithi038 ceph-mon[43582]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:03 smithi007 ceph-mon[33361]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.356+0000 7f94a9ffb700 1 -- 172.21.15.16:0/3799350679 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f94ac069a40 con 0x7f949c0a1ed0 2024-01-11T17:52:05.357 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 1 on host 'smithi016' 2024-01-11T17:52:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f949c0a1ed0 msgr2=0x7f949c0a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f949c0a1ed0 0x7f949c0a4390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94ac1015a0 tx=0x7f94a0009260 comp rx=0 tx=0).stop 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 msgr2=0x7f94ac103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 0x7f94ac103060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007ee0 tx=0x7f94a400d3b0 comp rx=0 tx=0).stop 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 shutdown_connections 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f949c0a1ed0 0x7f949c0a4390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ac1020e0 0x7f94ac103060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ac100540 0x7f94ac104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 --2- 172.21.15.16:0/3799350679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac101740 0x7f94ac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 >> 172.21.15.16:0/3799350679 conn(0x7f94ac0fbae0 msgr2=0x7f94ac069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 shutdown_connections 2024-01-11T17:52:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:05.359+0000 7f94b303d700 1 -- 172.21.15.16:0/3799350679 wait complete. 2024-01-11T17:52:05.561 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.561 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.562 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:05.562 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:05.562 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.562 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:05 smithi016 ceph-mon[43195]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:05 smithi007 ceph-mon[33361]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:05.765 DEBUG:teuthology.orchestra.run.smithi016:osd.1> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.1.service 2024-01-11T17:52:05.767 INFO:tasks.cephadm:Deploying osd.2 on smithi038 with /dev/vg_nvme/lv_4... 2024-01-11T17:52:05.768 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-11T17:52:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:05 smithi038 ceph-mon[43582]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:05.804 INFO:journalctl@ceph.osd.1.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:52:06.063 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:52:06.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:06.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:06.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:06.499 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:06.499 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:06.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:06.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:06.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:06.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:07.246 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:06 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1[48283]: 2024-01-11T17:52:06.866+0000 7f95b9ddc640 -1 osd.1 0 log_to_monitors true 2024-01-11T17:52:07.413 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-11T17:52:07.413 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-11T17:52:07.413 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-01-11T17:52:07.413 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-01-11T17:52:07.413 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112051 s, 936 MB/s 2024-01-11T17:52:07.414 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-01-11T17:52:07.591 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:07 smithi038 ceph-mon[43582]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:07.591 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:07 smithi038 ceph-mon[43582]: from='osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-11T17:52:07.591 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:07 smithi038 ceph-mon[43582]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-11T17:52:07.626 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:07 smithi016 ceph-mon[43195]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:07.626 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:07 smithi016 ceph-mon[43195]: from='osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-11T17:52:07.626 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:07 smithi016 ceph-mon[43195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-11T17:52:07.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-01-11T17:52:07.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:07 smithi007 ceph-mon[33361]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:07.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:07 smithi007 ceph-mon[33361]: from='osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-11T17:52:07.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:07 smithi007 ceph-mon[33361]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-11T17:52:08.143 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: osdmap e11: 2 total, 1 up, 2 in 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:08.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: osdmap e11: 2 total, 1 up, 2 in 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: osdmap e11: 2 total, 1 up, 2 in 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:09.246 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1[48283]: 2024-01-11T17:52:08.866+0000 7f95af442700 -1 osd.1 0 waiting for initial osdmap 2024-01-11T17:52:09.246 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1[48283]: 2024-01-11T17:52:08.869+0000 7f95a7234700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:52:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 -- 172.21.15.38:0/3558081549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 msgr2=0x7fcaa4100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/3558081549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4100960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca94009990 tx=0x7fca94009ca0 comp rx=0 tx=0).stop 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 -- 172.21.15.38:0/3558081549 shutdown_connections 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/3558081549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4100960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/3558081549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa41020e0 0x7fcaa4108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/3558081549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa4101740 0x7fcaa4101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 -- 172.21.15.38:0/3558081549 >> 172.21.15.38:0/3558081549 conn(0x7fcaa40fbae0 msgr2=0x7fcaa40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 -- 172.21.15.38:0/3558081549 shutdown_connections 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.251+0000 7fcaa91bd700 1 -- 172.21.15.38:0/3558081549 wait complete. 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.252+0000 7fcaa91bd700 1 Processor -- start 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.252+0000 7fcaa91bd700 1 -- start start 2024-01-11T17:52:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa91bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa91bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa4101740 0x7fcaa419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa91bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa41020e0 0x7fcaa419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa91bd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa419f0d0 con 0x7fcaa41020e0 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa91bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa419f240 con 0x7fcaa4101740 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa91bd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa419f400 con 0x7fcaa4100540 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41600/0 (socket says 172.21.15.38:41600) 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 -- 172.21.15.38:0/2445259647 learned_addr learned my addr 172.21.15.38:0/2445259647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fca9a59c700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa4101740 0x7fcaa419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa359e700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa41020e0 0x7fcaa419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 -- 172.21.15.38:0/2445259647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa4101740 msgr2=0x7fcaa419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa4101740 0x7fcaa419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 -- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa41020e0 msgr2=0x7fcaa419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:09.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa41020e0 0x7fcaa419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:09.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 -- 172.21.15.38:0/2445259647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940095f0 con 0x7fcaa4100540 2024-01-11T17:52:09.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.253+0000 7fcaa2d9d700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4069e20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca94000c00 tx=0x7fca94003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:52:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.254+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9401d070 con 0x7fcaa4100540 2024-01-11T17:52:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.254+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa419f680 con 0x7fcaa4100540 2024-01-11T17:52:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.254+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca94005580 con 0x7fcaa4100540 2024-01-11T17:52:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.254+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9400fdf0 con 0x7fcaa4100540 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.254+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa419fa90 con 0x7fcaa4100540 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.255+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa4107a40 con 0x7fcaa4100540 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.256+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) v1 ==== 136568+0+0 (secure 0 0 0) 0x7fca9400f4f0 con 0x7fcaa4100540 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.257+0000 7fcaa0d99700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fca900a1e50 0x7fca900a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.257+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7fca940cd040 con 0x7fcaa4100540 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.257+0000 7fca9a59c700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fca900a1e50 0x7fca900a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.258+0000 7fca9a59c700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fca900a1e50 0x7fca900a4310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406a3a0 tx=0x7fca8c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:52:09.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.262+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fca94097ae0 con 0x7fcaa4100540 2024-01-11T17:52:09.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:09.483+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaa402d4c0 con 0x7fca900a1e50 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi016 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi016 to 17118M 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: osdmap e12: 2 total, 1 up, 2 in 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:09.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:09 smithi016 ceph-mon[43195]: from='osd.1 ' entity='osd.1' 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi016 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi016 to 17118M 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: osdmap e12: 2 total, 1 up, 2 in 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:09 smithi038 ceph-mon[43582]: from='osd.1 ' entity='osd.1' 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi016 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi016 to 17118M 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: osdmap e12: 2 total, 1 up, 2 in 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-01-11T17:52:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:09 smithi007 ceph-mon[33361]: from='osd.1 ' entity='osd.1' 2024-01-11T17:52:10.717 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:52:10.717 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] boot 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: osdmap e13: 2 total, 2 up, 2 in 2024-01-11T17:52:10.718 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] boot 2024-01-11T17:52:10.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: osdmap e13: 2 total, 2 up, 2 in 2024-01-11T17:52:10.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: osd.1 [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] boot 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: osdmap e13: 2 total, 2 up, 2 in 2024-01-11T17:52:10.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-11T17:52:11.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:11 smithi016 ceph-mon[43195]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:11.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:11 smithi038 ceph-mon[43582]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:11.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:11 smithi007 ceph-mon[33361]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: osdmap e14: 2 total, 2 up, 2 in 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/1979836197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]: dispatch 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]: dispatch 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]': finished 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: osdmap e15: 3 total, 2 up, 3 in 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:12.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/36304426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:12.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: osdmap e14: 2 total, 2 up, 2 in 2024-01-11T17:52:12.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/1979836197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]: dispatch 2024-01-11T17:52:12.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]: dispatch 2024-01-11T17:52:12.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]': finished 2024-01-11T17:52:12.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: osdmap e15: 3 total, 2 up, 3 in 2024-01-11T17:52:12.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:12.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/36304426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: osdmap e14: 2 total, 2 up, 2 in 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/1979836197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]: dispatch 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]: dispatch 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85ec6460-ec16-4142-abff-60b2b9d83550"}]': finished 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: osdmap e15: 3 total, 2 up, 3 in 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:12.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/36304426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:13.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:13 smithi016 ceph-mon[43195]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:13.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:13 smithi038 ceph-mon[43582]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:13.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:13 smithi007 ceph-mon[33361]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:15.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:15 smithi038 ceph-mon[43582]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:15.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:15 smithi007 ceph-mon[33361]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:15 smithi016 ceph-mon[43195]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:17.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:17 smithi038 ceph-mon[43582]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:17.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:17 smithi016 ceph-mon[43195]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:18.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:17 smithi007 ceph-mon[33361]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:18.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-11T17:52:18.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-11T17:52:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:19.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-11T17:52:19.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:19.977 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:19 smithi038 ceph-mon[43582]: Deploying daemon osd.2 on smithi038 2024-01-11T17:52:19.977 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:19 smithi038 ceph-mon[43582]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:19 smithi016 ceph-mon[43195]: Deploying daemon osd.2 on smithi038 2024-01-11T17:52:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:19 smithi016 ceph-mon[43195]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:19 smithi007 ceph-mon[33361]: Deploying daemon osd.2 on smithi038 2024-01-11T17:52:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:19 smithi007 ceph-mon[33361]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:21.839 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:21 smithi038 ceph-mon[43582]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:21 smithi016 ceph-mon[43195]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:21 smithi007 ceph-mon[33361]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:22.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:23.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:23.942 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 2 on host 'smithi038' 2024-01-11T17:52:23.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.941+0000 7fcaa0d99700 1 -- 172.21.15.38:0/2445259647 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcaa402d4c0 con 0x7fca900a1e50 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fca900a1e50 msgr2=0x7fca900a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fca900a1e50 0x7fca900a4310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406a3a0 tx=0x7fca8c0073d0 comp rx=0 tx=0).stop 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 msgr2=0x7fcaa4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4069e20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca94000c00 tx=0x7fca94003b70 comp rx=0 tx=0).stop 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 shutdown_connections 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fca900a1e50 0x7fca900a4310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa4100540 0x7fcaa4069e20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa41020e0 0x7fcaa419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 --2- 172.21.15.38:0/2445259647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa4101740 0x7fcaa419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:23.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.944+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 >> 172.21.15.38:0/2445259647 conn(0x7fcaa40fbae0 msgr2=0x7fcaa4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:52:23.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.945+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 shutdown_connections 2024-01-11T17:52:23.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:52:23.945+0000 7fcaa91bd700 1 -- 172.21.15.38:0/2445259647 wait complete. 2024-01-11T17:52:23.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:23 smithi038 ceph-mon[43582]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:23.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:23.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:23.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:23 smithi016 ceph-mon[43195]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:23 smithi007 ceph-mon[33361]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:24.337 DEBUG:teuthology.orchestra.run.smithi038:osd.2> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.2.service 2024-01-11T17:52:24.339 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-01-11T17:52:24.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-11T17:52:24.376 INFO:journalctl@ceph.osd.2.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:52:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:52:24.968 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:24.968 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:24.968 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:24.968 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:24 smithi038 ceph-mon[43582]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:24 smithi016 ceph-mon[43195]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:25.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:25.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:25.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:25.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:24 smithi007 ceph-mon[33361]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:25.776 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2[49334]: 2024-01-11T17:52:25.331+0000 7f5413514640 -1 osd.2 0 log_to_monitors true 2024-01-11T17:52:26.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:26 smithi016 ceph-mon[43195]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-11T17:52:26.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:26 smithi038 ceph-mon[43582]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-11T17:52:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:26 smithi007 ceph-mon[33361]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-11T17:52:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-11T17:52:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-11T17:52:26.614 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-01-11T17:52:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-01-11T17:52:26.614 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0423132 s, 248 MB/s 2024-01-11T17:52:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-01-11T17:52:27.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-01-11T17:52:27.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-11T17:52:27.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: osdmap e16: 3 total, 2 up, 3 in 2024-01-11T17:52:27.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:52:27.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:27.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:27.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:27.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:27.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:27.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.464 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:52:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-11T17:52:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: osdmap e16: 3 total, 2 up, 3 in 2024-01-11T17:52:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:52:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: osdmap e16: 3 total, 2 up, 3 in 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:28.026 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2[49334]: 2024-01-11T17:52:27.735+0000 7f5408b7a700 -1 osd.2 0 waiting for initial osdmap 2024-01-11T17:52:28.026 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2[49334]: 2024-01-11T17:52:27.740+0000 7f540096c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:52:28.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi038 2024-01-11T17:52:28.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi038 to 17118M 2024-01-11T17:52:28.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:52:28.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: osdmap e17: 3 total, 2 up, 3 in 2024-01-11T17:52:28.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:28.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:28.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:28 smithi038 ceph-mon[43582]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi038 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi038 to 17118M 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: osdmap e17: 3 total, 2 up, 3 in 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:28 smithi007 ceph-mon[33361]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 -- 172.21.15.7:0/1845774424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20102850 msgr2=0x7eff20102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1845774424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20102850 0x7eff20102cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff140099c0 tx=0x7eff14009cd0 comp rx=0 tx=0).stop 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 -- 172.21.15.7:0/1845774424 shutdown_connections 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1845774424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff201031f0 0x7eff20109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1845774424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20101650 0x7eff20101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1845774424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20102850 0x7eff20102cb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 -- 172.21.15.7:0/1845774424 >> 172.21.15.7:0/1845774424 conn(0x7eff200fcbb0 msgr2=0x7eff200ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 -- 172.21.15.7:0/1845774424 shutdown_connections 2024-01-11T17:52:28.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.604+0000 7eff24f6e700 1 -- 172.21.15.7:0/1845774424 wait complete. 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.605+0000 7eff24f6e700 1 Processor -- start 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 -- start start 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20101650 0x7eff20199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 0x7eff20199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff201031f0 0x7eff201a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff201a12b0 con 0x7eff20101650 2024-01-11T17:52:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff201a13f0 con 0x7eff20102850 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff24f6e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff201a1580 con 0x7eff201031f0 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff1e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20101650 0x7eff20199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff1dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 0x7eff20199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff1dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 0x7eff20199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42698/0 (socket says 172.21.15.7:42698) 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff1dd9b700 1 -- 172.21.15.7:0/1696122634 learned_addr learned my addr 172.21.15.7:0/1696122634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.606+0000 7eff1ed9d700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff201031f0 0x7eff201a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff1dd9b700 1 -- 172.21.15.7:0/1696122634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff201031f0 msgr2=0x7eff201a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff1dd9b700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff201031f0 0x7eff201a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff1dd9b700 1 -- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20101650 msgr2=0x7eff20199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff1dd9b700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20101650 0x7eff20199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff1dd9b700 1 -- 172.21.15.7:0/1696122634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140095f0 con 0x7eff20102850 2024-01-11T17:52:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff1dd9b700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 0x7eff20199c60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff14006010 tx=0x7eff1400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:52:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1401d070 con 0x7eff20102850 2024-01-11T17:52:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff201a1800 con 0x7eff20102850 2024-01-11T17:52:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff14005030 con 0x7eff20102850 2024-01-11T17:52:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14004b10 con 0x7eff20102850 2024-01-11T17:52:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.607+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff201a1c70 con 0x7eff20102850 2024-01-11T17:52:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.609+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff200682d0 con 0x7eff20102850 2024-01-11T17:52:28.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.610+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) v1 ==== 136568+0+0 (secure 0 0 0) 0x7eff140051a0 con 0x7eff20102850 2024-01-11T17:52:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.612+0000 7eff0f7fe700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff080a1e50 0x7eff080a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.612+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7eff140ccee0 con 0x7eff20102850 2024-01-11T17:52:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.612+0000 7eff1e59c700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff080a1e50 0x7eff080a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.613+0000 7eff1e59c700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff080a1e50 0x7eff080a4310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff201026b0 tx=0x7eff10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:52:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.615+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7eff140977e0 con 0x7eff20102850 2024-01-11T17:52:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi038 2024-01-11T17:52:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi038 to 17118M 2024-01-11T17:52:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:52:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: osdmap e17: 3 total, 2 up, 3 in 2024-01-11T17:52:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:28 smithi016 ceph-mon[43195]: from='osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254]' entity='osd.2' 2024-01-11T17:52:28.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:28.835+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7eff20074f60 con 0x7eff080a1e50 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] boot 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: osdmap e18: 3 total, 3 up, 3 in 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:29.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:29.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:29.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] boot 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: osdmap e18: 3 total, 3 up, 3 in 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:29.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: osd.2 [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] boot 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: osdmap e18: 3 total, 3 up, 3 in 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:30.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:30 smithi038 ceph-mon[43582]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:30.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:30 smithi007 ceph-mon[33361]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:30 smithi016 ceph-mon[43195]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:31.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:31 smithi007 ceph-mon[33361]: osdmap e19: 3 total, 3 up, 3 in 2024-01-11T17:52:31.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:31 smithi007 ceph-mon[33361]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:31.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:31 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-11T17:52:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:31 smithi016 ceph-mon[43195]: osdmap e19: 3 total, 3 up, 3 in 2024-01-11T17:52:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:31 smithi016 ceph-mon[43195]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:31 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-11T17:52:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:31 smithi038 ceph-mon[43582]: osdmap e19: 3 total, 3 up, 3 in 2024-01-11T17:52:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:31 smithi038 ceph-mon[43582]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:31 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-11T17:52:32.571 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54110]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-11T17:52:32.572 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:32.572 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54110]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:32.572 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54110]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-11T17:52:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: osdmap e20: 3 total, 3 up, 3 in 2024-01-11T17:52:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-11T17:52:32.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3324343739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9f9282-a8c3-43df-954d-8bc2a5854cf1"}]: dispatch 2024-01-11T17:52:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-11T17:52:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3324343739' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad9f9282-a8c3-43df-954d-8bc2a5854cf1"}]': finished 2024-01-11T17:52:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: osdmap e21: 4 total, 3 up, 4 in 2024-01-11T17:52:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:32.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2162069264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-11T17:52:32.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: osdmap e20: 3 total, 3 up, 3 in 2024-01-11T17:52:32.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-11T17:52:32.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3324343739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9f9282-a8c3-43df-954d-8bc2a5854cf1"}]: dispatch 2024-01-11T17:52:32.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-11T17:52:32.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3324343739' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad9f9282-a8c3-43df-954d-8bc2a5854cf1"}]': finished 2024-01-11T17:52:32.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: osdmap e21: 4 total, 3 up, 4 in 2024-01-11T17:52:32.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:32.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2162069264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-11T17:52:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: osdmap e20: 3 total, 3 up, 3 in 2024-01-11T17:52:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-11T17:52:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3324343739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad9f9282-a8c3-43df-954d-8bc2a5854cf1"}]: dispatch 2024-01-11T17:52:32.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-11T17:52:32.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3324343739' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad9f9282-a8c3-43df-954d-8bc2a5854cf1"}]': finished 2024-01-11T17:52:32.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: osdmap e21: 4 total, 3 up, 4 in 2024-01-11T17:52:32.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:32.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2162069264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:33.071 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54166]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-11T17:52:33.071 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:33.071 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54166]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:33.071 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 17:52:32 smithi007 sudo[54166]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:33.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:33.366+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7eff14097bb0 con 0x7eff20102850 2024-01-11T17:52:33.496 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53164]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-11T17:52:33.496 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:33.496 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53164]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:33.496 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53164]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:33.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:33 smithi016 ceph-mon[43195]: osdmap e22: 4 total, 3 up, 4 in 2024-01-11T17:52:33.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:33.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:33 smithi016 ceph-mon[43195]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:33.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:33 smithi038 ceph-mon[43582]: osdmap e22: 4 total, 3 up, 4 in 2024-01-11T17:52:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:33 smithi038 ceph-mon[43582]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:33 smithi007 ceph-mon[33361]: osdmap e22: 4 total, 3 up, 4 in 2024-01-11T17:52:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:33 smithi007 ceph-mon[33361]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:33.996 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53167]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-11T17:52:33.996 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:33.996 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53167]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:33.996 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 17:52:33 smithi016 sudo[53167]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:34.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:34 smithi038 ceph-mon[43582]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-01-11T17:52:34.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:34 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:34.526 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54744]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-11T17:52:34.526 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54744]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:34.526 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54744]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:34.526 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54744]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:34.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:34 smithi016 ceph-mon[43195]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-01-11T17:52:34.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:34 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:34 smithi007 ceph-mon[33361]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-01-11T17:52:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:34 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:35.026 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54747]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-11T17:52:35.026 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:35.026 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54747]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:35.026 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 17:52:34 smithi038 sudo[54747]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:35.495 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55303]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-11T17:52:35.495 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:35.495 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55303]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:35.495 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55303]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:35.751 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55458]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-11T17:52:35.751 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:35.751 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55458]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 sudo[55458]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:36.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:35 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:36.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:35 smithi038 ceph-mon[43582]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:36.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:36.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:35 smithi007 ceph-mon[33361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:36.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:35 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:36.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:35 smithi016 ceph-mon[43195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:36.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54750]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-11T17:52:36.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:36.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54750]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:36.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54750]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:37.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54753]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-11T17:52:37.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:37.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54753]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:37.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:36 smithi038 sudo[54753]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:37.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53170]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-11T17:52:37.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:37.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53170]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:37.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53170]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:37.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:37.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:37.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:37.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:37.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:37.919 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:37.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53173]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53173]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53173]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 sudo[53173]: pam_unix(sudo:session): session closed for user root 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:37.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-11T17:52:38.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-11T17:52:38.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:38 smithi007 ceph-mon[33361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:38.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:38 smithi007 ceph-mon[33361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:38.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-11T17:52:38.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:38 smithi016 ceph-mon[43195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:38 smithi016 ceph-mon[43195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-11T17:52:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:38 smithi038 ceph-mon[43582]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-11T17:52:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:38 smithi038 ceph-mon[43582]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-11T17:52:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-11T17:52:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:39 smithi016 ceph-mon[43195]: Deploying daemon osd.3 on smithi007 2024-01-11T17:52:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:39 smithi016 ceph-mon[43195]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:39 smithi038 ceph-mon[43582]: Deploying daemon osd.3 on smithi007 2024-01-11T17:52:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:39 smithi038 ceph-mon[43582]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:40.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:39 smithi007 ceph-mon[33361]: Deploying daemon osd.3 on smithi007 2024-01-11T17:52:40.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:39 smithi007 ceph-mon[33361]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-11T17:52:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:41 smithi016 ceph-mon[43195]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-01-11T17:52:42.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:41 smithi038 ceph-mon[43582]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-01-11T17:52:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:41 smithi007 ceph-mon[33361]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-01-11T17:52:43.992 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:43 smithi007 ceph-mon[33361]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-01-11T17:52:43.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:43.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:43.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:43 smithi016 ceph-mon[43195]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-01-11T17:52:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:43 smithi038 ceph-mon[43582]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-01-11T17:52:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:45.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:45 smithi038 ceph-mon[43582]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-01-11T17:52:45.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:45 smithi007 ceph-mon[33361]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-01-11T17:52:45.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:45 smithi016 ceph-mon[43195]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-01-11T17:52:46.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.234+0000 7eff0f7fe700 1 -- 172.21.15.7:0/1696122634 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff20074f60 con 0x7eff080a1e50 2024-01-11T17:52:46.236 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.236+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff080a1e50 msgr2=0x7eff080a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.236+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff080a1e50 0x7eff080a4310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff201026b0 tx=0x7eff10009260 comp rx=0 tx=0).stop 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.236+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 msgr2=0x7eff20199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.236+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 0x7eff20199c60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7eff14006010 tx=0x7eff1400f690 comp rx=0 tx=0).stop 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 shutdown_connections 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff080a1e50 0x7eff080a4310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff201031f0 0x7eff201a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:46.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff20101650 0x7eff20199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:46.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 --2- 172.21.15.7:0/1696122634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff20102850 0x7eff20199c60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:46.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 >> 172.21.15.7:0/1696122634 conn(0x7eff200fcbb0 msgr2=0x7eff20073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:52:46.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 shutdown_connections 2024-01-11T17:52:46.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:52:46.237+0000 7eff24f6e700 1 -- 172.21.15.7:0/1696122634 wait complete. 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.630 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.3.service 2024-01-11T17:52:46.636 INFO:tasks.cephadm:Deploying osd.4 on smithi016 with /dev/vg_nvme/lv_3... 2024-01-11T17:52:46.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.672 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.716 INFO:journalctl@ceph.osd.3.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:52:47.644 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:47.644 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:47 smithi016 ceph-mon[43195]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-01-11T17:52:47.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:47.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:47 smithi038 ceph-mon[43582]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-01-11T17:52:47.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:52:47.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:47 smithi007 ceph-mon[33361]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-01-11T17:52:47.797 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 17:52:47 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T17:52:47.338+0000 7fb769482640 -1 osd.3 0 log_to_monitors true 2024-01-11T17:52:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-11T17:52:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-11T17:52:48.391 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-01-11T17:52:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-01-11T17:52:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00921279 s, 1.1 GB/s 2024-01-11T17:52:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-01-11T17:52:48.622 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:48 smithi016 ceph-mon[43195]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-11T17:52:48.622 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:48 smithi016 ceph-mon[43195]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-11T17:52:48.623 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:48 smithi016 ceph-mon[43195]: osdmap e23: 4 total, 3 up, 4 in 2024-01-11T17:52:48.623 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:48 smithi016 ceph-mon[43195]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:52:48.623 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:48 smithi038 ceph-mon[43582]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-11T17:52:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:48 smithi038 ceph-mon[43582]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-11T17:52:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:48 smithi038 ceph-mon[43582]: osdmap e23: 4 total, 3 up, 4 in 2024-01-11T17:52:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:48 smithi038 ceph-mon[43582]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:52:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:48.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:48 smithi007 ceph-mon[33361]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-11T17:52:48.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:48 smithi007 ceph-mon[33361]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-11T17:52:48.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:48 smithi007 ceph-mon[33361]: osdmap e23: 4 total, 3 up, 4 in 2024-01-11T17:52:48.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:48 smithi007 ceph-mon[33361]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:52:48.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:48.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-01-11T17:52:49.165 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:52:49.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:49 smithi007 ceph-mon[33361]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-01-11T17:52:49.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:49 smithi007 ceph-mon[33361]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:52:49.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:49 smithi007 ceph-mon[33361]: osdmap e24: 4 total, 3 up, 4 in 2024-01-11T17:52:49.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:49.663 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:49 smithi016 ceph-mon[43195]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-01-11T17:52:49.664 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:49 smithi016 ceph-mon[43195]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:52:49.664 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:49 smithi016 ceph-mon[43195]: osdmap e24: 4 total, 3 up, 4 in 2024-01-11T17:52:49.664 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:49 smithi038 ceph-mon[43582]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-01-11T17:52:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:49 smithi038 ceph-mon[43582]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:52:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:49 smithi038 ceph-mon[43582]: osdmap e24: 4 total, 3 up, 4 in 2024-01-11T17:52:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 -- 172.21.15.16:0/887021643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 msgr2=0x7fa248101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 --2- 172.21.15.16:0/887021643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248101a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2380099c0 tx=0x7fa238009cd0 comp rx=0 tx=0).stop 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 -- 172.21.15.16:0/887021643 shutdown_connections 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 --2- 172.21.15.16:0/887021643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248103200 0x7fa248109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 --2- 172.21.15.16:0/887021643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248101a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 --2- 172.21.15.16:0/887021643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102860 0x7fa248102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 -- 172.21.15.16:0/887021643 >> 172.21.15.16:0/887021643 conn(0x7fa2480fcc20 msgr2=0x7fa2480ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 -- 172.21.15.16:0/887021643 shutdown_connections 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.292+0000 7fa24e03d700 1 -- 172.21.15.16:0/887021643 wait complete. 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 Processor -- start 2024-01-11T17:52:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 -- start start 2024-01-11T17:52:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248102860 0x7fa248197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248103200 0x7fa24819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:50.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24819f0b0 con 0x7fa248103200 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24819f1f0 con 0x7fa248101660 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.293+0000 7fa24e03d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24819f380 con 0x7fa248102860 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47990/0 (socket says 172.21.15.16:47990) 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 -- 172.21.15.16:0/3907026108 learned_addr learned my addr 172.21.15.16:0/3907026108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa246ffd700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248102860 0x7fa248197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa247fff700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248103200 0x7fa24819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 -- 172.21.15.16:0/3907026108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248102860 msgr2=0x7fa248197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248102860 0x7fa248197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 -- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248103200 msgr2=0x7fa24819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248103200 0x7fa24819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 -- 172.21.15.16:0/3907026108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380095f0 con 0x7fa248101660 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa247fff700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248103200 0x7fa24819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:52:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa2477fe700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248197520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa238003730 tx=0x7fa23800f720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:52:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23801d070 con 0x7fa248101660 2024-01-11T17:52:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24819f600 con 0x7fa248101660 2024-01-11T17:52:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2380059a0 con 0x7fa248101660 2024-01-11T17:52:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238004b30 con 0x7fa248101660 2024-01-11T17:52:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.294+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24819faa0 con 0x7fa248101660 2024-01-11T17:52:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.295+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa248108af0 con 0x7fa248101660 2024-01-11T17:52:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.297+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa23800fc10 con 0x7fa248101660 2024-01-11T17:52:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.297+0000 7fa244ff9700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2300a1ee0 0x7fa2300a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:52:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.297+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fa2380cd760 con 0x7fa248101660 2024-01-11T17:52:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.298+0000 7fa246ffd700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2300a1ee0 0x7fa2300a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:52:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.298+0000 7fa246ffd700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2300a1ee0 0x7fa2300a43a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009bf0 tx=0x7fa23c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:52:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.301+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa238097bd0 con 0x7fa248101660 2024-01-11T17:52:50.321 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 17:52:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T17:52:49.995+0000 7fb75eae8700 -1 osd.3 0 waiting for initial osdmap 2024-01-11T17:52:50.321 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T17:52:50.014+0000 7fb7568da700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:52:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:52:50.521+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa24802d500 con 0x7fa2300a1ee0 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:50.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' 2024-01-11T17:52:50.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] boot 2024-01-11T17:52:50.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: osdmap e25: 4 total, 4 up, 4 in 2024-01-11T17:52:50.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] boot 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: osdmap e25: 4 total, 4 up, 4 in 2024-01-11T17:52:50.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:52:50.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736]' entity='osd.3' 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: osd.3 [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] boot 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: osdmap e25: 4 total, 4 up, 4 in 2024-01-11T17:52:50.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi007 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi007 to 8638M 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:51.632 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:51 smithi016 ceph-mon[43195]: osdmap e26: 4 total, 4 up, 4 in 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi007 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi007 to 8638M 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:51.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:51 smithi038 ceph-mon[43582]: osdmap e26: 4 total, 4 up, 4 in 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi007 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi007 to 8638M 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:52:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:51.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:51 smithi007 ceph-mon[33361]: osdmap e26: 4 total, 4 up, 4 in 2024-01-11T17:52:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: osdmap e27: 4 total, 4 up, 4 in 2024-01-11T17:52:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/3553999197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]: dispatch 2024-01-11T17:52:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]: dispatch 2024-01-11T17:52:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]': finished 2024-01-11T17:52:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: osdmap e28: 5 total, 4 up, 5 in 2024-01-11T17:52:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:52:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/3457862340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: osdmap e27: 4 total, 4 up, 4 in 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/3553999197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]: dispatch 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]: dispatch 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]': finished 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: osdmap e28: 5 total, 4 up, 5 in 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:52:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/3457862340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: osdmap e27: 4 total, 4 up, 4 in 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/3553999197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]: dispatch 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]: dispatch 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3740f4c2-4680-4dce-b64a-65e8405a98e8"}]': finished 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: osdmap e28: 5 total, 4 up, 5 in 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:52:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/3457862340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:52:54.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:54 smithi007 ceph-mon[33361]: osdmap e29: 5 total, 4 up, 5 in 2024-01-11T17:52:54.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:52:54.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:54 smithi016 ceph-mon[43195]: osdmap e29: 5 total, 4 up, 5 in 2024-01-11T17:52:54.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:52:55.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:54 smithi038 ceph-mon[43582]: osdmap e29: 5 total, 4 up, 5 in 2024-01-11T17:52:55.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:52:55.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:55 smithi007 ceph-mon[33361]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:55.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:55 smithi016 ceph-mon[43195]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:56.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:55 smithi038 ceph-mon[43582]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:52:57.792 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:57 smithi016 ceph-mon[43195]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-01-11T17:52:57.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:57 smithi007 ceph-mon[33361]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-01-11T17:52:58.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:57 smithi038 ceph-mon[43582]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-01-11T17:52:59.671 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:59 smithi016 ceph-mon[43195]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-01-11T17:52:59.671 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-11T17:52:59.671 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:52:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:52:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:59 smithi007 ceph-mon[33361]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-01-11T17:52:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-11T17:52:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:52:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:59 smithi038 ceph-mon[43582]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-01-11T17:53:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-11T17:53:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:52:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:00.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:00 smithi007 ceph-mon[33361]: Deploying daemon osd.4 on smithi016 2024-01-11T17:53:00.829 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:00 smithi016 ceph-mon[43195]: Deploying daemon osd.4 on smithi016 2024-01-11T17:53:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:00 smithi038 ceph-mon[43582]: Deploying daemon osd.4 on smithi016 2024-01-11T17:53:01.774 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:01 smithi016 ceph-mon[43195]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-11T17:53:01.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:01 smithi007 ceph-mon[33361]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-11T17:53:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:01 smithi038 ceph-mon[43582]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-11T17:53:03.808 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:03 smithi016 ceph-mon[43195]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-11T17:53:03.808 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:03.808 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:03.808 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:03.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:03 smithi007 ceph-mon[33361]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-11T17:53:03.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:03.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:03.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:03 smithi038 ceph-mon[43582]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-11T17:53:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:05.512 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:05 smithi016 ceph-mon[43195]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-01-11T17:53:05.512 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.512 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.513 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:05.513 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:05.513 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:05 smithi038 ceph-mon[43582]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-01-11T17:53:05.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:05.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:05.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:05 smithi007 ceph-mon[33361]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-01-11T17:53:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:05.784 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 4 on host 'smithi016' 2024-01-11T17:53:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.783+0000 7fa244ff9700 1 -- 172.21.15.16:0/3907026108 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa24802d500 con 0x7fa2300a1ee0 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2300a1ee0 msgr2=0x7fa2300a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2300a1ee0 0x7fa2300a43a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009bf0 tx=0x7fa23c009260 comp rx=0 tx=0).stop 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 msgr2=0x7fa248197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248197520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa238003730 tx=0x7fa23800f720 comp rx=0 tx=0).stop 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 shutdown_connections 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2300a1ee0 0x7fa2300a43a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248103200 0x7fa24819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa248101660 0x7fa248197520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 --2- 172.21.15.16:0/3907026108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa248102860 0x7fa248197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 >> 172.21.15.16:0/3907026108 conn(0x7fa2480fcc20 msgr2=0x7fa248105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:53:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 shutdown_connections 2024-01-11T17:53:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:05.786+0000 7fa24e03d700 1 -- 172.21.15.16:0/3907026108 wait complete. 2024-01-11T17:53:06.187 DEBUG:teuthology.orchestra.run.smithi016:osd.4> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.4.service 2024-01-11T17:53:06.189 INFO:tasks.cephadm:Deploying osd.5 on smithi038 with /dev/vg_nvme/lv_3... 2024-01-11T17:53:06.190 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-11T17:53:06.226 INFO:journalctl@ceph.osd.4.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:53:06.483 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:53:06.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:06.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:06.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:06.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:06 smithi016 ceph-mon[43195]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:53:06.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:06.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:06.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:06.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:06 smithi038 ceph-mon[43582]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:53:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:06 smithi007 ceph-mon[33361]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:53:07.496 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 17:53:07 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4[56313]: 2024-01-11T17:53:07.142+0000 7f79efc39640 -1 osd.4 0 log_to_monitors true 2024-01-11T17:53:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-11T17:53:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-11T17:53:07.936 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-01-11T17:53:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-01-11T17:53:07.936 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011362 s, 923 MB/s 2024-01-11T17:53:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-01-11T17:53:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:07 smithi016 ceph-mon[43195]: from='osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-11T17:53:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:07 smithi016 ceph-mon[43195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-11T17:53:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:07 smithi038 ceph-mon[43582]: from='osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-11T17:53:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:07 smithi038 ceph-mon[43582]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-11T17:53:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:07 smithi007 ceph-mon[33361]: from='osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-11T17:53:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:07 smithi007 ceph-mon[33361]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-11T17:53:08.288 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-01-11T17:53:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:53:09.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-11T17:53:09.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: osdmap e30: 5 total, 4 up, 5 in 2024-01-11T17:53:09.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi016 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi016 to 8559M 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:09.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: osdmap e30: 5 total, 4 up, 5 in 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi016 2024-01-11T17:53:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi016 to 8559M 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-11T17:53:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: osdmap e30: 5 total, 4 up, 5 in 2024-01-11T17:53:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:53:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:53:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi016 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi016 to 8559M 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/4074979194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c102c40 msgr2=0x7ff59c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/4074979194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c102c40 0x7ff59c103060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff598009ca0 comp rx=0 tx=0).stop 2024-01-11T17:53:09.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/4074979194 shutdown_connections 2024-01-11T17:53:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/4074979194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c102c40 0x7ff59c103060 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/4074979194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c100080 0x7ff59c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/4074979194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0ff6c0 0x7ff59c0ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/4074979194 >> 172.21.15.38:0/4074979194 conn(0x7ff59c0fb280 msgr2=0x7ff59c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/4074979194 shutdown_connections 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/4074979194 wait complete. 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 Processor -- start 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.585+0000 7ff5a2ad6700 1 -- start start 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0ff6c0 0x7ff59c102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 0x7ff59c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102c40 0x7ff59c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c101650 con 0x7ff59c0ff6c0 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1017c0 con 0x7ff59c102c40 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1a1580 con 0x7ff59c100080 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 0x7ff59c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 0x7ff59c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45856/0 (socket says 172.21.15.38:45856) 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 -- 172.21.15.38:0/3716739405 learned_addr learned my addr 172.21.15.38:0/3716739405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a1073700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102c40 0x7ff59c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a0872700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0ff6c0 0x7ff59c102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 -- 172.21.15.38:0/3716739405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102c40 msgr2=0x7ff59c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102c40 0x7ff59c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 -- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0ff6c0 msgr2=0x7ff59c102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0ff6c0 0x7ff59c102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 -- 172.21.15.38:0/3716739405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980095f0 con 0x7ff59c100080 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a1073700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102c40 0x7ff59c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff593fff700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 0x7ff59c100990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00b500 tx=0x7ff58c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c011830 con 0x7ff59c100080 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c1a1860 con 0x7ff59c100080 2024-01-11T17:53:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58c011e50 con 0x7ff59c100080 2024-01-11T17:53:09.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.586+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00f2e0 con 0x7ff59c100080 2024-01-11T17:53:09.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.587+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c1a1db0 con 0x7ff59c100080 2024-01-11T17:53:09.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.588+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c108ba0 con 0x7ff59c100080 2024-01-11T17:53:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.589+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7ff58c010580 con 0x7ff59c100080 2024-01-11T17:53:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.589+0000 7ff591ffb700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff5840a1ee0 0x7ff5840a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.590+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3821+0+0 (secure 0 0 0) 0x7ff58c0cc2f0 con 0x7ff59c100080 2024-01-11T17:53:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.590+0000 7ff5a0872700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff5840a1ee0 0x7ff5840a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.590+0000 7ff5a0872700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff5840a1ee0 0x7ff5840a43a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff59800b5c0 tx=0x7ff598005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:53:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.594+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff58c096520 con 0x7ff59c100080 2024-01-11T17:53:09.746 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4[56313]: 2024-01-11T17:53:09.284+0000 7f79e529f700 -1 osd.4 0 waiting for initial osdmap 2024-01-11T17:53:09.746 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4[56313]: 2024-01-11T17:53:09.290+0000 7f79dd091700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:53:09.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:09.817+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff59c02d3c0 con 0x7ff5840a1ee0 2024-01-11T17:53:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:09 smithi038 ceph-mon[43582]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:53:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:09 smithi038 ceph-mon[43582]: osdmap e31: 5 total, 4 up, 5 in 2024-01-11T17:53:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:09 smithi038 ceph-mon[43582]: from='osd.4 ' entity='osd.4' 2024-01-11T17:53:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:09 smithi007 ceph-mon[33361]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:53:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:09 smithi007 ceph-mon[33361]: osdmap e31: 5 total, 4 up, 5 in 2024-01-11T17:53:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:09 smithi007 ceph-mon[33361]: from='osd.4 ' entity='osd.4' 2024-01-11T17:53:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-mon[43195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:53:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-mon[43195]: osdmap e31: 5 total, 4 up, 5 in 2024-01-11T17:53:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:09 smithi016 ceph-mon[43195]: from='osd.4 ' entity='osd.4' 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] boot 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: osdmap e32: 5 total, 5 up, 5 in 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:10.993 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:10 smithi038 ceph-mon[43582]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:53:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:53:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:53:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] boot 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: osdmap e32: 5 total, 5 up, 5 in 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:10 smithi007 ceph-mon[33361]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:11.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: osd.4 [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] boot 2024-01-11T17:53:11.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: osdmap e32: 5 total, 5 up, 5 in 2024-01-11T17:53:11.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-11T17:53:11.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:10 smithi016 ceph-mon[43195]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-01-11T17:53:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: osdmap e33: 5 total, 5 up, 5 in 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/416065888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]: dispatch 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]: dispatch 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]': finished 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: osdmap e34: 6 total, 5 up, 6 in 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/732392377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: osdmap e33: 5 total, 5 up, 5 in 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/416065888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]: dispatch 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]: dispatch 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]': finished 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: osdmap e34: 6 total, 5 up, 6 in 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/732392377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: osdmap e33: 5 total, 5 up, 5 in 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/416065888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]: dispatch 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]: dispatch 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d"}]': finished 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: osdmap e34: 6 total, 5 up, 6 in 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/732392377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:15.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:15 smithi038 ceph-mon[43582]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:15.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:15 smithi007 ceph-mon[33361]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:15 smithi016 ceph-mon[43195]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:17.894 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:17 smithi038 ceph-mon[43582]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:17.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:17 smithi016 ceph-mon[43195]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:18.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:17 smithi007 ceph-mon[33361]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:18.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-11T17:53:18.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-11T17:53:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:19.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-11T17:53:19.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:19.886 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:19 smithi038 ceph-mon[43582]: Deploying daemon osd.5 on smithi038 2024-01-11T17:53:19.886 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:19 smithi038 ceph-mon[43582]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:19 smithi016 ceph-mon[43195]: Deploying daemon osd.5 on smithi038 2024-01-11T17:53:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:19 smithi016 ceph-mon[43195]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:19 smithi007 ceph-mon[33361]: Deploying daemon osd.5 on smithi038 2024-01-11T17:53:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:19 smithi007 ceph-mon[33361]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:21.864 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:21 smithi038 ceph-mon[43582]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:21 smithi016 ceph-mon[43195]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:21 smithi007 ceph-mon[33361]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:23.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:23.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:23.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:23.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:22 smithi038 ceph-mon[43582]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:23.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:23.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:23.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:23.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:22 smithi007 ceph-mon[33361]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:22 smithi016 ceph-mon[43195]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:24.478 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 5 on host 'smithi038' 2024-01-11T17:53:24.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.476+0000 7ff591ffb700 1 -- 172.21.15.38:0/3716739405 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff59c02d3c0 con 0x7ff5840a1ee0 2024-01-11T17:53:24.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff5840a1ee0 msgr2=0x7ff5840a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:24.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff5840a1ee0 0x7ff5840a43a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff59800b5c0 tx=0x7ff598005fb0 comp rx=0 tx=0).stop 2024-01-11T17:53:24.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 msgr2=0x7ff59c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 0x7ff59c100990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00b500 tx=0x7ff58c00b8c0 comp rx=0 tx=0).stop 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 shutdown_connections 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff5840a1ee0 0x7ff5840a43a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102c40 0x7ff59c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c0ff6c0 0x7ff59c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 --2- 172.21.15.38:0/3716739405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff59c100080 0x7ff59c100990 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 >> 172.21.15.38:0/3716739405 conn(0x7ff59c0fb280 msgr2=0x7ff59c105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 shutdown_connections 2024-01-11T17:53:24.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:53:24.479+0000 7ff5a2ad6700 1 -- 172.21.15.38:0/3716739405 wait complete. 2024-01-11T17:53:24.903 DEBUG:teuthology.orchestra.run.smithi038:osd.5> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.5.service 2024-01-11T17:53:24.905 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-01-11T17:53:24.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:24.936 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:24 smithi038 ceph-mon[43582]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:24.943 INFO:journalctl@ceph.osd.5.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:25.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:24 smithi007 ceph-mon[33361]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:24 smithi016 ceph-mon[43195]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:53:26.026 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 17:53:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T17:53:25.989+0000 7fea681c3640 -1 osd.5 0 log_to_monitors true 2024-01-11T17:53:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:26 smithi007 ceph-mon[33361]: from='osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-11T17:53:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:26 smithi007 ceph-mon[33361]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-11T17:53:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:26 smithi016 ceph-mon[43195]: from='osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-11T17:53:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:26 smithi016 ceph-mon[43195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-11T17:53:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:26 smithi038 ceph-mon[43582]: from='osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-11T17:53:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:26 smithi038 ceph-mon[43582]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-11T17:53:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-11T17:53:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-11T17:53:27.048 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-01-11T17:53:27.049 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-01-11T17:53:27.049 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0441585 s, 237 MB/s 2024-01-11T17:53:27.049 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-01-11T17:53:27.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-01-11T17:53:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-11T17:53:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: osdmap e35: 6 total, 5 up, 6 in 2024-01-11T17:53:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:53:27.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:27.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: osdmap e35: 6 total, 5 up, 6 in 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: osdmap e35: 6 total, 5 up, 6 in 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:27.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:27.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:53:28.276 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T17:53:27.972+0000 7fea5d829700 -1 osd.5 0 waiting for initial osdmap 2024-01-11T17:53:28.276 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 17:53:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T17:53:27.981+0000 7fea5561b700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi038 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi038 to 8559M 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: osdmap e36: 6 total, 5 up, 6 in 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:28 smithi007 ceph-mon[33361]: from='osd.5 ' entity='osd.5' 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi038 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi038 to 8559M 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: osdmap e36: 6 total, 5 up, 6 in 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:28 smithi016 ceph-mon[43195]: from='osd.5 ' entity='osd.5' 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi038 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi038 to 8559M 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: osdmap e36: 6 total, 5 up, 6 in 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:28 smithi038 ceph-mon[43582]: from='osd.5 ' entity='osd.5' 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 -- 172.21.15.7:0/2977447781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604101650 msgr2=0x7f8604101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 --2- 172.21.15.7:0/2977447781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604101650 0x7f8604101a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86000099c0 tx=0x7f8600009cd0 comp rx=0 tx=0).stop 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 -- 172.21.15.7:0/2977447781 shutdown_connections 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 --2- 172.21.15.7:0/2977447781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86041031f0 0x7f8604109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 --2- 172.21.15.7:0/2977447781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604101650 0x7f8604101a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 --2- 172.21.15.7:0/2977447781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8604102850 0x7f8604102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 -- 172.21.15.7:0/2977447781 >> 172.21.15.7:0/2977447781 conn(0x7f86040fcbb0 msgr2=0x7f86040ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:53:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 -- 172.21.15.7:0/2977447781 shutdown_connections 2024-01-11T17:53:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.949+0000 7f860c248700 1 -- 172.21.15.7:0/2977447781 wait complete. 2024-01-11T17:53:28.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.950+0000 7f860c248700 1 Processor -- start 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 -- start start 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 0x7f8604199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604102850 0x7f8604199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86041031f0 0x7f86041a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86041a12f0 con 0x7f8604102850 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86041a1430 con 0x7f86041031f0 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860c248700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86041a15c0 con 0x7f8604101650 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f8609fe4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 0x7f8604199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f86097e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604102850 0x7f8604199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f8609fe4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 0x7f8604199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:57430/0 (socket says 172.21.15.7:57430) 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f8609fe4700 1 -- 172.21.15.7:0/3943959281 learned_addr learned my addr 172.21.15.7:0/3943959281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:53:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.951+0000 7f860a7e5700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86041031f0 0x7f86041a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:28.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f8609fe4700 1 -- 172.21.15.7:0/3943959281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86041031f0 msgr2=0x7f86041a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:28.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f8609fe4700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86041031f0 0x7f86041a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:28.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f8609fe4700 1 -- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604102850 msgr2=0x7f8604199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:28.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f8609fe4700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604102850 0x7f8604199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:28.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f8609fe4700 1 -- 172.21.15.7:0/3943959281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000095f0 con 0x7f8604101650 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f8609fe4700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 0x7f8604199760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8600003730 tx=0x7f860000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860001d070 con 0x7f8604101650 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86041a1840 con 0x7f8604101650 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8600005030 con 0x7f8604101650 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86000049a0 con 0x7f8604101650 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.952+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86041a1cb0 con 0x7f8604101650 2024-01-11T17:53:28.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.953+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86040682d0 con 0x7f8604101650 2024-01-11T17:53:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.956+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f86000051a0 con 0x7f8604101650 2024-01-11T17:53:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.957+0000 7f85faffd700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f85f00a1ee0 0x7f85f00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.957+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f86000cdbe0 con 0x7f8604101650 2024-01-11T17:53:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.957+0000 7f86097e3700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f85f00a1ee0 0x7f85f00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.958+0000 7f86097e3700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f85f00a1ee0 0x7f85f00a43a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009c60 tx=0x7f85f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:53:28.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:28.959+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f8600097b10 con 0x7f8604101650 2024-01-11T17:53:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:29.180+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8604074f60 con 0x7f85f00a1ee0 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] boot 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: osdmap e37: 6 total, 6 up, 6 in 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:29 smithi016 ceph-mon[43195]: osdmap e38: 6 total, 6 up, 6 in 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] boot 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: osdmap e37: 6 total, 6 up, 6 in 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:29 smithi038 ceph-mon[43582]: osdmap e38: 6 total, 6 up, 6 in 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: osd.5 [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] boot 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: osdmap e37: 6 total, 6 up, 6 in 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:29.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:29.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:29 smithi007 ceph-mon[33361]: osdmap e38: 6 total, 6 up, 6 in 2024-01-11T17:53:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:30 smithi016 ceph-mon[43195]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:30 smithi016 ceph-mon[43195]: osdmap e39: 6 total, 6 up, 6 in 2024-01-11T17:53:30.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:30 smithi038 ceph-mon[43582]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:30.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:30 smithi038 ceph-mon[43582]: osdmap e39: 6 total, 6 up, 6 in 2024-01-11T17:53:30.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:30 smithi007 ceph-mon[33361]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:30.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:30 smithi007 ceph-mon[33361]: osdmap e39: 6 total, 6 up, 6 in 2024-01-11T17:53:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:31 smithi016 ceph-mon[43195]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-11T17:53:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:31 smithi016 ceph-mon[43195]: osdmap e40: 6 total, 6 up, 6 in 2024-01-11T17:53:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:31 smithi038 ceph-mon[43582]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-11T17:53:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:31 smithi038 ceph-mon[43582]: osdmap e40: 6 total, 6 up, 6 in 2024-01-11T17:53:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:31 smithi007 ceph-mon[33361]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-11T17:53:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:31 smithi007 ceph-mon[33361]: osdmap e40: 6 total, 6 up, 6 in 2024-01-11T17:53:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2054191863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "731711fe-aa80-4c77-a022-b1a16f24306b"}]: dispatch 2024-01-11T17:53:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2054191863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "731711fe-aa80-4c77-a022-b1a16f24306b"}]': finished 2024-01-11T17:53:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:32 smithi016 ceph-mon[43195]: osdmap e41: 7 total, 6 up, 7 in 2024-01-11T17:53:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2054191863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "731711fe-aa80-4c77-a022-b1a16f24306b"}]: dispatch 2024-01-11T17:53:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2054191863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "731711fe-aa80-4c77-a022-b1a16f24306b"}]': finished 2024-01-11T17:53:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:32 smithi038 ceph-mon[43582]: osdmap e41: 7 total, 6 up, 7 in 2024-01-11T17:53:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2054191863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "731711fe-aa80-4c77-a022-b1a16f24306b"}]: dispatch 2024-01-11T17:53:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2054191863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "731711fe-aa80-4c77-a022-b1a16f24306b"}]': finished 2024-01-11T17:53:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:32 smithi007 ceph-mon[33361]: osdmap e41: 7 total, 6 up, 7 in 2024-01-11T17:53:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:33.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:33 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3297215538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:33.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:33 smithi016 ceph-mon[43195]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-11T17:53:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:33 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3297215538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:33 smithi038 ceph-mon[43582]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-11T17:53:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:33 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3297215538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:33 smithi007 ceph-mon[33361]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-11T17:53:35.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:35 smithi038 ceph-mon[43582]: pgmap v126: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-01-11T17:53:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:35 smithi007 ceph-mon[33361]: pgmap v126: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-01-11T17:53:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:35 smithi016 ceph-mon[43195]: pgmap v126: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-01-11T17:53:37.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:37 smithi007 ceph-mon[33361]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-11T17:53:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:37 smithi016 ceph-mon[43195]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-11T17:53:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:37 smithi038 ceph-mon[43582]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-11T17:53:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-11T17:53:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-11T17:53:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:39.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-11T17:53:39.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:39 smithi007 ceph-mon[33361]: Deploying daemon osd.6 on smithi007 2024-01-11T17:53:39.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:39 smithi007 ceph-mon[33361]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:39 smithi016 ceph-mon[43195]: Deploying daemon osd.6 on smithi007 2024-01-11T17:53:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:39 smithi016 ceph-mon[43195]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:39 smithi038 ceph-mon[43582]: Deploying daemon osd.6 on smithi007 2024-01-11T17:53:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:39 smithi038 ceph-mon[43582]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:41.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:41 smithi007 ceph-mon[33361]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:41 smithi016 ceph-mon[43195]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:42.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:41 smithi038 ceph-mon[43582]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:43 smithi016 ceph-mon[43195]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:43 smithi038 ceph-mon[43582]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:43 smithi007 ceph-mon[33361]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:44 smithi016 ceph-mon[43195]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:45.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:45.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:45.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:45.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:44 smithi038 ceph-mon[43582]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:44 smithi007 ceph-mon[33361]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:46.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.580+0000 7f85faffd700 1 -- 172.21.15.7:0/3943959281 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8604074f60 con 0x7f85f00a1ee0 2024-01-11T17:53:46.582 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-01-11T17:53:46.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f85f00a1ee0 msgr2=0x7f85f00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:46.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f85f00a1ee0 0x7f85f00a43a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009c60 tx=0x7f85f4009260 comp rx=0 tx=0).stop 2024-01-11T17:53:46.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 msgr2=0x7f8604199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:46.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 0x7f8604199760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8600003730 tx=0x7f860000f690 comp rx=0 tx=0).stop 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 shutdown_connections 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f85f00a1ee0 0x7f85f00a43a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8604101650 0x7f8604199760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86041031f0 0x7f86041a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 --2- 172.21.15.7:0/3943959281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8604102850 0x7f8604199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 >> 172.21.15.7:0/3943959281 conn(0x7f86040fcbb0 msgr2=0x7f8604073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 shutdown_connections 2024-01-11T17:53:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:53:46.582+0000 7f860c248700 1 -- 172.21.15.7:0/3943959281 wait complete. 2024-01-11T17:53:47.007 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.6.service 2024-01-11T17:53:47.011 INFO:tasks.cephadm:Deploying osd.7 on smithi016 with /dev/vg_nvme/lv_2... 2024-01-11T17:53:47.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-11T17:53:47.062 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.062 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.062 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:47.062 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:47.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:47.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:46 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:46 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:47.121 INFO:journalctl@ceph.osd.6.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:46 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:53:47.330 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:53:48.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:47 smithi007 ceph-mon[33361]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-11T17:53:48.037 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 17:53:47 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T17:53:47.784+0000 7f1bdeb25640 -1 osd.6 0 log_to_monitors true 2024-01-11T17:53:48.101 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:47 smithi016 ceph-mon[43195]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-11T17:53:48.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:47 smithi038 ceph-mon[43582]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-11T17:53:48.649 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-11T17:53:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-11T17:53:48.650 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-01-11T17:53:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-01-11T17:53:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0092053 s, 1.1 GB/s 2024-01-11T17:53:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-01-11T17:53:48.977 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:48 smithi016 ceph-mon[43195]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-11T17:53:48.977 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:48 smithi016 ceph-mon[43195]: osdmap e42: 7 total, 6 up, 7 in 2024-01-11T17:53:48.977 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:48 smithi016 ceph-mon[43195]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:53:48.977 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:48.978 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:48 smithi016 ceph-mon[43195]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:49.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-01-11T17:53:49.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:48 smithi007 ceph-mon[33361]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-11T17:53:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:48 smithi007 ceph-mon[33361]: osdmap e42: 7 total, 6 up, 7 in 2024-01-11T17:53:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:48 smithi007 ceph-mon[33361]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:53:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:48 smithi007 ceph-mon[33361]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:49.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:48 smithi038 ceph-mon[43582]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-11T17:53:49.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:48 smithi038 ceph-mon[43582]: osdmap e42: 7 total, 6 up, 7 in 2024-01-11T17:53:49.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:48 smithi038 ceph-mon[43582]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:53:49.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:49.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:48 smithi038 ceph-mon[43582]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:53:49.821 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T17:53:49.382+0000 7f1bd418b700 -1 osd.6 0 waiting for initial osdmap 2024-01-11T17:53:49.821 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T17:53:49.406+0000 7f1bcbf7d700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:53:50.165 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:49 smithi016 ceph-mon[43195]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:53:50.165 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:49 smithi016 ceph-mon[43195]: osdmap e43: 7 total, 6 up, 7 in 2024-01-11T17:53:50.165 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:50.165 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:50.166 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:49 smithi016 ceph-mon[43195]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' 2024-01-11T17:53:50.237 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-mon[33361]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:53:50.237 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-mon[33361]: osdmap e43: 7 total, 6 up, 7 in 2024-01-11T17:53:50.237 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:50.237 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:50.238 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:49 smithi007 ceph-mon[33361]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' 2024-01-11T17:53:50.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:49 smithi038 ceph-mon[43582]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:53:50.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:49 smithi038 ceph-mon[43582]: osdmap e43: 7 total, 6 up, 7 in 2024-01-11T17:53:50.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:50.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:50.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:49 smithi038 ceph-mon[43582]: from='osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079]' entity='osd.6' 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 -- 172.21.15.16:0/3356059761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8102180 msgr2=0x7f4ed8104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 --2- 172.21.15.16:0/3356059761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8102180 0x7f4ed8104570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009990 tx=0x7f4ecc009ca0 comp rx=0 tx=0).stop 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 -- 172.21.15.16:0/3356059761 shutdown_connections 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 --2- 172.21.15.16:0/3356059761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed81073e0 0x7f4ed81097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 --2- 172.21.15.16:0/3356059761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8102180 0x7f4ed8104570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 --2- 172.21.15.16:0/3356059761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed8104ab0 0x7f4ed8106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 -- 172.21.15.16:0/3356059761 >> 172.21.15.16:0/3356059761 conn(0x7f4ed80fbad0 msgr2=0x7f4ed80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:53:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 -- 172.21.15.16:0/3356059761 shutdown_connections 2024-01-11T17:53:50.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.386+0000 7f4edeb56700 1 -- 172.21.15.16:0/3356059761 wait complete. 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 Processor -- start 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 -- start start 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8102180 0x7f4ed81974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed8104ab0 0x7f4ed8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 0x7f4ed819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed819f070 con 0x7f4ed8102180 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed819f1b0 con 0x7f4ed81073e0 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edeb56700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed819f340 con 0x7f4ed8104ab0 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edd0f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 0x7f4ed819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edd0f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 0x7f4ed819eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39864/0 (socket says 172.21.15.16:39864) 2024-01-11T17:53:50.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edc8f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8102180 0x7f4ed81974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.387+0000 7f4edd0f3700 1 -- 172.21.15.16:0/988138810 learned_addr learned my addr 172.21.15.16:0/988138810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.388+0000 7f4edd0f3700 1 -- 172.21.15.16:0/988138810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed8104ab0 msgr2=0x7f4ed8197a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.388+0000 7f4edd0f3700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed8104ab0 0x7f4ed8197a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.388+0000 7f4edd0f3700 1 -- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8102180 msgr2=0x7f4ed81974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.388+0000 7f4edd0f3700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8102180 0x7f4ed81974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.388+0000 7f4edd0f3700 1 -- 172.21.15.16:0/988138810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0095f0 con 0x7f4ed81073e0 2024-01-11T17:53:50.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.388+0000 7f4edd0f3700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 0x7f4ed819eaa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0007ee0 tx=0x7f4ed000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:53:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.390+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000ddd0 con 0x7f4ed81073e0 2024-01-11T17:53:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.390+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed000f040 con 0x7f4ed81073e0 2024-01-11T17:53:50.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.390+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed819f620 con 0x7f4ed81073e0 2024-01-11T17:53:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.390+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0015400 con 0x7f4ed81073e0 2024-01-11T17:53:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.391+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed819fb20 con 0x7f4ed81073e0 2024-01-11T17:53:50.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.392+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed80fd650 con 0x7f4ed81073e0 2024-01-11T17:53:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.392+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4ed0003a00 con 0x7f4ed81073e0 2024-01-11T17:53:50.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.393+0000 7f4ed5ffb700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec80a1f60 0x7f4ec80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:53:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.393+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f4ed00cc4a0 con 0x7f4ed81073e0 2024-01-11T17:53:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.393+0000 7f4edc8f2700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec80a1f60 0x7f4ec80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:53:50.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.394+0000 7f4edc8f2700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec80a1f60 0x7f4ec80a4420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b5c0 tx=0x7f4ecc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:53:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.398+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f4ed004f380 con 0x7f4ed81073e0 2024-01-11T17:53:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:53:50.620+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed8061a50 con 0x7f4ec80a1f60 2024-01-11T17:53:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:53:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:53:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] boot 2024-01-11T17:53:51.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: osdmap e44: 7 total, 7 up, 7 in 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:51.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:51.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.139 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:53:51.139 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:53:51.139 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] boot 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: osdmap e44: 7 total, 7 up, 7 in 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.140 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:51.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:51.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:53:51.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:53:51.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:51.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: osd.6 [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] boot 2024-01-11T17:53:51.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: osdmap e44: 7 total, 7 up, 7 in 2024-01-11T17:53:51.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:51.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:53:51.278 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:53:51.278 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:53:52.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:52 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi007 2024-01-11T17:53:52.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:52 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi007 to 5758M 2024-01-11T17:53:52.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:52 smithi016 ceph-mon[43195]: osdmap e45: 7 total, 7 up, 7 in 2024-01-11T17:53:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:52 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi007 2024-01-11T17:53:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:52 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi007 to 5758M 2024-01-11T17:53:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:52 smithi038 ceph-mon[43582]: osdmap e45: 7 total, 7 up, 7 in 2024-01-11T17:53:52.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:52 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi007 2024-01-11T17:53:52.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:52 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi007 to 5758M 2024-01-11T17:53:52.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:52 smithi007 ceph-mon[33361]: osdmap e45: 7 total, 7 up, 7 in 2024-01-11T17:53:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/2386472312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]: dispatch 2024-01-11T17:53:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]: dispatch 2024-01-11T17:53:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]': finished 2024-01-11T17:53:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: osdmap e46: 8 total, 7 up, 8 in 2024-01-11T17:53:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:53:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/1959988919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/2386472312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]: dispatch 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]: dispatch 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]': finished 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: osdmap e46: 8 total, 7 up, 8 in 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:53:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/1959988919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/2386472312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]: dispatch 2024-01-11T17:53:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]: dispatch 2024-01-11T17:53:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1e933c6-92f2-4974-83cf-633e3530103e"}]': finished 2024-01-11T17:53:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: osdmap e46: 8 total, 7 up, 8 in 2024-01-11T17:53:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:53:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/1959988919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:53:55.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:55 smithi007 ceph-mon[33361]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:55.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:55 smithi016 ceph-mon[43195]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:55.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:55 smithi038 ceph-mon[43582]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:57.924 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:57 smithi016 ceph-mon[43195]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:58.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:57 smithi038 ceph-mon[43582]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:57 smithi007 ceph-mon[33361]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:59.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:59 smithi016 ceph-mon[43195]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:53:59.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-11T17:53:59.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:53:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:59 smithi038 ceph-mon[43582]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-11T17:54:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:53:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:59 smithi007 ceph-mon[33361]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-11T17:54:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:53:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:00.959 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:00 smithi016 ceph-mon[43195]: Deploying daemon osd.7 on smithi016 2024-01-11T17:54:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:00 smithi038 ceph-mon[43582]: Deploying daemon osd.7 on smithi016 2024-01-11T17:54:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:00 smithi007 ceph-mon[33361]: Deploying daemon osd.7 on smithi016 2024-01-11T17:54:01.870 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:01 smithi016 ceph-mon[43195]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:01 smithi038 ceph-mon[43582]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:01 smithi007 ceph-mon[33361]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:03.726 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:03 smithi016 ceph-mon[43195]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:03.726 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:03.726 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:03.726 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:03 smithi038 ceph-mon[43582]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:03 smithi007 ceph-mon[33361]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:05.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.081+0000 7f4ed5ffb700 1 -- 172.21.15.16:0/988138810 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4ed8061a50 con 0x7f4ec80a1f60 2024-01-11T17:54:05.083 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 7 on host 'smithi016' 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec80a1f60 msgr2=0x7f4ec80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec80a1f60 0x7f4ec80a4420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b5c0 tx=0x7f4ecc005fd0 comp rx=0 tx=0).stop 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 msgr2=0x7f4ed819eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 0x7f4ed819eaa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0007ee0 tx=0x7f4ed000d3b0 comp rx=0 tx=0).stop 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 shutdown_connections 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec80a1f60 0x7f4ec80a4420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed81073e0 0x7f4ed819eaa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8102180 0x7f4ed81974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 --2- 172.21.15.16:0/988138810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed8104ab0 0x7f4ed8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 >> 172.21.15.16:0/988138810 conn(0x7f4ed80fbad0 msgr2=0x7f4ed8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:54:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 shutdown_connections 2024-01-11T17:54:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:05.085+0000 7f4edeb56700 1 -- 172.21.15.16:0/988138810 wait complete. 2024-01-11T17:54:05.467 DEBUG:teuthology.orchestra.run.smithi016:osd.7> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.7.service 2024-01-11T17:54:05.469 INFO:tasks.cephadm:Deploying osd.8 on smithi038 with /dev/vg_nvme/lv_2... 2024-01-11T17:54:05.469 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-11T17:54:05.505 INFO:journalctl@ceph.osd.7.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:05.630 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.630 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.630 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:05.630 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:05.630 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.631 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:05.631 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.631 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.631 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:05.785 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:54:06.997 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 17:54:06 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T17:54:06.647+0000 7fb5a95ab640 -1 osd.7 0 log_to_monitors true 2024-01-11T17:54:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-11T17:54:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-11T17:54:07.174 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-01-11T17:54:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-01-11T17:54:07.174 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.008212 s, 1.3 GB/s 2024-01-11T17:54:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-01-11T17:54:07.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-01-11T17:54:07.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:07 smithi038 ceph-mon[43582]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:07.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:07 smithi038 ceph-mon[43582]: from='osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-11T17:54:07.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:07 smithi038 ceph-mon[43582]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-11T17:54:07.825 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:07 smithi016 ceph-mon[43195]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:07.893 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:07 smithi016 ceph-mon[43195]: from='osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-11T17:54:07.893 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:07 smithi016 ceph-mon[43195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-11T17:54:07.894 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:54:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:07 smithi007 ceph-mon[33361]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:07 smithi007 ceph-mon[33361]: from='osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-11T17:54:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:07 smithi007 ceph-mon[33361]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 -- 172.21.15.38:0/1505140986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0101660 msgr2=0x7f58e0101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 --2- 172.21.15.38:0/1505140986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0101660 0x7f58e0101a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009990 tx=0x7f58d4009ca0 comp rx=0 tx=0).stop 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 -- 172.21.15.38:0/1505140986 shutdown_connections 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 --2- 172.21.15.38:0/1505140986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0101660 0x7f58e0101a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 --2- 172.21.15.38:0/1505140986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e0103200 0x7f58e0109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 --2- 172.21.15.38:0/1505140986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e0102860 0x7f58e0102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 -- 172.21.15.38:0/1505140986 >> 172.21.15.38:0/1505140986 conn(0x7f58e00fcc20 msgr2=0x7f58e00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 -- 172.21.15.38:0/1505140986 shutdown_connections 2024-01-11T17:54:08.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.878+0000 7f58e5775700 1 -- 172.21.15.38:0/1505140986 wait complete. 2024-01-11T17:54:08.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 Processor -- start 2024-01-11T17:54:08.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 -- start start 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e0101660 0x7f58e00753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e0102860 0x7f58e0073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 0x7f58e0073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00746c0 con 0x7f58e0102860 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0074830 con 0x7f58e0101660 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58e5775700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e01a16e0 con 0x7f58e0103200 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58df7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 0x7f58e0073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58df7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 0x7f58e0073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42298/0 (socket says 172.21.15.38:42298) 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58df7fe700 1 -- 172.21.15.38:0/1220430308 learned_addr learned my addr 172.21.15.38:0/1220430308 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.879+0000 7f58deffd700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e0101660 0x7f58e00753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58df7fe700 1 -- 172.21.15.38:0/1220430308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e0101660 msgr2=0x7f58e00753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58df7fe700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e0101660 0x7f58e00753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58df7fe700 1 -- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e0102860 msgr2=0x7f58e0073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58df7fe700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e0102860 0x7f58e0073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58df7fe700 1 -- 172.21.15.38:0/1220430308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40095f0 con 0x7f58e0103200 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58df7fe700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 0x7f58e0073f40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b970 tx=0x7f58d800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800c9b0 con 0x7f58e0103200 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58d8014440 con 0x7f58e0103200 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e01a19c0 con 0x7f58e0103200 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d80075d0 con 0x7f58e0103200 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.880+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e01a1f40 con 0x7f58e0103200 2024-01-11T17:54:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.881+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e0108b40 con 0x7f58e0103200 2024-01-11T17:54:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.882+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f58d80145b0 con 0x7f58e0103200 2024-01-11T17:54:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.883+0000 7f58c7fff700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f58c80a1f60 0x7f58c80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.883+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f58d8006e40 con 0x7f58e0103200 2024-01-11T17:54:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.883+0000 7f58deffd700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f58c80a1f60 0x7f58c80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.884+0000 7f58deffd700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f58c80a1f60 0x7f58c80a4420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f58d40038c0 tx=0x7f58d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:54:08.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:08.887+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f58d8096590 con 0x7f58e0103200 2024-01-11T17:54:08.996 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T17:54:08.626+0000 7fb59ec11700 -1 osd.7 0 waiting for initial osdmap 2024-01-11T17:54:08.996 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T17:54:08.637+0000 7fb59aa0b700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:54:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-11T17:54:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: osdmap e47: 8 total, 7 up, 8 in 2024-01-11T17:54:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-11T17:54:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: osdmap e47: 8 total, 7 up, 8 in 2024-01-11T17:54:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:54:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: osdmap e47: 8 total, 7 up, 8 in 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:09.104+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f58e002d460 con 0x7f58c80a1f60 2024-01-11T17:54:09.841 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi016 2024-01-11T17:54:09.841 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi016 to 5706M 2024-01-11T17:54:09.841 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:09.841 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:54:09.841 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: osdmap e48: 8 total, 7 up, 8 in 2024-01-11T17:54:09.841 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] boot 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: osdmap e49: 8 total, 8 up, 8 in 2024-01-11T17:54:09.842 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi016 2024-01-11T17:54:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi016 to 5706M 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: osdmap e48: 8 total, 7 up, 8 in 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] boot 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: osdmap e49: 8 total, 8 up, 8 in 2024-01-11T17:54:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi016 2024-01-11T17:54:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi016 to 5706M 2024-01-11T17:54:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-01-11T17:54:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: osdmap e48: 8 total, 7 up, 8 in 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: osd.7 [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] boot 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: osdmap e49: 8 total, 8 up, 8 in 2024-01-11T17:54:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-11T17:54:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:10 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:54:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:10 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:54:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:10 smithi016 ceph-mon[43195]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:10 smithi016 ceph-mon[43195]: osdmap e50: 8 total, 8 up, 8 in 2024-01-11T17:54:11.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:10 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:54:11.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:10 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:54:11.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:10 smithi038 ceph-mon[43582]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:11.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:10 smithi038 ceph-mon[43582]: osdmap e50: 8 total, 8 up, 8 in 2024-01-11T17:54:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:10 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:54:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:10 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:54:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:10 smithi007 ceph-mon[33361]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:10 smithi007 ceph-mon[33361]: osdmap e50: 8 total, 8 up, 8 in 2024-01-11T17:54:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:11 smithi016 ceph-mon[43195]: pgmap v153: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:11 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/2480233606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]: dispatch 2024-01-11T17:54:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:11 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]: dispatch 2024-01-11T17:54:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:11 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]': finished 2024-01-11T17:54:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:11 smithi016 ceph-mon[43195]: osdmap e51: 9 total, 8 up, 9 in 2024-01-11T17:54:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:11 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:12.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:11 smithi038 ceph-mon[43582]: pgmap v153: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:12.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:11 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/2480233606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]: dispatch 2024-01-11T17:54:12.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:11 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]: dispatch 2024-01-11T17:54:12.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:11 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]': finished 2024-01-11T17:54:12.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:11 smithi038 ceph-mon[43582]: osdmap e51: 9 total, 8 up, 9 in 2024-01-11T17:54:12.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:11 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:11 smithi007 ceph-mon[33361]: pgmap v153: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:11 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/2480233606' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]: dispatch 2024-01-11T17:54:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:11 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]: dispatch 2024-01-11T17:54:12.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:11 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3bb6d44-a03b-4e41-a286-3f6e2104397d"}]': finished 2024-01-11T17:54:12.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:11 smithi007 ceph-mon[33361]: osdmap e51: 9 total, 8 up, 9 in 2024-01-11T17:54:12.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:11 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/1455035061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:13.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:12 smithi016 ceph-mon[43195]: osdmap e52: 9 total, 8 up, 9 in 2024-01-11T17:54:13.019 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:12 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/1455035061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:12 smithi038 ceph-mon[43582]: osdmap e52: 9 total, 8 up, 9 in 2024-01-11T17:54:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:12 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/1455035061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:12 smithi007 ceph-mon[33361]: osdmap e52: 9 total, 8 up, 9 in 2024-01-11T17:54:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:12 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:13 smithi016 ceph-mon[43195]: pgmap v156: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:13 smithi016 ceph-mon[43195]: osdmap e53: 9 total, 8 up, 9 in 2024-01-11T17:54:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:13 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:13 smithi038 ceph-mon[43582]: pgmap v156: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:13 smithi038 ceph-mon[43582]: osdmap e53: 9 total, 8 up, 9 in 2024-01-11T17:54:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:13 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:13 smithi007 ceph-mon[33361]: pgmap v156: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:13 smithi007 ceph-mon[33361]: osdmap e53: 9 total, 8 up, 9 in 2024-01-11T17:54:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:13 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:15.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:15 smithi038 ceph-mon[43582]: pgmap v158: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:15.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:15 smithi007 ceph-mon[33361]: pgmap v158: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:15 smithi016 ceph-mon[43195]: pgmap v158: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:17.984 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:17 smithi038 ceph-mon[43582]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:17.984 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:17 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-11T17:54:17.984 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:17 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:17.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:17 smithi016 ceph-mon[43195]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:17.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:17 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-11T17:54:17.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:17 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:18.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:17 smithi007 ceph-mon[33361]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:18.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:17 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-11T17:54:18.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:17 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:18 smithi016 ceph-mon[43195]: Deploying daemon osd.8 on smithi038 2024-01-11T17:54:19.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:18 smithi038 ceph-mon[43582]: Deploying daemon osd.8 on smithi038 2024-01-11T17:54:19.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:18 smithi007 ceph-mon[33361]: Deploying daemon osd.8 on smithi038 2024-01-11T17:54:19.967 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:19 smithi038 ceph-mon[43582]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:19 smithi016 ceph-mon[43195]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:19 smithi007 ceph-mon[33361]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:21 smithi016 ceph-mon[43195]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:22.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:21 smithi038 ceph-mon[43582]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:21 smithi007 ceph-mon[33361]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:22.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:22.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:22 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:22 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:23.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:23.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:23.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:22 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:23.927 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:23 smithi038 ceph-mon[43582]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:23.927 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:23.927 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:23 smithi016 ceph-mon[43195]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:23 smithi007 ceph-mon[33361]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:24.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.074+0000 7f58c7fff700 1 -- 172.21.15.38:0/1220430308 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f58e002d460 con 0x7f58c80a1f60 2024-01-11T17:54:24.076 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 8 on host 'smithi038' 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f58c80a1f60 msgr2=0x7f58c80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f58c80a1f60 0x7f58c80a4420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f58d40038c0 tx=0x7f58d4005fb0 comp rx=0 tx=0).stop 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 msgr2=0x7f58e0073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 0x7f58e0073f40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b970 tx=0x7f58d800bd30 comp rx=0 tx=0).stop 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 shutdown_connections 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f58c80a1f60 0x7f58c80a4420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0103200 0x7f58e0073f40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e0101660 0x7f58e00753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:24.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 --2- 172.21.15.38:0/1220430308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e0102860 0x7f58e0073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 >> 172.21.15.38:0/1220430308 conn(0x7f58e00fcc20 msgr2=0x7f58e0105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:54:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 shutdown_connections 2024-01-11T17:54:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:54:24.077+0000 7f58e5775700 1 -- 172.21.15.38:0/1220430308 wait complete. 2024-01-11T17:54:24.518 DEBUG:teuthology.orchestra.run.smithi038:osd.8> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.8.service 2024-01-11T17:54:24.520 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-01-11T17:54:24.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-11T17:54:24.560 INFO:journalctl@ceph.osd.8.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:54:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:54:25.144 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:25.144 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:25.144 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:25 smithi038 ceph-mon[43582]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:25.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:25.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:25.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:25 smithi007 ceph-mon[33361]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:25.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:25.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:25.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:25 smithi016 ceph-mon[43195]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:26.027 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 17:54:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T17:54:25.625+0000 7f594af57640 -1 osd.8 0 log_to_monitors true 2024-01-11T17:54:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:26 smithi007 ceph-mon[33361]: from='osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-11T17:54:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:26 smithi007 ceph-mon[33361]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-11T17:54:26.618 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:26 smithi038 ceph-mon[43582]: from='osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-11T17:54:26.619 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:26 smithi038 ceph-mon[43582]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-11T17:54:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:26 smithi016 ceph-mon[43195]: from='osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-11T17:54:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:26 smithi016 ceph-mon[43195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-11T17:54:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-11T17:54:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-11T17:54:26.772 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-01-11T17:54:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-01-11T17:54:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350912 s, 299 MB/s 2024-01-11T17:54:26.772 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 2024-01-11T17:54:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-01-11T17:54:27.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: osdmap e54: 9 total, 8 up, 9 in 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:27.353 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:54:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-11T17:54:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: osdmap e54: 9 total, 8 up, 9 in 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:27.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:27.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:27.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.776 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T17:54:27.308+0000 7f59405bd700 -1 osd.8 0 waiting for initial osdmap 2024-01-11T17:54:27.776 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T17:54:27.320+0000 7f593c3b7700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:54:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-11T17:54:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: osdmap e54: 9 total, 8 up, 9 in 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:27.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:27.778 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:27.778 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:27.778 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:28 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi038 2024-01-11T17:54:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:28 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi038 to 5706M 2024-01-11T17:54:28.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:28 smithi007 ceph-mon[33361]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:54:28.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:28 smithi007 ceph-mon[33361]: osdmap e55: 9 total, 8 up, 9 in 2024-01-11T17:54:28.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:28.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:28 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi038 2024-01-11T17:54:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:28 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi038 to 5706M 2024-01-11T17:54:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:28 smithi016 ceph-mon[43195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:54:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:28 smithi016 ceph-mon[43195]: osdmap e55: 9 total, 8 up, 9 in 2024-01-11T17:54:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:28 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi038 2024-01-11T17:54:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:28 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi038 to 5706M 2024-01-11T17:54:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:28 smithi038 ceph-mon[43582]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:54:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:28 smithi038 ceph-mon[43582]: osdmap e55: 9 total, 8 up, 9 in 2024-01-11T17:54:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:28.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.926+0000 7fd7eda1f700 1 -- 172.21.15.7:0/2414521906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8102850 msgr2=0x7fd7e8102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:28.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.926+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/2414521906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8102850 0x7fd7e8102cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0099c0 tx=0x7fd7dc009cd0 comp rx=0 tx=0).stop 2024-01-11T17:54:28.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 -- 172.21.15.7:0/2414521906 shutdown_connections 2024-01-11T17:54:28.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/2414521906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e81031f0 0x7fd7e8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:28.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/2414521906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e8101650 0x7fd7e8101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:28.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/2414521906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8102850 0x7fd7e8102cb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:28.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 -- 172.21.15.7:0/2414521906 >> 172.21.15.7:0/2414521906 conn(0x7fd7e80fcbb0 msgr2=0x7fd7e80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:54:28.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 -- 172.21.15.7:0/2414521906 shutdown_connections 2024-01-11T17:54:28.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.927+0000 7fd7eda1f700 1 -- 172.21.15.7:0/2414521906 wait complete. 2024-01-11T17:54:28.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 Processor -- start 2024-01-11T17:54:28.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 -- start start 2024-01-11T17:54:28.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 0x7fd7e8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102850 0x7fd7e8199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e81031f0 0x7fd7e81a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81a1290 con 0x7fd7e8101650 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81a13d0 con 0x7fd7e81031f0 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7eda1f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81a1560 con 0x7fd7e8102850 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.928+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 0x7fd7e8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 0x7fd7e8199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41992/0 (socket says 172.21.15.7:41992) 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 -- 172.21.15.7:0/1721024906 learned_addr learned my addr 172.21.15.7:0/1721024906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e77fe700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e81031f0 0x7fd7e81a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e67fc700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102850 0x7fd7e8199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 -- 172.21.15.7:0/1721024906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102850 msgr2=0x7fd7e8199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102850 0x7fd7e8199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 -- 172.21.15.7:0/1721024906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e81031f0 msgr2=0x7fd7e81a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e81031f0 0x7fd7e81a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 -- 172.21.15.7:0/1721024906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0095f0 con 0x7fd7e8101650 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7e6ffd700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 0x7fd7e8199700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800e9a0 tx=0x7fd7d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8009790 con 0x7fd7e8101650 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.929+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d800c810 con 0x7fd7e8101650 2024-01-11T17:54:28.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.930+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e81a1840 con 0x7fd7e8101650 2024-01-11T17:54:28.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.930+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8003c70 con 0x7fd7e8101650 2024-01-11T17:54:28.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.930+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e81a1d90 con 0x7fd7e8101650 2024-01-11T17:54:28.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.931+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e80682d0 con 0x7fd7e8101650 2024-01-11T17:54:28.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.933+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fd7d800cc80 con 0x7fd7e8101650 2024-01-11T17:54:28.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.934+0000 7fd7eca1d700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d00a1f60 0x7fd7d00a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:28.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.934+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fd7d8014070 con 0x7fd7e8101650 2024-01-11T17:54:28.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.934+0000 7fd7e67fc700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d00a1f60 0x7fd7d00a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:28.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.935+0000 7fd7e67fc700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d00a1f60 0x7fd7d00a4420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc006010 tx=0x7fd7dc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:54:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:28.937+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd7d8096c20 con 0x7fd7e8101650 2024-01-11T17:54:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:29.156+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7e8074f60 con 0x7fd7d00a1f60 2024-01-11T17:54:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:54:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:54:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] boot 2024-01-11T17:54:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: osdmap e56: 9 total, 9 up, 9 in 2024-01-11T17:54:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:29.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:29.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:29 smithi007 ceph-mon[33361]: osdmap e57: 9 total, 9 up, 9 in 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] boot 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: osdmap e56: 9 total, 9 up, 9 in 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:29 smithi016 ceph-mon[43195]: osdmap e57: 9 total, 9 up, 9 in 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: osd.8 [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] boot 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: osdmap e56: 9 total, 9 up, 9 in 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:29 smithi038 ceph-mon[43582]: osdmap e57: 9 total, 9 up, 9 in 2024-01-11T17:54:30.451 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:30 smithi007 ceph-mon[33361]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:30.451 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:30 smithi007 ceph-mon[33361]: osdmap e58: 9 total, 9 up, 9 in 2024-01-11T17:54:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:30 smithi016 ceph-mon[43195]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:30 smithi016 ceph-mon[43195]: osdmap e58: 9 total, 9 up, 9 in 2024-01-11T17:54:30.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:30 smithi038 ceph-mon[43582]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:30.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:30 smithi038 ceph-mon[43582]: osdmap e58: 9 total, 9 up, 9 in 2024-01-11T17:54:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:31 smithi016 ceph-mon[43195]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:31 smithi038 ceph-mon[43582]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:31 smithi007 ceph-mon[33361]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:32 smithi016 ceph-mon[43195]: osdmap e59: 9 total, 9 up, 9 in 2024-01-11T17:54:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1033496334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb7e8378-c57e-4c86-9184-fe68617c8169"}]: dispatch 2024-01-11T17:54:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1033496334' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb7e8378-c57e-4c86-9184-fe68617c8169"}]': finished 2024-01-11T17:54:32.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:32 smithi016 ceph-mon[43195]: osdmap e60: 10 total, 9 up, 10 in 2024-01-11T17:54:32.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:32 smithi038 ceph-mon[43582]: osdmap e59: 9 total, 9 up, 9 in 2024-01-11T17:54:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1033496334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb7e8378-c57e-4c86-9184-fe68617c8169"}]: dispatch 2024-01-11T17:54:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1033496334' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb7e8378-c57e-4c86-9184-fe68617c8169"}]': finished 2024-01-11T17:54:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:32 smithi038 ceph-mon[43582]: osdmap e60: 10 total, 9 up, 10 in 2024-01-11T17:54:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:32 smithi007 ceph-mon[33361]: osdmap e59: 9 total, 9 up, 9 in 2024-01-11T17:54:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1033496334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb7e8378-c57e-4c86-9184-fe68617c8169"}]: dispatch 2024-01-11T17:54:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1033496334' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb7e8378-c57e-4c86-9184-fe68617c8169"}]': finished 2024-01-11T17:54:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:32 smithi007 ceph-mon[33361]: osdmap e60: 10 total, 9 up, 10 in 2024-01-11T17:54:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:33.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:33 smithi016 ceph-mon[43195]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:33.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:33 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/317166357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:33 smithi038 ceph-mon[43582]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:33 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/317166357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:33 smithi007 ceph-mon[33361]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:33 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/317166357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:35 smithi007 ceph-mon[33361]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:35.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:35 smithi016 ceph-mon[43195]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:35.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:35 smithi038 ceph-mon[43582]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:37 smithi016 ceph-mon[43195]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-11T17:54:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:37 smithi038 ceph-mon[43582]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-11T17:54:38.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:37 smithi007 ceph-mon[33361]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-11T17:54:38.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-11T17:54:38.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-11T17:54:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:39.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-11T17:54:39.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:39 smithi016 ceph-mon[43195]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-11T17:54:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:39 smithi016 ceph-mon[43195]: Deploying daemon osd.9 on smithi007 2024-01-11T17:54:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:39 smithi038 ceph-mon[43582]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-11T17:54:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:39 smithi038 ceph-mon[43582]: Deploying daemon osd.9 on smithi007 2024-01-11T17:54:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:39 smithi007 ceph-mon[33361]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-11T17:54:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:39 smithi007 ceph-mon[33361]: Deploying daemon osd.9 on smithi007 2024-01-11T17:54:41.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:41 smithi007 ceph-mon[33361]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-01-11T17:54:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:41 smithi016 ceph-mon[43195]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-01-11T17:54:42.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:41 smithi038 ceph-mon[43582]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-01-11T17:54:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:43 smithi016 ceph-mon[43195]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-01-11T17:54:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:43 smithi038 ceph-mon[43582]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-01-11T17:54:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:43 smithi007 ceph-mon[33361]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-01-11T17:54:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:45.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:44 smithi007 ceph-mon[33361]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:54:45.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:45.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:45.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:45.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:44 smithi016 ceph-mon[43195]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:54:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:44 smithi038 ceph-mon[43582]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:54:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.856+0000 7fd7eca1d700 1 -- 172.21.15.7:0/1721024906 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd7e8074f60 con 0x7fd7d00a1f60 2024-01-11T17:54:46.858 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-01-11T17:54:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d00a1f60 msgr2=0x7fd7d00a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d00a1f60 0x7fd7d00a4420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc006010 tx=0x7fd7dc00b540 comp rx=0 tx=0).stop 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 msgr2=0x7fd7e8199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 0x7fd7e8199700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800e9a0 tx=0x7fd7d800ecb0 comp rx=0 tx=0).stop 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 shutdown_connections 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd7d00a1f60 0x7fd7d00a4420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e81031f0 0x7fd7e81a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7e8101650 0x7fd7e8199700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 --2- 172.21.15.7:0/1721024906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102850 0x7fd7e8199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 >> 172.21.15.7:0/1721024906 conn(0x7fd7e80fcbb0 msgr2=0x7fd7e8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 shutdown_connections 2024-01-11T17:54:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:54:46.858+0000 7fd7eda1f700 1 -- 172.21.15.7:0/1721024906 wait complete. 2024-01-11T17:54:47.319 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.9.service 2024-01-11T17:54:47.324 INFO:tasks.cephadm:Deploying osd.10 on smithi016 with /dev/vg_nvme/lv_1... 2024-01-11T17:54:47.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:47.403 INFO:journalctl@ceph.osd.9.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:54:47.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:47.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:47.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:54:47.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:47.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:54:47.625 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:54:48.023 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 17:54:47 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T17:54:47.900+0000 7f6444327640 -1 osd.9 0 log_to_monitors true 2024-01-11T17:54:48.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:48 smithi007 ceph-mon[33361]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-11T17:54:48.452 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:48 smithi016 ceph-mon[43195]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-11T17:54:48.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:48 smithi038 ceph-mon[43582]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-11T17:54:49.031 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-11T17:54:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-11T17:54:49.032 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-01-11T17:54:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-01-11T17:54:49.032 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00923554 s, 1.1 GB/s 2024-01-11T17:54:49.032 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-01-11T17:54:49.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:49 smithi016 ceph-mon[43195]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-11T17:54:49.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:49 smithi016 ceph-mon[43195]: osdmap e61: 10 total, 9 up, 10 in 2024-01-11T17:54:49.362 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:49 smithi016 ceph-mon[43195]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:54:49.362 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:49.362 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:49 smithi016 ceph-mon[43195]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:49.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-01-11T17:54:49.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:49 smithi038 ceph-mon[43582]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-11T17:54:49.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:49 smithi038 ceph-mon[43582]: osdmap e61: 10 total, 9 up, 10 in 2024-01-11T17:54:49.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:49 smithi038 ceph-mon[43582]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:54:49.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:49.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:49 smithi038 ceph-mon[43582]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:49.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-mon[33361]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-11T17:54:49.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-mon[33361]: osdmap e61: 10 total, 9 up, 10 in 2024-01-11T17:54:49.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-mon[33361]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-01-11T17:54:49.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:49.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-mon[33361]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:49.572 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T17:54:49.188+0000 7f643998d700 -1 osd.9 0 waiting for initial osdmap 2024-01-11T17:54:49.572 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 17:54:49 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T17:54:49.203+0000 7f6435787700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:54:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T17:54:50.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:50 smithi007 ceph-mon[33361]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:54:50.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:50 smithi007 ceph-mon[33361]: osdmap e62: 10 total, 9 up, 10 in 2024-01-11T17:54:50.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:50.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:50.428 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:50 smithi016 ceph-mon[43195]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:54:50.428 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:50 smithi016 ceph-mon[43195]: osdmap e62: 10 total, 9 up, 10 in 2024-01-11T17:54:50.428 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:50.428 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:50.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:50 smithi038 ceph-mon[43582]: from='osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-01-11T17:54:50.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:50 smithi038 ceph-mon[43582]: osdmap e62: 10 total, 9 up, 10 in 2024-01-11T17:54:50.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:50.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 -- 172.21.15.16:0/182369857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c102880 msgr2=0x7f361c102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 --2- 172.21.15.16:0/182369857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c102880 0x7f361c102ce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f36040099c0 tx=0x7f3604009cd0 comp rx=0 tx=0).stop 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 -- 172.21.15.16:0/182369857 shutdown_connections 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 --2- 172.21.15.16:0/182369857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c101680 0x7f361c101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 --2- 172.21.15.16:0/182369857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c103220 0x7f361c109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 --2- 172.21.15.16:0/182369857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c102880 0x7f361c102ce0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.819+0000 7f3622a5c700 1 -- 172.21.15.16:0/182369857 >> 172.21.15.16:0/182369857 conn(0x7f361c0fcc60 msgr2=0x7f361c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.820+0000 7f3622a5c700 1 -- 172.21.15.16:0/182369857 shutdown_connections 2024-01-11T17:54:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.820+0000 7f3622a5c700 1 -- 172.21.15.16:0/182369857 wait complete. 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.820+0000 7f3622a5c700 1 Processor -- start 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.820+0000 7f3622a5c700 1 -- start start 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3622a5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 0x7f361c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3622a5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c102880 0x7f361c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3622a5c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c103220 0x7f361c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3622a5c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c19f030 con 0x7f361c102880 2024-01-11T17:54:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3622a5c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c19f170 con 0x7f361c101680 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3622a5c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c19f300 con 0x7f361c103220 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 0x7f361c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 0x7f361c1974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40110/0 (socket says 172.21.15.16:40110) 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 -- 172.21.15.16:0/3660469481 learned_addr learned my addr 172.21.15.16:0/3660469481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361b7fe700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c102880 0x7f361c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3620ff9700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c103220 0x7f361c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 -- 172.21.15.16:0/3660469481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c103220 msgr2=0x7f361c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c103220 0x7f361c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 -- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c102880 msgr2=0x7f361c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c102880 0x7f361c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361bfff700 1 -- 172.21.15.16:0/3660469481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040095f0 con 0x7f361c101680 2024-01-11T17:54:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f361b7fe700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c102880 0x7f361c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.821+0000 7f3620ff9700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c103220 0x7f361c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.822+0000 7f361bfff700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 0x7f361c1974a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f360c00e970 tx=0x7f360c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.822+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c009790 con 0x7f361c101680 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.822+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f360c0044e0 con 0x7f361c101680 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.822+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c19f5e0 con 0x7f361c101680 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.822+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c003e30 con 0x7f361c101680 2024-01-11T17:54:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.822+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c19fb30 con 0x7f361c101680 2024-01-11T17:54:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.823+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c108a40 con 0x7f361c101680 2024-01-11T17:54:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.824+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f360c003f90 con 0x7f361c101680 2024-01-11T17:54:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.825+0000 7f36197fa700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36080a1f60 0x7f36080a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:54:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.825+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f360c014070 con 0x7f361c101680 2024-01-11T17:54:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.825+0000 7f361b7fe700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36080a1f60 0x7f36080a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:54:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.826+0000 7f361b7fe700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36080a1f60 0x7f36080a4420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f360400b580 tx=0x7f360401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:54:50.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:50.830+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f360c096c80 con 0x7f361c101680 2024-01-11T17:54:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:54:51.047+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f361c02d500 con 0x7f36080a1f60 2024-01-11T17:54:51.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:54:51.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:54:51.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] boot 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: osdmap e63: 10 total, 10 up, 10 in 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi007 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi007 to 4319M 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:51.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:51.483 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:51 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:51.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:54:51.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:54:51.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] boot 2024-01-11T17:54:51.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: osdmap e63: 10 total, 10 up, 10 in 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi007 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi007 to 4319M 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:51.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:51.528 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:51.528 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:51 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:51.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:54:51.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:54:51.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: osd.9 [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] boot 2024-01-11T17:54:51.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: osdmap e63: 10 total, 10 up, 10 in 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi007 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi007 to 4319M 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:54:51.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:54:51.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:51 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:54:52.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:52 smithi016 ceph-mon[43195]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:52.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:52 smithi016 ceph-mon[43195]: osdmap e64: 10 total, 10 up, 10 in 2024-01-11T17:54:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:52 smithi038 ceph-mon[43582]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:52 smithi038 ceph-mon[43582]: osdmap e64: 10 total, 10 up, 10 in 2024-01-11T17:54:52.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:52 smithi007 ceph-mon[33361]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:54:52.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:52 smithi007 ceph-mon[33361]: osdmap e64: 10 total, 10 up, 10 in 2024-01-11T17:54:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:53 smithi016 ceph-mon[43195]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/4002709469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]: dispatch 2024-01-11T17:54:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]: dispatch 2024-01-11T17:54:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:53 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]': finished 2024-01-11T17:54:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:53 smithi016 ceph-mon[43195]: osdmap e65: 11 total, 10 up, 11 in 2024-01-11T17:54:53.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:54:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:53 smithi038 ceph-mon[43582]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/4002709469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]: dispatch 2024-01-11T17:54:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]: dispatch 2024-01-11T17:54:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:53 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]': finished 2024-01-11T17:54:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:53 smithi038 ceph-mon[43582]: osdmap e65: 11 total, 10 up, 11 in 2024-01-11T17:54:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:54:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:53 smithi007 ceph-mon[33361]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/4002709469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]: dispatch 2024-01-11T17:54:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]: dispatch 2024-01-11T17:54:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:53 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf5a2cb2-377f-4650-9971-178ba7b1b176"}]': finished 2024-01-11T17:54:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:53 smithi007 ceph-mon[33361]: osdmap e65: 11 total, 10 up, 11 in 2024-01-11T17:54:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:54:54.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:54 smithi016 ceph-mon[43195]: from='client.? 172.21.15.16:0/1664014401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:54.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:54 smithi038 ceph-mon[43582]: from='client.? 172.21.15.16:0/1664014401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:54.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:54 smithi007 ceph-mon[33361]: from='client.? 172.21.15.16:0/1664014401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:54:55.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:55 smithi038 ceph-mon[43582]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:55.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:55 smithi007 ceph-mon[33361]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:55.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:55 smithi016 ceph-mon[43195]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:57.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:57 smithi016 ceph-mon[43195]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:58.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:57 smithi038 ceph-mon[43582]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:57 smithi007 ceph-mon[33361]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:59 smithi016 ceph-mon[43195]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:54:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-11T17:54:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:54:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:59 smithi038 ceph-mon[43582]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-11T17:55:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:54:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:00.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:59 smithi007 ceph-mon[33361]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:00.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-11T17:55:00.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:54:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:00.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:00 smithi016 ceph-mon[43195]: Deploying daemon osd.10 on smithi016 2024-01-11T17:55:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:00 smithi038 ceph-mon[43582]: Deploying daemon osd.10 on smithi016 2024-01-11T17:55:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:00 smithi007 ceph-mon[33361]: Deploying daemon osd.10 on smithi016 2024-01-11T17:55:01.888 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:01 smithi016 ceph-mon[43195]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:01 smithi038 ceph-mon[43582]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:01 smithi007 ceph-mon[33361]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:03.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:03 smithi016 ceph-mon[43195]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:03.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:03.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:03.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:03 smithi038 ceph-mon[43582]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:03 smithi007 ceph-mon[33361]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:05 smithi016 ceph-mon[43195]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:05.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:05 smithi007 ceph-mon[33361]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.656+0000 7f36197fa700 1 -- 172.21.15.16:0/3660469481 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f361c02d500 con 0x7f36080a1f60 2024-01-11T17:55:05.657 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 10 on host 'smithi016' 2024-01-11T17:55:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36080a1f60 msgr2=0x7f36080a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36080a1f60 0x7f36080a4420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f360400b580 tx=0x7f360401a040 comp rx=0 tx=0).stop 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 msgr2=0x7f361c1974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 0x7f361c1974a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f360c00e970 tx=0x7f360c00ed30 comp rx=0 tx=0).stop 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 shutdown_connections 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36080a1f60 0x7f36080a4420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f361c103220 0x7f361c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c101680 0x7f361c1974a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 --2- 172.21.15.16:0/3660469481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c102880 0x7f361c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 >> 172.21.15.16:0/3660469481 conn(0x7f361c0fcc60 msgr2=0x7f361c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 shutdown_connections 2024-01-11T17:55:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-01-11T17:55:05.659+0000 7f3622a5c700 1 -- 172.21.15.16:0/3660469481 wait complete. 2024-01-11T17:55:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:05 smithi038 ceph-mon[43582]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:06.067 DEBUG:teuthology.orchestra.run.smithi016:osd.10> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.10.service 2024-01-11T17:55:06.069 INFO:tasks.cephadm:Deploying osd.11 on smithi038 with /dev/vg_nvme/lv_1... 2024-01-11T17:55:06.069 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-11T17:55:06.104 INFO:journalctl@ceph.osd.10.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:55:06.374 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:55:06.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:06.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:06.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:06 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:07.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:07.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:07.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:06 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:06 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:07.238 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 17:55:07 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T17:55:07.234+0000 7f6300b8e640 -1 osd.10 0 log_to_monitors true 2024-01-11T17:55:07.872 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-11T17:55:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-11T17:55:07.873 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-01-11T17:55:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-01-11T17:55:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0082407 s, 1.3 GB/s 2024-01-11T17:55:07.873 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 2024-01-11T17:55:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-01-11T17:55:07.914 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:07 smithi038 ceph-mon[43582]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:07.914 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:07 smithi038 ceph-mon[43582]: from='osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-11T17:55:07.915 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:07 smithi038 ceph-mon[43582]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-11T17:55:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:07 smithi016 ceph-mon[43195]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:07 smithi016 ceph-mon[43195]: from='osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-11T17:55:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:07 smithi016 ceph-mon[43195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-11T17:55:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:07 smithi007 ceph-mon[33361]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:07 smithi007 ceph-mon[33361]: from='osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-11T17:55:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:07 smithi007 ceph-mon[33361]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-11T17:55:08.287 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-01-11T17:55:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:55:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-11T17:55:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: osdmap e66: 11 total, 10 up, 11 in 2024-01-11T17:55:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:55:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-11T17:55:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: osdmap e66: 11 total, 10 up, 11 in 2024-01-11T17:55:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-11T17:55:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: osdmap e66: 11 total, 10 up, 11 in 2024-01-11T17:55:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:09.496 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T17:55:09.197+0000 7f62f61f4700 -1 osd.10 0 waiting for initial osdmap 2024-01-11T17:55:09.496 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T17:55:09.213+0000 7f62edfe6700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.636+0000 7efcc66e1700 1 -- 172.21.15.38:0/4228017329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 msgr2=0x7efcc0100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.636+0000 7efcc66e1700 1 --2- 172.21.15.38:0/4228017329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc0100960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efca8009990 tx=0x7efca8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 -- 172.21.15.38:0/4228017329 shutdown_connections 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 --2- 172.21.15.38:0/4228017329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc0100960 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 --2- 172.21.15.38:0/4228017329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc01020e0 0x7efcc0108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 --2- 172.21.15.38:0/4228017329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc0101740 0x7efcc0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:09.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 -- 172.21.15.38:0/4228017329 >> 172.21.15.38:0/4228017329 conn(0x7efcc00fbae0 msgr2=0x7efcc00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 -- 172.21.15.38:0/4228017329 shutdown_connections 2024-01-11T17:55:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 -- 172.21.15.38:0/4228017329 wait complete. 2024-01-11T17:55:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 Processor -- start 2024-01-11T17:55:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.637+0000 7efcc66e1700 1 -- start start 2024-01-11T17:55:09.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcc66e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:09.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcc66e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc0101740 0x7efcc0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcc66e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc01020e0 0x7efcc01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcc66e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc01a1360 con 0x7efcc01020e0 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcc66e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc01a14a0 con 0x7efcc0101740 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcc66e1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc01a1630 con 0x7efcc0100540 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40604/0 (socket says 172.21.15.38:40604) 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 -- 172.21.15.38:0/2223813062 learned_addr learned my addr 172.21.15.38:0/2223813062 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbf7fe700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc0101740 0x7efcc0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 -- 172.21.15.38:0/2223813062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc0101740 msgr2=0x7efcc0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc0101740 0x7efcc0199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:09.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 -- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc01020e0 msgr2=0x7efcc01a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc01020e0 0x7efcc01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 -- 172.21.15.38:0/2223813062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80095f0 con 0x7efcc0100540 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.638+0000 7efcbffff700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc01997d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efca8000c00 tx=0x7efca800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.639+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca801d070 con 0x7efcc0100540 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.639+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efca8022470 con 0x7efcc0100540 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.639+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc01a18b0 con 0x7efcc0100540 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.639+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8017650 con 0x7efcc0100540 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.639+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc01a1d40 con 0x7efcc0100540 2024-01-11T17:55:09.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.640+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc00682d0 con 0x7efcc0100540 2024-01-11T17:55:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.641+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7efca80225e0 con 0x7efcc0100540 2024-01-11T17:55:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.642+0000 7efcbd7fa700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efcac0a1ee0 0x7efcac0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.642+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6727+0+0 (secure 0 0 0) 0x7efca80ce7d0 con 0x7efcc0100540 2024-01-11T17:55:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.642+0000 7efcbf7fe700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efcac0a1ee0 0x7efcac0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.643+0000 7efcbf7fe700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efcac0a1ee0 0x7efcac0a43a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005950 tx=0x7efcb000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:09.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.646+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7efca8097e90 con 0x7efcc0100540 2024-01-11T17:55:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi016 2024-01-11T17:55:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi016 to 4279M 2024-01-11T17:55:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: osdmap e67: 11 total, 10 up, 11 in 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: from='osd.10 ' entity='osd.10' 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] boot 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: osdmap e68: 11 total, 11 up, 11 in 2024-01-11T17:55:09.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:09.869+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efcc0069a00 con 0x7efcac0a1ee0 2024-01-11T17:55:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi016 2024-01-11T17:55:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi016 to 4279M 2024-01-11T17:55:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:55:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: osdmap e67: 11 total, 10 up, 11 in 2024-01-11T17:55:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: from='osd.10 ' entity='osd.10' 2024-01-11T17:55:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] boot 2024-01-11T17:55:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: osdmap e68: 11 total, 11 up, 11 in 2024-01-11T17:55:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi016 2024-01-11T17:55:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi016 to 4279M 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: osdmap e67: 11 total, 10 up, 11 in 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: from='osd.10 ' entity='osd.10' 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: osd.10 [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] boot 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: osdmap e68: 11 total, 11 up, 11 in 2024-01-11T17:55:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-11T17:55:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:10 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:55:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:10 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:55:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:55:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:55:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:10 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:10.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:10 smithi016 ceph-mon[43195]: osdmap e69: 11 total, 11 up, 11 in 2024-01-11T17:55:11.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:10 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:55:11.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:10 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:55:11.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:55:11.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:55:11.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:10 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:11.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:10 smithi038 ceph-mon[43582]: osdmap e69: 11 total, 11 up, 11 in 2024-01-11T17:55:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:10 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:55:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:10 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:55:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-11T17:55:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-11T17:55:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:10 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:10 smithi007 ceph-mon[33361]: osdmap e69: 11 total, 11 up, 11 in 2024-01-11T17:55:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:11 smithi016 ceph-mon[43195]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:55:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:11 smithi016 ceph-mon[43195]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:11 smithi016 ceph-mon[43195]: osdmap e70: 11 total, 11 up, 11 in 2024-01-11T17:55:12.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:11 smithi038 ceph-mon[43582]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:55:12.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:11 smithi038 ceph-mon[43582]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:12.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:11 smithi038 ceph-mon[43582]: osdmap e70: 11 total, 11 up, 11 in 2024-01-11T17:55:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:11 smithi007 ceph-mon[33361]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:55:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:11 smithi007 ceph-mon[33361]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:11 smithi007 ceph-mon[33361]: osdmap e70: 11 total, 11 up, 11 in 2024-01-11T17:55:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/1937993732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]: dispatch 2024-01-11T17:55:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:12 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]: dispatch 2024-01-11T17:55:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:12 smithi016 ceph-mon[43195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]': finished 2024-01-11T17:55:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:12 smithi016 ceph-mon[43195]: osdmap e71: 12 total, 11 up, 12 in 2024-01-11T17:55:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:12 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.38:0/619588824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:55:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/1937993732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]: dispatch 2024-01-11T17:55:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:12 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]: dispatch 2024-01-11T17:55:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:12 smithi038 ceph-mon[43582]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]': finished 2024-01-11T17:55:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:12 smithi038 ceph-mon[43582]: osdmap e71: 12 total, 11 up, 12 in 2024-01-11T17:55:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:12 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:13.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.38:0/619588824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:55:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/1937993732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]: dispatch 2024-01-11T17:55:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:12 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]: dispatch 2024-01-11T17:55:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:12 smithi007 ceph-mon[33361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc65b78-877e-4fc7-8aef-6d5d8af09353"}]': finished 2024-01-11T17:55:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:12 smithi007 ceph-mon[33361]: osdmap e71: 12 total, 11 up, 12 in 2024-01-11T17:55:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:12 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:13.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.38:0/619588824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-11T17:55:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:13 smithi016 ceph-mon[43195]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:13 smithi016 ceph-mon[43195]: osdmap e72: 12 total, 11 up, 12 in 2024-01-11T17:55:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:13 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:13 smithi038 ceph-mon[43582]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:13 smithi038 ceph-mon[43582]: osdmap e72: 12 total, 11 up, 12 in 2024-01-11T17:55:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:13 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:13 smithi007 ceph-mon[33361]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:13 smithi007 ceph-mon[33361]: osdmap e72: 12 total, 11 up, 12 in 2024-01-11T17:55:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:13 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:15.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:15 smithi007 ceph-mon[33361]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:15.609 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:15 smithi038 ceph-mon[43582]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:15 smithi016 ceph-mon[43195]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:17.937 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:17 smithi038 ceph-mon[43582]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-11T17:55:17.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:17 smithi016 ceph-mon[43195]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-11T17:55:18.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:17 smithi007 ceph-mon[33361]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-11T17:55:18.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-11T17:55:18.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-11T17:55:18.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:19.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-11T17:55:19.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:19.961 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:19 smithi038 ceph-mon[43582]: Deploying daemon osd.11 on smithi038 2024-01-11T17:55:19.961 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:19 smithi038 ceph-mon[43582]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-01-11T17:55:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:19 smithi016 ceph-mon[43195]: Deploying daemon osd.11 on smithi038 2024-01-11T17:55:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:19 smithi016 ceph-mon[43195]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-01-11T17:55:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:19 smithi007 ceph-mon[33361]: Deploying daemon osd.11 on smithi038 2024-01-11T17:55:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:19 smithi007 ceph-mon[33361]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-01-11T17:55:21.976 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:21 smithi038 ceph-mon[43582]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-01-11T17:55:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:21 smithi016 ceph-mon[43195]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-01-11T17:55:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:21 smithi007 ceph-mon[33361]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-01-11T17:55:23.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:23.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:23.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:23.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:23 smithi038 ceph-mon[43582]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-11T17:55:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:23.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:23 smithi016 ceph-mon[43195]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-11T17:55:23.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:23.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:23.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:23.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:23 smithi007 ceph-mon[33361]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-11T17:55:24.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.434+0000 7efcbd7fa700 1 -- 172.21.15.38:0/2223813062 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7efcc0069a00 con 0x7efcac0a1ee0 2024-01-11T17:55:24.436 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 11 on host 'smithi038' 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efcac0a1ee0 msgr2=0x7efcac0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efcac0a1ee0 0x7efcac0a43a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005950 tx=0x7efcb000a300 comp rx=0 tx=0).stop 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 msgr2=0x7efcc01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc01997d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efca8000c00 tx=0x7efca800f690 comp rx=0 tx=0).stop 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 shutdown_connections 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efcac0a1ee0 0x7efcac0a43a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc0100540 0x7efcc01997d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc01020e0 0x7efcc01a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 --2- 172.21.15.38:0/2223813062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc0101740 0x7efcc0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:24.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 >> 172.21.15.38:0/2223813062 conn(0x7efcc00fbae0 msgr2=0x7efcc00692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:24.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 shutdown_connections 2024-01-11T17:55:24.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:24.437+0000 7efcc66e1700 1 -- 172.21.15.38:0/2223813062 wait complete. 2024-01-11T17:55:24.870 DEBUG:teuthology.orchestra.run.smithi038:osd.11> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.11.service 2024-01-11T17:55:24.872 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-01-11T17:55:24.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd stat -f json 2024-01-11T17:55:24.905 INFO:journalctl@ceph.osd.11.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:55:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:25.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:25.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:24 smithi016 ceph-mon[43195]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-01-11T17:55:25.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:25.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:25.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:25.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:24 smithi038 ceph-mon[43582]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-01-11T17:55:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:55:25.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:25.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:25.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:25.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:25.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:24 smithi007 ceph-mon[33361]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-01-11T17:55:26.401 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 17:55:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T17:55:26.146+0000 7fe4f0fbb640 -1 osd.11 0 log_to_monitors true 2024-01-11T17:55:26.480 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:26 smithi007 ceph-mon[33361]: from='osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-11T17:55:26.480 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:26 smithi007 ceph-mon[33361]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-11T17:55:26.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.686+0000 7f8597b73700 1 -- 172.21.15.7:0/583167070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85901020d0 msgr2=0x7f85901044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.686+0000 7f8597b73700 1 --2- 172.21.15.7:0/583167070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85901020d0 0x7f85901044c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f858c009990 tx=0x7f858c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 -- 172.21.15.7:0/583167070 shutdown_connections 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 --2- 172.21.15.7:0/583167070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8590107330 0x7f85901097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 --2- 172.21.15.7:0/583167070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85901020d0 0x7f85901044c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 --2- 172.21.15.7:0/583167070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8590104a00 0x7f8590106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 -- 172.21.15.7:0/583167070 >> 172.21.15.7:0/583167070 conn(0x7f85900fbae0 msgr2=0x7f85900fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 -- 172.21.15.7:0/583167070 shutdown_connections 2024-01-11T17:55:26.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.687+0000 7f8597b73700 1 -- 172.21.15.7:0/583167070 wait complete. 2024-01-11T17:55:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 Processor -- start 2024-01-11T17:55:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 -- start start 2024-01-11T17:55:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 0x7f8590199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:26.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8590104a00 0x7f8590199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8590107330 0x7f85901a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85901a1300 con 0x7f8590107330 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85901a1440 con 0x7f8590104a00 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f8597b73700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85901a15d0 con 0x7f85901020d0 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f859590f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 0x7f8590199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.688+0000 7f859590f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 0x7f8590199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:51050/0 (socket says 172.21.15.7:51050) 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 -- 172.21.15.7:0/140183824 learned_addr learned my addr 172.21.15.7:0/140183824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f8596110700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8590107330 0x7f85901a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859510e700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8590104a00 0x7f8590199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 -- 172.21.15.7:0/140183824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8590104a00 msgr2=0x7f8590199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8590104a00 0x7f8590199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 -- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8590107330 msgr2=0x7f85901a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8590107330 0x7f85901a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 -- 172.21.15.7:0/140183824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0095f0 con 0x7f85901020d0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f859590f700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 0x7f8590199770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f858c000c00 tx=0x7f858c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c01d070 con 0x7f85901020d0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f858c021410 con 0x7f85901020d0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85901a1850 con 0x7f85901020d0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.689+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c017650 con 0x7f85901020d0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.690+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85901a1c60 con 0x7f85901020d0 2024-01-11T17:55:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.691+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85900682d0 con 0x7f85901020d0 2024-01-11T17:55:26.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.693+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f858c021580 con 0x7f85901020d0 2024-01-11T17:55:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.694+0000 7f8586ffd700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f857c0a1ee0 0x7f857c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:26.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.694+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f858c0ce9e0 con 0x7f85901020d0 2024-01-11T17:55:26.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.694+0000 7f859510e700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f857c0a1ee0 0x7f857c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:26.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.695+0000 7f859510e700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f857c0a1ee0 0x7f857c0a43a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8580009bf0 tx=0x7f8580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:26.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.697+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f858c098040 con 0x7f85901020d0 2024-01-11T17:55:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:26 smithi016 ceph-mon[43195]: from='osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-11T17:55:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:26 smithi016 ceph-mon[43195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-11T17:55:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:26 smithi038 ceph-mon[43582]: from='osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-11T17:55:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:26 smithi038 ceph-mon[43582]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-11T17:55:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.911+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f859010cf90 con 0x7f85901020d0 2024-01-11T17:55:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.912+0000 7f8586ffd700 1 -- 172.21.15.7:0/140183824 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f858c0bb090 con 0x7f85901020d0 2024-01-11T17:55:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.914+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f857c0a1ee0 msgr2=0x7f857c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.914+0000 7f8597b73700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f857c0a1ee0 0x7f857c0a43a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8580009bf0 tx=0x7f8580009260 comp rx=0 tx=0).stop 2024-01-11T17:55:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 msgr2=0x7f8590199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 0x7f8590199770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f858c000c00 tx=0x7f858c009580 comp rx=0 tx=0).stop 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 shutdown_connections 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f857c0a1ee0 0x7f857c0a43a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85901020d0 0x7f8590199770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8590107330 0x7f85901a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 --2- 172.21.15.7:0/140183824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8590104a00 0x7f8590199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 >> 172.21.15.7:0/140183824 conn(0x7f85900fbae0 msgr2=0x7f85901017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 shutdown_connections 2024-01-11T17:55:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:26.915+0000 7f8597b73700 1 -- 172.21.15.7:0/140183824 wait complete. 2024-01-11T17:55:27.314 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1704995709,"num_in_osds":12,"osd_in_since":1704995711,"num_remapped_pgs":0} 2024-01-11T17:55:27.429 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-11T17:55:27.429 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: osdmap e73: 12 total, 11 up, 12 in 2024-01-11T17:55:27.429 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:27.429 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: from='osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:55:27.429 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:55:27.430 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:27.430 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/140183824' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-11T17:55:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-11T17:55:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: osdmap e73: 12 total, 11 up, 12 in 2024-01-11T17:55:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: from='osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:55:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:55:27.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:27.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:27 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/140183824' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-11T17:55:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-11T17:55:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: osdmap e73: 12 total, 11 up, 12 in 2024-01-11T17:55:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: from='osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:55:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-01-11T17:55:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:27.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:27 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/140183824' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-11T17:55:28.276 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T17:55:27.926+0000 7fe4e6e22700 -1 osd.11 0 waiting for initial osdmap 2024-01-11T17:55:28.276 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 17:55:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T17:55:27.943+0000 7fe4de413700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T17:55:28.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd stat -f json 2024-01-11T17:55:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: osdmap e74: 12 total, 11 up, 12 in 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: Detected new or changed devices on smithi038 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi038 to 4279M 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:28 smithi016 ceph-mon[43195]: from='osd.11 ' entity='osd.11' 2024-01-11T17:55:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: osdmap e74: 12 total, 11 up, 12 in 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: Detected new or changed devices on smithi038 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi038 to 4279M 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:28.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.778 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:28 smithi038 ceph-mon[43582]: from='osd.11 ' entity='osd.11' 2024-01-11T17:55:28.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: osdmap e74: 12 total, 11 up, 12 in 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: Detected new or changed devices on smithi038 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi038 to 4279M 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:28.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:28.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:28.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:28 smithi007 ceph-mon[33361]: from='osd.11 ' entity='osd.11' 2024-01-11T17:55:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: purged_snaps scrub starts 2024-01-11T17:55:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: purged_snaps scrub ok 2024-01-11T17:55:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] boot 2024-01-11T17:55:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: osdmap e75: 12 total, 12 up, 12 in 2024-01-11T17:55:29.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: purged_snaps scrub starts 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: purged_snaps scrub ok 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] boot 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: osdmap e75: 12 total, 12 up, 12 in 2024-01-11T17:55:29.760 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: purged_snaps scrub starts 2024-01-11T17:55:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: purged_snaps scrub ok 2024-01-11T17:55:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-01-11T17:55:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: osd.11 [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] boot 2024-01-11T17:55:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: osdmap e75: 12 total, 12 up, 12 in 2024-01-11T17:55:29.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-11T17:55:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 -- 172.21.15.7:0/1408533706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 msgr2=0x7f35ac102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 --2- 172.21.15.7:0/1408533706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac102cb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f359c0099c0 tx=0x7f359c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 -- 172.21.15.7:0/1408533706 shutdown_connections 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 --2- 172.21.15.7:0/1408533706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac1031f0 0x7f35ac109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 --2- 172.21.15.7:0/1408533706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac101650 0x7f35ac101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 --2- 172.21.15.7:0/1408533706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac102cb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 -- 172.21.15.7:0/1408533706 >> 172.21.15.7:0/1408533706 conn(0x7f35ac0fcbb0 msgr2=0x7f35ac0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 -- 172.21.15.7:0/1408533706 shutdown_connections 2024-01-11T17:55:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.001+0000 7f35b4712700 1 -- 172.21.15.7:0/1408533706 wait complete. 2024-01-11T17:55:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.002+0000 7f35b4712700 1 Processor -- start 2024-01-11T17:55:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 -- start start 2024-01-11T17:55:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101650 0x7f35ac199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac1031f0 0x7f35ac1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1a12c0 con 0x7f35ac102850 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1a1400 con 0x7f35ac1031f0 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b4712700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1a1590 con 0x7f35ac101650 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b1cad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b2caf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac1031f0 0x7f35ac1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b1cad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33548/0 (socket says 172.21.15.7:33548) 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b1cad700 1 -- 172.21.15.7:0/4175237160 learned_addr learned my addr 172.21.15.7:0/4175237160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.003+0000 7f35b24ae700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101650 0x7f35ac199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b1cad700 1 -- 172.21.15.7:0/4175237160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101650 msgr2=0x7f35ac199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b1cad700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101650 0x7f35ac199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b1cad700 1 -- 172.21.15.7:0/4175237160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac1031f0 msgr2=0x7f35ac1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b1cad700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac1031f0 0x7f35ac1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b1cad700 1 -- 172.21.15.7:0/4175237160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0095f0 con 0x7f35ac102850 2024-01-11T17:55:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b2caf700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac1031f0 0x7f35ac1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:55:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b1cad700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac199c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f359c009fd0 tx=0x7f359c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c01d070 con 0x7f35ac102850 2024-01-11T17:55:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c005050 con 0x7f35ac102850 2024-01-11T17:55:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac1a1810 con 0x7f35ac102850 2024-01-11T17:55:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.004+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c004a50 con 0x7f35ac102850 2024-01-11T17:55:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.005+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac1a1c80 con 0x7f35ac102850 2024-01-11T17:55:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.006+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac0682d0 con 0x7f35ac102850 2024-01-11T17:55:30.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.009+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f359c0051c0 con 0x7f35ac102850 2024-01-11T17:55:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.010+0000 7f35a37fe700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35980a1f60 0x7f35980a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.011+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f359c0ce650 con 0x7f35ac102850 2024-01-11T17:55:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.011+0000 7f35b24ae700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35980a1f60 0x7f35980a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.011+0000 7f35b24ae700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35980a1f60 0x7f35980a4420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35ac1026b0 tx=0x7f35a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:30.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.013+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f359c097b80 con 0x7f35ac102850 2024-01-11T17:55:30.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.224+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f35ac10d070 con 0x7f35ac102850 2024-01-11T17:55:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.225+0000 7f35a37fe700 1 -- 172.21.15.7:0/4175237160 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f359c09b1a0 con 0x7f35ac102850 2024-01-11T17:55:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35980a1f60 msgr2=0x7f35980a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35980a1f60 0x7f35980a4420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35ac1026b0 tx=0x7f35a8009260 comp rx=0 tx=0).stop 2024-01-11T17:55:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 msgr2=0x7f35ac199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac199c70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f359c009fd0 tx=0x7f359c00f690 comp rx=0 tx=0).stop 2024-01-11T17:55:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 shutdown_connections 2024-01-11T17:55:30.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35980a1f60 0x7f35980a4420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35ac101650 0x7f35ac199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac1031f0 0x7f35ac1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 --2- 172.21.15.7:0/4175237160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac102850 0x7f35ac199c70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 >> 172.21.15.7:0/4175237160 conn(0x7f35ac0fcbb0 msgr2=0x7f35ac073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:30.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 shutdown_connections 2024-01-11T17:55:30.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:30.227+0000 7f35b4712700 1 -- 172.21.15.7:0/4175237160 wait complete. 2024-01-11T17:55:30.634 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1704995728,"num_in_osds":12,"osd_in_since":1704995711,"num_remapped_pgs":0} 2024-01-11T17:55:30.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd dump --format=json 2024-01-11T17:55:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:55:31.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:30 smithi007 ceph-mon[33361]: osdmap e76: 12 total, 12 up, 12 in 2024-01-11T17:55:31.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:30 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4175237160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-11T17:55:31.220 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:30 smithi007 ceph-mon[33361]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:31.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:30 smithi016 ceph-mon[43195]: osdmap e76: 12 total, 12 up, 12 in 2024-01-11T17:55:31.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:30 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4175237160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-11T17:55:31.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:30 smithi016 ceph-mon[43195]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:31.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:30 smithi038 ceph-mon[43582]: osdmap e76: 12 total, 12 up, 12 in 2024-01-11T17:55:31.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:30 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4175237160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-11T17:55:31.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:30 smithi038 ceph-mon[43582]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:32.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 -- 172.21.15.7:0/3816983603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 msgr2=0x7fea08106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:32.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 --2- 172.21.15.7:0/3816983603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 0x7fea08106900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009990 tx=0x7fe9f8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 -- 172.21.15.7:0/3816983603 shutdown_connections 2024-01-11T17:55:32.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 --2- 172.21.15.7:0/3816983603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 0x7fea08106900 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 --2- 172.21.15.7:0/3816983603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 0x7fea0810d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 --2- 172.21.15.7:0/3816983603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea08100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 -- 172.21.15.7:0/3816983603 >> 172.21.15.7:0/3816983603 conn(0x7fea08077440 msgr2=0x7fea08077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 -- 172.21.15.7:0/3816983603 shutdown_connections 2024-01-11T17:55:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.253+0000 7fea0e03f700 1 -- 172.21.15.7:0/3816983603 wait complete. 2024-01-11T17:55:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.254+0000 7fea0e03f700 1 Processor -- start 2024-01-11T17:55:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.254+0000 7fea0e03f700 1 -- start start 2024-01-11T17:55:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.254+0000 7fea0e03f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea0819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:32.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea0e03f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 0x7fea0819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea0e03f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 0x7fea081999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea0e03f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0819a040 con 0x7fea08100510 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea0e03f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0819a180 con 0x7fea08100ed0 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea0e03f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0819a340 con 0x7fea08106520 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea0819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea0819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57032/0 (socket says 172.21.15.7:57032) 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 -- 172.21.15.7:0/1817448171 learned_addr learned my addr 172.21.15.7:0/1817448171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea07fff700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 0x7fea081999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea06ffd700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 0x7fea0819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 -- 172.21.15.7:0/1817448171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 msgr2=0x7fea081999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 0x7fea081999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 -- 172.21.15.7:0/1817448171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 msgr2=0x7fea0819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 0x7fea0819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 -- 172.21.15.7:0/1817448171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80095f0 con 0x7fea08100510 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea06ffd700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 0x7fea0819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.255+0000 7fea077fe700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea0819f820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000c00 tx=0x7fe9f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.256+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f801d070 con 0x7fea08100510 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.256+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0819a5c0 con 0x7fea08100510 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.256+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f8022470 con 0x7fea08100510 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.256+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8017650 con 0x7fea08100510 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.256+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea081a68c0 con 0x7fea08100510 2024-01-11T17:55:32.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.257+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea08103820 con 0x7fea08100510 2024-01-11T17:55:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.259+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fe9f80225e0 con 0x7fea08100510 2024-01-11T17:55:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.260+0000 7fea04ff9700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe9f00a1ee0 0x7fe9f00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.261+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9f80cefc0 con 0x7fea08100510 2024-01-11T17:55:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.261+0000 7fea06ffd700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe9f00a1ee0 0x7fe9f00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.261+0000 7fea06ffd700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe9f00a1ee0 0x7fe9f00a43a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea08077c00 tx=0x7fe9fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.262+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe9f80984f0 con 0x7fea08100510 2024-01-11T17:55:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.474+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea0819af10 con 0x7fea08100510 2024-01-11T17:55:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.475+0000 7fea04ff9700 1 -- 172.21.15.7:0/1817448171 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18268 (secure 0 0 0) 0x7fe9f809bb10 con 0x7fea08100510 2024-01-11T17:55:32.476 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:32.476 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:55:29.931492+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe9f00a1ee0 msgr2=0x7fe9f00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe9f00a1ee0 0x7fe9f00a43a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea08077c00 tx=0x7fe9fc009260 comp rx=0 tx=0).stop 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 msgr2=0x7fea0819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea0819f820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000c00 tx=0x7fe9f8005740 comp rx=0 tx=0).stop 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 shutdown_connections 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe9f00a1ee0 0x7fe9f00a43a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea08106520 0x7fea081999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea08100510 0x7fea0819f820 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 --2- 172.21.15.7:0/1817448171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea08100ed0 0x7fea0819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 >> 172.21.15.7:0/1817448171 conn(0x7fea08077440 msgr2=0x7fea0810e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 shutdown_connections 2024-01-11T17:55:32.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:32.477+0000 7fea0e03f700 1 -- 172.21.15.7:0/1817448171 wait complete. 2024-01-11T17:55:32.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1817448171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:55:32.816 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-11T17:55:32.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-01-11T17:55:32.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1817448171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:55:33.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1817448171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:55:33.233 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:55:33.723 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:33 smithi007 ceph-mon[33361]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:33 smithi016 ceph-mon[43195]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:33 smithi038 ceph-mon[43582]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 -- 172.21.15.7:0/3622686226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8107650 msgr2=0x7f51f8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 --2- 172.21.15.7:0/3622686226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8107650 0x7f51f8107a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009990 tx=0x7f51ec009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 -- 172.21.15.7:0/3622686226 shutdown_connections 2024-01-11T17:55:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 --2- 172.21.15.7:0/3622686226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f8102020 0x7f51f810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 --2- 172.21.15.7:0/3622686226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8107650 0x7f51f8107a30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 --2- 172.21.15.7:0/3622686226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8101660 0x7f51f8101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 -- 172.21.15.7:0/3622686226 >> 172.21.15.7:0/3622686226 conn(0x7f51f8078770 msgr2=0x7f51f8078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 -- 172.21.15.7:0/3622686226 shutdown_connections 2024-01-11T17:55:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.472+0000 7f51fef59700 1 -- 172.21.15.7:0/3622686226 wait complete. 2024-01-11T17:55:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.473+0000 7f51fef59700 1 Processor -- start 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.473+0000 7f51fef59700 1 -- start start 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fef59700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 0x7f51f8199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fef59700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f8102020 0x7f51f819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fef59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8107650 0x7f51f819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fef59700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f819ec70 con 0x7f51f8101660 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fef59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f819ede0 con 0x7f51f8107650 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fef59700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f819efa0 con 0x7f51f8102020 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 0x7f51f8199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 0x7f51f8199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57056/0 (socket says 172.21.15.7:57056) 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 -- 172.21.15.7:0/4120062920 learned_addr learned my addr 172.21.15.7:0/4120062920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fd4f6700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8107650 0x7f51f819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51f7fff700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f8102020 0x7f51f819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 -- 172.21.15.7:0/4120062920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f8102020 msgr2=0x7f51f819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f8102020 0x7f51f819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 -- 172.21.15.7:0/4120062920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8107650 msgr2=0x7f51f819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8107650 0x7f51f819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.474+0000 7f51fccf5700 1 -- 172.21.15.7:0/4120062920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0095f0 con 0x7f51f8101660 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.475+0000 7f51fccf5700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 0x7f51f8199d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51ec000c00 tx=0x7f51ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.475+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec01d070 con 0x7f51f8101660 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.475+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51ec022470 con 0x7f51f8101660 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.475+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f819f220 con 0x7f51f8101660 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.475+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec017650 con 0x7f51f8101660 2024-01-11T17:55:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.475+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81a6ae0 con 0x7f51f8101660 2024-01-11T17:55:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.476+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f8102d90 con 0x7f51f8101660 2024-01-11T17:55:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.478+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f51ec0225e0 con 0x7f51f8101660 2024-01-11T17:55:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.480+0000 7f51f5ffb700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f51e80a1f60 0x7f51e80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.481+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f51ec0ce840 con 0x7f51f8101660 2024-01-11T17:55:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.481+0000 7f51f7fff700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f51e80a1f60 0x7f51e80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.481+0000 7f51f7fff700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f51e80a1f60 0x7f51e80a4420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51f80758a0 tx=0x7f51e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.485+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f51ec097d70 con 0x7f51f8101660 2024-01-11T17:55:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.699+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51f819b0f0 con 0x7f51f8101660 2024-01-11T17:55:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.699+0000 7f51f5ffb700 1 -- 172.21.15.7:0/4120062920 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51ec09b390 con 0x7f51f8101660 2024-01-11T17:55:34.700 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:55:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.701+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f51e80a1f60 msgr2=0x7f51e80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f51e80a1f60 0x7f51e80a4420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51f80758a0 tx=0x7f51e4009260 comp rx=0 tx=0).stop 2024-01-11T17:55:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 msgr2=0x7f51f8199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 0x7f51f8199d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51ec000c00 tx=0x7f51ec005740 comp rx=0 tx=0).stop 2024-01-11T17:55:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 shutdown_connections 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f51e80a1f60 0x7f51e80a4420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f8107650 0x7f51f819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f8101660 0x7f51f8199d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 --2- 172.21.15.7:0/4120062920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f8102020 0x7f51f819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 >> 172.21.15.7:0/4120062920 conn(0x7f51f8078770 msgr2=0x7f51f80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 shutdown_connections 2024-01-11T17:55:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:34.702+0000 7f51fef59700 1 -- 172.21.15.7:0/4120062920 wait complete. 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.b on smithi016 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.e on smithi016 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.h on smithi016 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.k on smithi016 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.c on smithi038 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.f on smithi038 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.i on smithi038 2024-01-11T17:55:35.175 INFO:tasks.cephadm:Adding mds.l on smithi038 2024-01-11T17:55:35.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l' 2024-01-11T17:55:35.481 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T17:55:35.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:35 smithi038 ceph-mon[43582]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:35.526 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:35 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4120062920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:55:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:35 smithi007 ceph-mon[33361]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:35 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4120062920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:55:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:35 smithi016 ceph-mon[43195]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:35 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4120062920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:55:36.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 -- 172.21.15.38:0/1049349696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 msgr2=0x7f0734108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:36.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 --2- 172.21.15.38:0/1049349696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734108990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b0a0 tx=0x7f072c00b3b0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 -- 172.21.15.38:0/1049349696 shutdown_connections 2024-01-11T17:55:36.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 --2- 172.21.15.38:0/1049349696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734108990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 --2- 172.21.15.38:0/1049349696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0734100540 0x7f0734100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 --2- 172.21.15.38:0/1049349696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0734101740 0x7f0734101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 -- 172.21.15.38:0/1049349696 >> 172.21.15.38:0/1049349696 conn(0x7f07340fbae0 msgr2=0x7f07340fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 -- 172.21.15.38:0/1049349696 shutdown_connections 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.501+0000 7f073aa73700 1 -- 172.21.15.38:0/1049349696 wait complete. 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 Processor -- start 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 -- start start 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0734100540 0x7f07341044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0734101740 0x7f0734102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07341037e0 con 0x7f0734100540 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734103950 con 0x7f0734101740 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073aa73700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07341a16c0 con 0x7f07341020e0 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49216/0 (socket says 172.21.15.38:49216) 2024-01-11T17:55:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 -- 172.21.15.38:0/3500676510 learned_addr learned my addr 172.21.15.38:0/3500676510 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f072bfff700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0734101740 0x7f0734102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f073880f700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0734100540 0x7f07341044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 -- 172.21.15.38:0/3500676510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0734101740 msgr2=0x7f0734102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0734101740 0x7f0734102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 -- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0734100540 msgr2=0x7f07341044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0734100540 0x7f07341044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.502+0000 7f0739010700 1 -- 172.21.15.38:0/3500676510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c009d00 con 0x7f07341020e0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.503+0000 7f0739010700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734103060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f072c006070 tx=0x7f072c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.503+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00e040 con 0x7f07341020e0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.503+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07341a1940 con 0x7f07341020e0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.503+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f072c00ce70 con 0x7f07341020e0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.503+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c004020 con 0x7f07341020e0 2024-01-11T17:55:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.503+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07341a1e30 con 0x7f07341020e0 2024-01-11T17:55:36.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.504+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07340682d0 con 0x7f07341020e0 2024-01-11T17:55:36.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.505+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f072c01b660 con 0x7f07341020e0 2024-01-11T17:55:36.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.506+0000 7f0729ffb700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f071c0a1f60 0x7f071c0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:36.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.506+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f072c0cee00 con 0x7f07341020e0 2024-01-11T17:55:36.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.506+0000 7f073880f700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f071c0a1f60 0x7f071c0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:36.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.507+0000 7f073880f700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f071c0a1f60 0x7f071c0a4420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0730006fd0 tx=0x7f0730008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:36.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.510+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f072c098330 con 0x7f07341020e0 2024-01-11T17:55:36.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.740+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f07340fdd50 con 0x7f071c0a1f60 2024-01-11T17:55:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.748+0000 7f0729ffb700 1 -- 172.21.15.38:0/3500676510 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f07340fdd50 con 0x7f071c0a1f60 2024-01-11T17:55:36.749 INFO:teuthology.orchestra.run.smithi038.stdout:Scheduled mds.all update... 2024-01-11T17:55:36.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f071c0a1f60 msgr2=0x7f071c0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f071c0a1f60 0x7f071c0a4420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0730006fd0 tx=0x7f0730008040 comp rx=0 tx=0).stop 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 msgr2=0x7f0734103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734103060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f072c006070 tx=0x7f072c004d00 comp rx=0 tx=0).stop 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 shutdown_connections 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f071c0a1f60 0x7f071c0a4420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07341020e0 0x7f0734103060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0734100540 0x7f07341044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 --2- 172.21.15.38:0/3500676510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0734101740 0x7f0734102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 >> 172.21.15.38:0/3500676510 conn(0x7f07340fbae0 msgr2=0x7f07340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 shutdown_connections 2024-01-11T17:55:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-01-11T17:55:36.751+0000 7f073aa73700 1 -- 172.21.15.38:0/3500676510 wait complete. 2024-01-11T17:55:37.213 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.a.service 2024-01-11T17:55:37.219 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.d.service 2024-01-11T17:55:37.265 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.g.service 2024-01-11T17:55:37.301 INFO:journalctl@ceph.mds.a.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:55:37.305 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.j.service 2024-01-11T17:55:37.340 INFO:journalctl@ceph.mds.d.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:55:37.344 DEBUG:teuthology.orchestra.run.smithi016:mds.b> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.b.service 2024-01-11T17:55:37.347 DEBUG:teuthology.orchestra.run.smithi016:mds.e> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.e.service 2024-01-11T17:55:37.378 INFO:journalctl@ceph.mds.g.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:55:37.387 INFO:journalctl@ceph.mds.b.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:55:37.388 DEBUG:teuthology.orchestra.run.smithi016:mds.h> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h.service 2024-01-11T17:55:37.414 INFO:journalctl@ceph.mds.j.smithi007.stdout:-- Logs begin at Thu 2024-01-11 17:36:41 UTC. -- 2024-01-11T17:55:37.421 INFO:journalctl@ceph.mds.e.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:55:37.422 DEBUG:teuthology.orchestra.run.smithi016:mds.k> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.k.service 2024-01-11T17:55:37.453 INFO:journalctl@ceph.mds.h.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:55:37.455 DEBUG:teuthology.orchestra.run.smithi038:mds.c> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.c.service 2024-01-11T17:55:37.457 DEBUG:teuthology.orchestra.run.smithi038:mds.f> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.f.service 2024-01-11T17:55:37.486 INFO:journalctl@ceph.mds.k.smithi016.stdout:-- Logs begin at Thu 2024-01-11 17:30:33 UTC. -- 2024-01-11T17:55:37.496 INFO:journalctl@ceph.mds.c.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:55:37.498 DEBUG:teuthology.orchestra.run.smithi038:mds.i> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.i.service 2024-01-11T17:55:37.531 INFO:journalctl@ceph.mds.f.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:55:37.533 DEBUG:teuthology.orchestra.run.smithi038:mds.l> sudo journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.l.service 2024-01-11T17:55:37.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:55:37.566 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-01-11T17:55:37.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:55:37.596 INFO:journalctl@ceph.mds.l.smithi038.stdout:-- Logs begin at Thu 2024-01-11 17:30:36 UTC. -- 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:37.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:37.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:37 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:37.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:37.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:37.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:37 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/272130422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88102780 msgr2=0x7f0d88102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/272130422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88102780 0x7f0d88102be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009990 tx=0x7f0d74009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/272130422 shutdown_connections 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/272130422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 0x7f0d881099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/272130422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88101590 0x7f0d881019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/272130422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88102780 0x7f0d88102be0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/272130422 >> 172.21.15.7:0/272130422 conn(0x7f0d880fcb00 msgr2=0x7f0d880fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/272130422 shutdown_connections 2024-01-11T17:55:37.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.781+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/272130422 wait complete. 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.782+0000 7f0d8f1ff700 1 Processor -- start 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.782+0000 7f0d8f1ff700 1 -- start start 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8f1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 0x7f0d88197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8f1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88102780 0x7f0d88197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 0x7f0d8819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819f080 con 0x7f0d88103120 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8f1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819f1c0 con 0x7f0d88102780 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8f1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819f350 con 0x7f0d88101590 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 0x7f0d88197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8d79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 0x7f0d8819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 0x7f0d88197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41952/0 (socket says 172.21.15.7:41952) 2024-01-11T17:55:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 -- 172.21.15.7:0/1467095799 learned_addr learned my addr 172.21.15.7:0/1467095799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d87fff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88102780 0x7f0d88197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 -- 172.21.15.7:0/1467095799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88102780 msgr2=0x7f0d88197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88102780 0x7f0d88197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 -- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 msgr2=0x7f0d8819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 0x7f0d8819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8cf9b700 1 -- 172.21.15.7:0/1467095799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d740095f0 con 0x7f0d88101590 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.783+0000 7f0d8d79c700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 0x7f0d8819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.784+0000 7f0d8cf9b700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 0x7f0d88197500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d980 tx=0x7f0d7c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.784+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00f890 con 0x7f0d88101590 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.784+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8819f630 con 0x7f0d88101590 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.784+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d7c010460 con 0x7f0d88101590 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.784+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0195c0 con 0x7f0d88101590 2024-01-11T17:55:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.784+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8819fb40 con 0x7f0d88101590 2024-01-11T17:55:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.785+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d88108a60 con 0x7f0d88101590 2024-01-11T17:55:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.787+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0d7c0105d0 con 0x7f0d88101590 2024-01-11T17:55:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.789+0000 7f0d85ffb700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d780a1ee0 0x7f0d780a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.789+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0d7c0cd400 con 0x7f0d88101590 2024-01-11T17:55:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.789+0000 7f0d87fff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d780a1ee0 0x7f0d780a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.789+0000 7f0d87fff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d780a1ee0 0x7f0d780a43a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400b2e0 tx=0x7f0d74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:37.791+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0d7c0968a0 con 0x7f0d88101590 2024-01-11T17:55:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.006+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d880683f0 con 0x7f0d88101590 2024-01-11T17:55:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.009+0000 7f0d85ffb700 1 -- 172.21.15.7:0/1467095799 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f0d7c096030 con 0x7f0d88101590 2024-01-11T17:55:38.010 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:38.010 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:55:29.931492+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d780a1ee0 msgr2=0x7f0d780a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d780a1ee0 0x7f0d780a43a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400b2e0 tx=0x7f0d74005fd0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 msgr2=0x7f0d88197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 0x7f0d88197500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00d980 tx=0x7f0d7c00dc90 comp rx=0 tx=0).stop 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 shutdown_connections 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d780a1ee0 0x7f0d780a43a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d88101590 0x7f0d88197500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d88103120 0x7f0d8819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 --2- 172.21.15.7:0/1467095799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d88102780 0x7f0d88197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 >> 172.21.15.7:0/1467095799 conn(0x7f0d880fcb00 msgr2=0x7f0d88105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 shutdown_connections 2024-01-11T17:55:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.011+0000 7f0d8f1ff700 1 -- 172.21.15.7:0/1467095799 wait complete. 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:38.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:37 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:38.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-11T17:55:38.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:55:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/1280544280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 msgr2=0x7f4f581019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/1280544280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581019d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f480099c0 tx=0x7f4f48009cd0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/1280544280 shutdown_connections 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/1280544280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 0x7f4f581099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/1280544280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581019d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/1280544280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f581027a0 0x7f4f58102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.239+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/1280544280 >> 172.21.15.7:0/1280544280 conn(0x7f4f580fcb00 msgr2=0x7f4f580fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.240+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/1280544280 shutdown_connections 2024-01-11T17:55:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.240+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/1280544280 wait complete. 2024-01-11T17:55:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.241+0000 7f4f5e3ff700 1 Processor -- start 2024-01-11T17:55:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.241+0000 7f4f5e3ff700 1 -- start start 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5e3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f581027a0 0x7f4f58199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5e3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 0x7f4f581a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581a12d0 con 0x7f4f581015b0 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5e3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581a1410 con 0x7f4f581027a0 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5e3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581a15a0 con 0x7f4f58103140 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57106/0 (socket says 172.21.15.7:57106) 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5c99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 0x7f4f581a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 -- 172.21.15.7:0/3268792898 learned_addr learned my addr 172.21.15.7:0/3268792898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f577fe700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f581027a0 0x7f4f58199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 -- 172.21.15.7:0/3268792898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 msgr2=0x7f4f581a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 0x7f4f581a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 -- 172.21.15.7:0/3268792898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f581027a0 msgr2=0x7f4f58199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f581027a0 0x7f4f58199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f57fff700 1 -- 172.21.15.7:0/3268792898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480095f0 con 0x7f4f581015b0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.242+0000 7f4f5c99c700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 0x7f4f581a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.243+0000 7f4f57fff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581997a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009990 tx=0x7f4f4800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.243+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4801d070 con 0x7f4f581015b0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.243+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f48005030 con 0x7f4f581015b0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.243+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004970 con 0x7f4f581015b0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.243+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f581a1820 con 0x7f4f581015b0 2024-01-11T17:55:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.243+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f581a1bf0 con 0x7f4f581015b0 2024-01-11T17:55:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.244+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f58068430 con 0x7f4f581015b0 2024-01-11T17:55:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.246+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4f48005410 con 0x7f4f581015b0 2024-01-11T17:55:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.248+0000 7f4f557fa700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4f400a1f60 0x7f4f400a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.249+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f480ce180 con 0x7f4f581015b0 2024-01-11T17:55:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.249+0000 7f4f577fe700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4f400a1f60 0x7f4f400a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.249+0000 7f4f577fe700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4f400a1f60 0x7f4f400a4420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009c60 tx=0x7f4f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.252+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f4f480976b0 con 0x7f4f581015b0 2024-01-11T17:55:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.469+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f581a1e10 con 0x7f4f581015b0 2024-01-11T17:55:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.470+0000 7f4f557fa700 1 -- 172.21.15.7:0/3268792898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f4809acd0 con 0x7f4f581015b0 2024-01-11T17:55:38.470 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4f400a1f60 msgr2=0x7f4f400a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4f400a1f60 0x7f4f400a4420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009c60 tx=0x7f4f4c009260 comp rx=0 tx=0).stop 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 msgr2=0x7f4f581997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581997a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009990 tx=0x7f4f4800f690 comp rx=0 tx=0).stop 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 shutdown_connections 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4f400a1f60 0x7f4f400a4420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f58103140 0x7f4f581a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f581015b0 0x7f4f581997a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 --2- 172.21.15.7:0/3268792898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f581027a0 0x7f4f58199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 >> 172.21.15.7:0/3268792898 conn(0x7f4f580fcb00 msgr2=0x7f4f58073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 shutdown_connections 2024-01-11T17:55:38.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.472+0000 7f4f5e3ff700 1 -- 172.21.15.7:0/3268792898 wait complete. 2024-01-11T17:55:38.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:38.519 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 17:55:38 smithi038 systemd[1]: Starting Ceph mds.l for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 -- 172.21.15.7:0/3742752971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38101590 msgr2=0x7f9a381019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/3742752971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38101590 0x7f9a381019b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009920 tx=0x7f9a28009c30 comp rx=0 tx=0).stop 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 -- 172.21.15.7:0/3742752971 shutdown_connections 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/3742752971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38103120 0x7f9a381099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/3742752971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38101590 0x7f9a381019b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/3742752971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a38102780 0x7f9a38102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 -- 172.21.15.7:0/3742752971 >> 172.21.15.7:0/3742752971 conn(0x7f9a380fcb00 msgr2=0x7f9a380fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 -- 172.21.15.7:0/3742752971 shutdown_connections 2024-01-11T17:55:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.699+0000 7f9a3ddff700 1 -- 172.21.15.7:0/3742752971 wait complete. 2024-01-11T17:55:38.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.700+0000 7f9a3ddff700 1 Processor -- start 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.700+0000 7f9a3ddff700 1 -- start start 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.700+0000 7f9a3ddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a38101590 0x7f9a38199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a3ddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 0x7f9a38199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a3ddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38103120 0x7f9a381a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a3ddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a1300 con 0x7f9a38102780 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a3ddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a1440 con 0x7f9a38103120 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a3ddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a15d0 con 0x7f9a38101590 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 0x7f9a38199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 0x7f9a38199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57134/0 (socket says 172.21.15.7:57134) 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 -- 172.21.15.7:0/1130546130 learned_addr learned my addr 172.21.15.7:0/1130546130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a38101590 0x7f9a38199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a37fff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38103120 0x7f9a381a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 -- 172.21.15.7:0/1130546130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a38101590 msgr2=0x7f9a38199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a38101590 0x7f9a38199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 -- 172.21.15.7:0/1130546130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38103120 msgr2=0x7f9a381a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38103120 0x7f9a381a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 -- 172.21.15.7:0/1130546130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30009d00 con 0x7f9a38102780 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a37fff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38103120 0x7f9a381a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.701+0000 7f9a36ffd700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 0x7f9a38199d60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e9a0 tx=0x7f9a2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.702+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c009790 con 0x7f9a38102780 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.702+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a2c00cbd0 con 0x7f9a38102780 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.702+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c018680 con 0x7f9a38102780 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.702+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a28009520 con 0x7f9a38102780 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.702+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a381a1c80 con 0x7f9a38102780 2024-01-11T17:55:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.703+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a380683f0 con 0x7f9a38102780 2024-01-11T17:55:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.705+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f9a2c00cd40 con 0x7f9a38102780 2024-01-11T17:55:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.706+0000 7f9a34ff9700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9a200a1f60 0x7f9a200a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.706+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9a2c014070 con 0x7f9a38102780 2024-01-11T17:55:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.707+0000 7f9a377fe700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9a200a1f60 0x7f9a200a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.707+0000 7f9a377fe700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9a200a1f60 0x7f9a200a4420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098f0 tx=0x7f9a28019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.710+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f9a2c052380 con 0x7f9a38102780 2024-01-11T17:55:38.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:38 smithi007 ceph-mon[33361]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:55:38.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:38 smithi007 ceph-mon[33361]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l;count:12 2024-01-11T17:55:38.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:38 smithi007 ceph-mon[33361]: Deploying daemon mds.l on smithi038 2024-01-11T17:55:38.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:38 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1467095799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:55:38.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:38 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3268792898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:55:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.973+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a3810cdb0 con 0x7f9a38102780 2024-01-11T17:55:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.973+0000 7f9a34ff9700 1 -- 172.21.15.7:0/1130546130 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f9a2c096c20 con 0x7f9a38102780 2024-01-11T17:55:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:38.974 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-11T17:55:38.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9a200a1f60 msgr2=0x7f9a200a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9a200a1f60 0x7f9a200a4420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098f0 tx=0x7f9a28019040 comp rx=0 tx=0).stop 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 msgr2=0x7f9a38199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 0x7f9a38199d60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e9a0 tx=0x7f9a2c00ecb0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 shutdown_connections 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9a200a1f60 0x7f9a200a4420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a38101590 0x7f9a38199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a38103120 0x7f9a381a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 --2- 172.21.15.7:0/1130546130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a38102780 0x7f9a38199d60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 >> 172.21.15.7:0/1130546130 conn(0x7f9a380fcb00 msgr2=0x7f9a38107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 shutdown_connections 2024-01-11T17:55:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:38.976+0000 7f9a3ddff700 1 -- 172.21.15.7:0/1130546130 wait complete. 2024-01-11T17:55:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-01-11T17:55:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:38 smithi016 ceph-mon[43195]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:55:38.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:38 smithi016 ceph-mon[43195]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l;count:12 2024-01-11T17:55:38.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:38 smithi016 ceph-mon[43195]: Deploying daemon mds.l on smithi038 2024-01-11T17:55:38.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:38 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1467095799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:55:38.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:38 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3268792898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:55:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-mon[43582]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:55:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-mon[43582]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l;count:12 2024-01-11T17:55:39.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-mon[43582]: Deploying daemon mds.l on smithi038 2024-01-11T17:55:39.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1467095799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:55:39.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3268792898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:55:39.027 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-mds[79486]: -- 172.21.15.38:0/3930650629 <== mon.1 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5612c4c8b520 con 0x5612c4c61000 2024-01-11T17:55:39.027 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 17:55:38 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-l[79463]: starting mds.l at 2024-01-11T17:55:39.027 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 17:55:38 smithi038 bash[79448]: 908f17be23f4cb51676da166ca26c0f8be09e4fb19513017683ccdaa6219be6f 2024-01-11T17:55:39.027 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 17:55:38 smithi038 systemd[1]: Started Ceph mds.l for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6832/3639639157,v1:172.21.15.38:6833/3639639157] up:boot 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: fsmap 1 up:standby 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:39.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:39.895 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:39.895 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:39 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1130546130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6832/3639639157,v1:172.21.15.38:6833/3639639157] up:boot 2024-01-11T17:55:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: fsmap 1 up:standby 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:39 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1130546130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:40.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:40.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6832/3639639157,v1:172.21.15.38:6833/3639639157] up:boot 2024-01-11T17:55:40.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: fsmap 1 up:standby 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:39 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1130546130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:40.445 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 17:55:40 smithi016 systemd[1]: Starting Ceph mds.k for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:40.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:41.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:40 smithi016 ceph-mon[43195]: Deploying daemon mds.k on smithi016 2024-01-11T17:55:41.000 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 17:55:40 smithi016 ceph-mds[79202]: -- 172.21.15.16:0/583994515 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563266251520 con 0x563266226800 2024-01-11T17:55:41.000 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 17:55:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-k[79179]: starting mds.k at 2024-01-11T17:55:41.000 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 17:55:40 smithi016 bash[79164]: 3d25785caeeb3bfca01526a23364595d3d3c936d4dee7306fba8912a47441e67 2024-01-11T17:55:41.000 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 17:55:40 smithi016 systemd[1]: Started Ceph mds.k for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:41.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:40 smithi038 ceph-mon[43582]: Deploying daemon mds.k on smithi016 2024-01-11T17:55:41.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:40 smithi007 ceph-mon[33361]: Deploying daemon mds.k on smithi016 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 -- 172.21.15.7:0/2368191533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 msgr2=0x7f70081099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 --2- 172.21.15.7:0/2368191533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f70081099d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f700000b0d0 tx=0x7f700000b3e0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 -- 172.21.15.7:0/2368191533 shutdown_connections 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 --2- 172.21.15.7:0/2368191533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f70081099d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 --2- 172.21.15.7:0/2368191533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f70081019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 --2- 172.21.15.7:0/2368191533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7008102780 0x7f7008102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.219+0000 7f700ddff700 1 -- 172.21.15.7:0/2368191533 >> 172.21.15.7:0/2368191533 conn(0x7f70080fcb00 msgr2=0x7f70080fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.220+0000 7f700ddff700 1 -- 172.21.15.7:0/2368191533 shutdown_connections 2024-01-11T17:55:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.220+0000 7f700ddff700 1 -- 172.21.15.7:0/2368191533 wait complete. 2024-01-11T17:55:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 Processor -- start 2024-01-11T17:55:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 -- start start 2024-01-11T17:55:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f7008197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7008102780 0x7f7008197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f700819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700819f0c0 con 0x7f7008103120 2024-01-11T17:55:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700819f200 con 0x7f7008101590 2024-01-11T17:55:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.221+0000 7f700ddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700819f390 con 0x7f7008102780 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f7008197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f7008197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50306/0 (socket says 172.21.15.7:50306) 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 -- 172.21.15.7:0/1575265489 learned_addr learned my addr 172.21.15.7:0/1575265489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f7007fff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f700819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f7006ffd700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7008102780 0x7f7008197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 -- 172.21.15.7:0/1575265489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7008102780 msgr2=0x7f7008197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7008102780 0x7f7008197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 -- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 msgr2=0x7f700819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f700819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 -- 172.21.15.7:0/1575265489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000009d00 con 0x7f7008101590 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f7007fff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f700819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.222+0000 7f70077fe700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f7008197540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800e970 tx=0x7f6ff800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.223+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8009790 con 0x7f7008101590 2024-01-11T17:55:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.223+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700819f670 con 0x7f7008101590 2024-01-11T17:55:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.223+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ff800c7e0 con 0x7f7008101590 2024-01-11T17:55:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.223+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8003c70 con 0x7f7008101590 2024-01-11T17:55:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.223+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700819fbd0 con 0x7f7008101590 2024-01-11T17:55:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.224+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008108940 con 0x7f7008101590 2024-01-11T17:55:41.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.228+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6ff800cc50 con 0x7f7008101590 2024-01-11T17:55:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.229+0000 7f7004ff9700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ff00a1ee0 0x7f6ff00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.229+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ff8014070 con 0x7f7008101590 2024-01-11T17:55:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.229+0000 7f7006ffd700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ff00a1ee0 0x7f6ff00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.230+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6ff8096d30 con 0x7f7008101590 2024-01-11T17:55:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.230+0000 7f7006ffd700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ff00a1ee0 0x7f6ff00a43a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009c60 tx=0x7f6ffc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.506+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70080683f0 con 0x7f7008101590 2024-01-11T17:55:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.507+0000 7f7004ff9700 1 -- 172.21.15.7:0/1575265489 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2113 (secure 0 0 0) 0x7f6ff80b8e60 con 0x7f7008101590 2024-01-11T17:55:41.508 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-11T17:55:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ff00a1ee0 msgr2=0x7f6ff00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ff00a1ee0 0x7f6ff00a43a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009c60 tx=0x7f6ffc009260 comp rx=0 tx=0).stop 2024-01-11T17:55:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 msgr2=0x7f7008197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f7008197540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800e970 tx=0x7f6ff800ec80 comp rx=0 tx=0).stop 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 shutdown_connections 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ff00a1ee0 0x7f6ff00a43a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7008103120 0x7f700819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7008101590 0x7f7008197540 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 --2- 172.21.15.7:0/1575265489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7008102780 0x7f7008197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 >> 172.21.15.7:0/1575265489 conn(0x7f70080fcb00 msgr2=0x7f7008105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 shutdown_connections 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:41.510+0000 7f700ddff700 1 -- 172.21.15.7:0/1575265489 wait complete. 2024-01-11T17:55:41.511 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-01-11T17:55:41.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:41.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6832/189478188,v1:172.21.15.16:6833/189478188] up:boot 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: fsmap 2 up:standby 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:41.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:41 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1575265489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6832/189478188,v1:172.21.15.16:6833/189478188] up:boot 2024-01-11T17:55:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: fsmap 2 up:standby 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:41.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:41 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1575265489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6832/189478188,v1:172.21.15.16:6833/189478188] up:boot 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: fsmap 2 up:standby 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:41 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1575265489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:42.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:42 smithi007 ceph-mon[33361]: Deploying daemon mds.j on smithi007 2024-01-11T17:55:42.755 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 17:55:42 smithi007 systemd[1]: Starting Ceph mds.j for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:42.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:42 smithi016 ceph-mon[43195]: Deploying daemon mds.j on smithi007 2024-01-11T17:55:43.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:42 smithi038 ceph-mon[43582]: Deploying daemon mds.j on smithi007 2024-01-11T17:55:43.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:43.571 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 17:55:43 smithi007 bash[79784]: 7a159006276d8475188fbb34ef2dd70a43ceb031ad58ac2ef398df59e226921b 2024-01-11T17:55:43.571 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mds[79823]: -- 172.21.15.7:0/2601845061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55859f231520 con 0x55859f209000 2024-01-11T17:55:43.571 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-j[79800]: starting mds.j at 2024-01-11T17:55:43.571 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 17:55:43 smithi007 systemd[1]: Started Ceph mds.j for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 -- 172.21.15.7:0/2191662117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078075650 msgr2=0x7f0078073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 --2- 172.21.15.7:0/2191662117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078075650 0x7f0078073aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0068009990 tx=0x7f0068009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 -- 172.21.15.7:0/2191662117 shutdown_connections 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 --2- 172.21.15.7:0/2191662117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007810db60 0x7f007810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 --2- 172.21.15.7:0/2191662117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078075650 0x7f0078073aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 --2- 172.21.15.7:0/2191662117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0078074070 0x7f00780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 -- 172.21.15.7:0/2191662117 >> 172.21.15.7:0/2191662117 conn(0x7f00780fcb00 msgr2=0x7f00780fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 -- 172.21.15.7:0/2191662117 shutdown_connections 2024-01-11T17:55:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.747+0000 7f007ddff700 1 -- 172.21.15.7:0/2191662117 wait complete. 2024-01-11T17:55:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 Processor -- start 2024-01-11T17:55:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 -- start start 2024-01-11T17:55:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 0x7f007819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0078075650 0x7f007819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007810db60 0x7f00781a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a5620 con 0x7f0078074070 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a5760 con 0x7f007810db60 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.749+0000 7f007ddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a58f0 con 0x7f0078075650 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 0x7f007819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 0x7f007819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41174/0 (socket says 172.21.15.7:41174) 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 -- 172.21.15.7:0/2201083432 learned_addr learned my addr 172.21.15.7:0/2201083432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f0077fff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007810db60 0x7f00781a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f0076ffd700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0078075650 0x7f007819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 -- 172.21.15.7:0/2201083432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0078075650 msgr2=0x7f007819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0078075650 0x7f007819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 -- 172.21.15.7:0/2201083432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007810db60 msgr2=0x7f00781a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007810db60 0x7f00781a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 -- 172.21.15.7:0/2201083432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680095f0 con 0x7f0078074070 2024-01-11T17:55:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.750+0000 7f00777fe700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 0x7f007819daa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.751+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006801d070 con 0x7f0078074070 2024-01-11T17:55:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.751+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0068021410 con 0x7f0078074070 2024-01-11T17:55:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.751+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00781a5b70 con 0x7f0078074070 2024-01-11T17:55:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.751+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068017650 con 0x7f0078074070 2024-01-11T17:55:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.751+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00781a5ff0 con 0x7f0078074070 2024-01-11T17:55:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.752+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0078103dc0 con 0x7f0078074070 2024-01-11T17:55:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.756+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f00680217f0 con 0x7f0078074070 2024-01-11T17:55:43.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.757+0000 7f0074ff9700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f00600a1f60 0x7f00600a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:43.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.757+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f00680ce980 con 0x7f0078074070 2024-01-11T17:55:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.757+0000 7f0076ffd700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f00600a1f60 0x7f00600a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.758+0000 7f0076ffd700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f00600a1f60 0x7f00600a4420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0078075370 tx=0x7f006c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:43.762+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0068097eb0 con 0x7f0078074070 2024-01-11T17:55:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:43.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:43.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:43.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:43.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:43.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:43 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:43 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.058+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00780683f0 con 0x7f0078074070 2024-01-11T17:55:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.059+0000 7f0074ff9700 1 -- 172.21.15.7:0/2201083432 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2901 (secure 0 0 0) 0x7f006809b4d0 con 0x7f0078074070 2024-01-11T17:55:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-11T17:55:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f00600a1f60 msgr2=0x7f00600a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f00600a1f60 0x7f00600a4420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0078075370 tx=0x7f006c007400 comp rx=0 tx=0).stop 2024-01-11T17:55:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 msgr2=0x7f007819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 0x7f007819daa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c00 tx=0x7f0068005740 comp rx=0 tx=0).stop 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 shutdown_connections 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f00600a1f60 0x7f00600a4420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f007810db60 0x7f00781a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0078074070 0x7f007819daa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 --2- 172.21.15.7:0/2201083432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0078075650 0x7f007819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 >> 172.21.15.7:0/2201083432 conn(0x7f00780fcb00 msgr2=0x7f0078100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 shutdown_connections 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:44.062+0000 7f007ddff700 1 -- 172.21.15.7:0/2201083432 wait complete. 2024-01-11T17:55:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:44.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:43 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:44.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:44 smithi007 ceph-mon[33361]: Deploying daemon mds.d on smithi007 2024-01-11T17:55:44.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:44 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6834/2806336658,v1:172.21.15.7:6835/2806336658] up:boot 2024-01-11T17:55:44.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:44 smithi007 ceph-mon[33361]: fsmap 3 up:standby 2024-01-11T17:55:44.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-11T17:55:44.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:44 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2201083432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:44 smithi016 ceph-mon[43195]: Deploying daemon mds.d on smithi007 2024-01-11T17:55:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:44 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6834/2806336658,v1:172.21.15.7:6835/2806336658] up:boot 2024-01-11T17:55:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:44 smithi016 ceph-mon[43195]: fsmap 3 up:standby 2024-01-11T17:55:44.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-11T17:55:44.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:44 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2201083432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:45.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:44 smithi038 ceph-mon[43582]: Deploying daemon mds.d on smithi007 2024-01-11T17:55:45.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:44 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6834/2806336658,v1:172.21.15.7:6835/2806336658] up:boot 2024-01-11T17:55:45.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:44 smithi038 ceph-mon[43582]: fsmap 3 up:standby 2024-01-11T17:55:45.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-11T17:55:45.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:44 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2201083432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:45.399 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 17:55:45 smithi007 systemd[1]: Starting Ceph mds.d for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:45.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:45 smithi016 ceph-mon[43195]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:46.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:45 smithi038 ceph-mon[43582]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:46.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:45 smithi007 ceph-mon[33361]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:46.071 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 17:55:45 smithi007 bash[80170]: 97079eb5cee61a87944621fc1e4b4695d553434e722b58c04c2cd7c783a59fbc 2024-01-11T17:55:46.071 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 17:55:45 smithi007 systemd[1]: Started Ceph mds.d for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:46.071 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 17:55:45 smithi007 ceph-mds[80208]: -- 172.21.15.7:0/307572109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a225ef1520 con 0x55a225ec8c00 2024-01-11T17:55:46.072 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 17:55:45 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-d[80185]: starting mds.d at 2024-01-11T17:55:46.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:46.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 -- 172.21.15.7:0/1098005432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758109450 msgr2=0x7fc75810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:46.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1098005432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758109450 0x7fc75810b8e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc75000b0a0 tx=0x7fc75000b3b0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 -- 172.21.15.7:0/1098005432 shutdown_connections 2024-01-11T17:55:46.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1098005432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758109450 0x7fc75810b8e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1098005432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1098005432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc758074070 0x7fc7580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 -- 172.21.15.7:0/1098005432 >> 172.21.15.7:0/1098005432 conn(0x7fc7580fcb00 msgr2=0x7fc7580fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:46.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 -- 172.21.15.7:0/1098005432 shutdown_connections 2024-01-11T17:55:46.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.294+0000 7fc7603ff700 1 -- 172.21.15.7:0/1098005432 wait complete. 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.295+0000 7fc7603ff700 1 Processor -- start 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 -- start start 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758074070 0x7fc758199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc758109450 0x7fc7581a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581a1350 con 0x7fc758075650 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581a1490 con 0x7fc758074070 2024-01-11T17:55:46.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc7603ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581a1620 con 0x7fc758109450 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc75d99a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc75e19b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758074070 0x7fc758199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc75e19b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758074070 0x7fc758199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39212/0 (socket says 172.21.15.7:39212) 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc75d99a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41214/0 (socket says 172.21.15.7:41214) 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc75e19b700 1 -- 172.21.15.7:0/1995053953 learned_addr learned my addr 172.21.15.7:0/1995053953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.296+0000 7fc75e99c700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc758109450 0x7fc7581a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc75d99a700 1 -- 172.21.15.7:0/1995053953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc758109450 msgr2=0x7fc7581a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc75d99a700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc758109450 0x7fc7581a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc75d99a700 1 -- 172.21.15.7:0/1995053953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758074070 msgr2=0x7fc758199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc75d99a700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758074070 0x7fc758199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc75d99a700 1 -- 172.21.15.7:0/1995053953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc754009520 con 0x7fc758075650 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc75d99a700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758199d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc74800e9c0 tx=0x7fc74800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748004220 con 0x7fc758075650 2024-01-11T17:55:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc750009d00 con 0x7fc758075650 2024-01-11T17:55:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc748004380 con 0x7fc758075650 2024-01-11T17:55:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748003bc0 con 0x7fc758075650 2024-01-11T17:55:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.297+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7581a1cd0 con 0x7fc758075650 2024-01-11T17:55:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.298+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc758108210 con 0x7fc758075650 2024-01-11T17:55:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.300+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc748003d20 con 0x7fc758075650 2024-01-11T17:55:46.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.302+0000 7fc74f7fe700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7440a1ee0 0x7fc7440a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:46.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.302+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc748014070 con 0x7fc758075650 2024-01-11T17:55:46.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.302+0000 7fc75e19b700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7440a1ee0 0x7fc7440a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:46.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.302+0000 7fc75e19b700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7440a1ee0 0x7fc7440a43a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc754009ef0 tx=0x7fc754009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:46.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.306+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc748096c70 con 0x7fc758075650 2024-01-11T17:55:46.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.565+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7580683f0 con 0x7fc758075650 2024-01-11T17:55:46.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.565+0000 7fc74f7fe700 1 -- 172.21.15.7:0/1995053953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2901 (secure 0 0 0) 0x7fc74809a290 con 0x7fc758075650 2024-01-11T17:55:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-11T17:55:46.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7440a1ee0 msgr2=0x7fc7440a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:46.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7440a1ee0 0x7fc7440a43a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc754009ef0 tx=0x7fc754009260 comp rx=0 tx=0).stop 2024-01-11T17:55:46.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 msgr2=0x7fc758199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:46.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758199d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc74800e9c0 tx=0x7fc74800ecd0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 shutdown_connections 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7440a1ee0 0x7fc7440a43a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc758109450 0x7fc7581a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc758074070 0x7fc758199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 --2- 172.21.15.7:0/1995053953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc758075650 0x7fc758199d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 >> 172.21.15.7:0/1995053953 conn(0x7fc7580fcb00 msgr2=0x7fc758105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 shutdown_connections 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:46.568+0000 7fc7603ff700 1 -- 172.21.15.7:0/1995053953 wait complete. 2024-01-11T17:55:46.570 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-01-11T17:55:47.147 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: Deploying daemon mds.e on smithi016 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1995053953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6836/1543402898,v1:172.21.15.7:6837/1543402898] up:boot 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: fsmap 4 up:standby 2024-01-11T17:55:47.148 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-11T17:55:47.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: Deploying daemon mds.e on smithi016 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1995053953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6836/1543402898,v1:172.21.15.7:6837/1543402898] up:boot 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: fsmap 4 up:standby 2024-01-11T17:55:47.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:47 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-11T17:55:47.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: Deploying daemon mds.e on smithi016 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1995053953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6836/1543402898,v1:172.21.15.7:6837/1543402898] up:boot 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: fsmap 4 up:standby 2024-01-11T17:55:47.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:47 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-11T17:55:47.435 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 17:55:47 smithi016 systemd[1]: Starting Ceph mds.e for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:48.246 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 17:55:47 smithi016 bash[79514]: 0c4e69b9f9b8ebd11db6f7eb2a4d44a7360656c0108b55270eb7609399b13800 2024-01-11T17:55:48.246 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 17:55:47 smithi016 systemd[1]: Started Ceph mds.e for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:48.246 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-mds[79552]: -- 172.21.15.16:0/2440982809 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55703160f520 con 0x5570315e4c00 2024-01-11T17:55:48.246 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 17:55:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e[79529]: starting mds.e at 2024-01-11T17:55:48.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 -- 172.21.15.7:0/2529208307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 msgr2=0x7fd2f0102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/2529208307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0102bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40099c0 tx=0x7fd2e4009cd0 comp rx=0 tx=0).stop 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 -- 172.21.15.7:0/2529208307 shutdown_connections 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/2529208307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2f0103130 0x7fd2f01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/2529208307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f01015a0 0x7fd2f01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/2529208307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0102bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 -- 172.21.15.7:0/2529208307 >> 172.21.15.7:0/2529208307 conn(0x7fd2f00fcb30 msgr2=0x7fd2f00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 -- 172.21.15.7:0/2529208307 shutdown_connections 2024-01-11T17:55:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.804+0000 7fd2f67ff700 1 -- 172.21.15.7:0/2529208307 wait complete. 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 Processor -- start 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 -- start start 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2f01015a0 0x7fd2f0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0103130 0x7fd2f01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f01a1200 con 0x7fd2f0102790 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f01a1340 con 0x7fd2f0103130 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2f67ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f01a14d0 con 0x7fd2f01015a0 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.805+0000 7fd2ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41238/0 (socket says 172.21.15.7:41238) 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 -- 172.21.15.7:0/3049109506 learned_addr learned my addr 172.21.15.7:0/3049109506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2effff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2f01015a0 0x7fd2f0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2f4d9c700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0103130 0x7fd2f01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 -- 172.21.15.7:0/3049109506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2f01015a0 msgr2=0x7fd2f0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2f01015a0 0x7fd2f0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 -- 172.21.15.7:0/3049109506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0103130 msgr2=0x7fd2f01a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0103130 0x7fd2f01a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 -- 172.21.15.7:0/3049109506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40095f0 con 0x7fd2f0102790 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ef7fe700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0199bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4003730 tx=0x7fd2e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e401d070 con 0x7fd2f0102790 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f01a1750 con 0x7fd2f0102790 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2e4005050 con 0x7fd2f0102790 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.806+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4004b10 con 0x7fd2f0102790 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.807+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f01a1bd0 con 0x7fd2f0102790 2024-01-11T17:55:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.807+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f00683f0 con 0x7fd2f0102790 2024-01-11T17:55:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.809+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fd2e40051c0 con 0x7fd2f0102790 2024-01-11T17:55:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.812+0000 7fd2ed7fa700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd2d80a1ee0 0x7fd2d80a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.812+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2e40ce2b0 con 0x7fd2f0102790 2024-01-11T17:55:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.812+0000 7fd2effff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd2d80a1ee0 0x7fd2d80a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.812+0000 7fd2effff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd2d80a1ee0 0x7fd2d80a43a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd2f01025f0 tx=0x7fd2e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:48.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:48.816+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd2e40977e0 con 0x7fd2f0102790 2024-01-11T17:55:49.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.076+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2f01a1fb0 con 0x7fd2f0102790 2024-01-11T17:55:49.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.076+0000 7fd2ed7fa700 1 -- 172.21.15.7:0/3049109506 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4480 (secure 0 0 0) 0x7fd2e409ae00 con 0x7fd2f0102790 2024-01-11T17:55:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:49.078 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd2d80a1ee0 msgr2=0x7fd2d80a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd2d80a1ee0 0x7fd2d80a43a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd2f01025f0 tx=0x7fd2e0009260 comp rx=0 tx=0).stop 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 msgr2=0x7fd2f0199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0199bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4003730 tx=0x7fd2e400f690 comp rx=0 tx=0).stop 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 shutdown_connections 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd2d80a1ee0 0x7fd2d80a43a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2f01015a0 0x7fd2f0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0103130 0x7fd2f01a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 --2- 172.21.15.7:0/3049109506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0102790 0x7fd2f0199bc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 >> 172.21.15.7:0/3049109506 conn(0x7fd2f00fcb30 msgr2=0x7fd2f0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 shutdown_connections 2024-01-11T17:55:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:49.079+0000 7fd2f67ff700 1 -- 172.21.15.7:0/3049109506 wait complete. 2024-01-11T17:55:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: Deploying daemon mds.f on smithi038 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6834/4278122554,v1:172.21.15.16:6835/4278122554] up:boot 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: fsmap 5 up:standby 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-11T17:55:49.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:48 smithi038 ceph-mon[43582]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:49.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: Deploying daemon mds.f on smithi038 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6834/4278122554,v1:172.21.15.16:6835/4278122554] up:boot 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: fsmap 5 up:standby 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-11T17:55:49.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:48 smithi016 ceph-mon[43195]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:49.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: Deploying daemon mds.f on smithi038 2024-01-11T17:55:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6834/4278122554,v1:172.21.15.16:6835/4278122554] up:boot 2024-01-11T17:55:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: fsmap 5 up:standby 2024-01-11T17:55:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-11T17:55:49.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:48 smithi007 ceph-mon[33361]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:49.776 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 17:55:49 smithi038 systemd[1]: Starting Ceph mds.f for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:50.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:49 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3049109506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:50.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:49 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3049109506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:50.277 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 17:55:49 smithi038 ceph-mds[79832]: -- 172.21.15.38:0/2101755240 <== mon.1 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fdf5eef520 con 0x55fdf5ec5000 2024-01-11T17:55:50.277 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 17:55:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f[79809]: starting mds.f at 2024-01-11T17:55:50.277 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 17:55:49 smithi038 bash[79794]: 69a1c96d2a89fbdbf93e21844af4107ef2001ac98f0bc188212f7827390f21c4 2024-01-11T17:55:50.278 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 17:55:49 smithi038 systemd[1]: Started Ceph mds.f for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:50.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:49 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3049109506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:51.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: Deploying daemon mds.c on smithi038 2024-01-11T17:55:51.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6834/1587519060,v1:172.21.15.38:6835/1587519060] up:boot 2024-01-11T17:55:51.156 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: fsmap 6 up:standby 2024-01-11T17:55:51.156 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-11T17:55:51.156 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:50 smithi038 ceph-mon[43582]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:51.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: Deploying daemon mds.c on smithi038 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6834/1587519060,v1:172.21.15.38:6835/1587519060] up:boot 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: fsmap 6 up:standby 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-11T17:55:51.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:50 smithi016 ceph-mon[43195]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:51.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 -- 172.21.15.7:0/1851968632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 msgr2=0x7fbe7c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1851968632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c102be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009990 tx=0x7fbe68009ca0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 -- 172.21.15.7:0/1851968632 shutdown_connections 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1851968632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c103120 0x7fbe7c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1851968632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c101590 0x7fbe7c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1851968632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c102be0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 -- 172.21.15.7:0/1851968632 >> 172.21.15.7:0/1851968632 conn(0x7fbe7c0fcb00 msgr2=0x7fbe7c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 -- 172.21.15.7:0/1851968632 shutdown_connections 2024-01-11T17:55:51.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.318+0000 7fbe82fff700 1 -- 172.21.15.7:0/1851968632 wait complete. 2024-01-11T17:55:51.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.319+0000 7fbe82fff700 1 Processor -- start 2024-01-11T17:55:51.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.319+0000 7fbe82fff700 1 -- start start 2024-01-11T17:55:51.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe82fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c101590 0x7fbe7c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:51.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe82fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:51.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: Deploying daemon mds.c on smithi038 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6834/1587519060,v1:172.21.15.38:6835/1587519060] up:boot 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: fsmap 6 up:standby 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-11T17:55:51.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:50 smithi007 ceph-mon[33361]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe82fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c103120 0x7fbe7c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe82fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1a1270 con 0x7fbe7c101590 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe82fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1a13b0 con 0x7fbe7c103120 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe82fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1a1540 con 0x7fbe7c102780 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe80d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c101590 0x7fbe7c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37610/0 (socket says 172.21.15.7:37610) 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 -- 172.21.15.7:0/1311337175 learned_addr learned my addr 172.21.15.7:0/1311337175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe8159c700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c103120 0x7fbe7c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 -- 172.21.15.7:0/1311337175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c103120 msgr2=0x7fbe7c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c103120 0x7fbe7c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 -- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c101590 msgr2=0x7fbe7c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c101590 0x7fbe7c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe7bfff700 1 -- 172.21.15.7:0/1311337175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe680095f0 con 0x7fbe7c102780 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe8159c700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c103120 0x7fbe7c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.320+0000 7fbe80d9b700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c101590 0x7fbe7c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.321+0000 7fbe7bfff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c199c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b680 tx=0x7fbe68009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.321+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6801d070 con 0x7fbe7c102780 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.321+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe68022470 con 0x7fbe7c102780 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.321+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c1a17c0 con 0x7fbe7c102780 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.321+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68017650 con 0x7fbe7c102780 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.321+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c1a1be0 con 0x7fbe7c102780 2024-01-11T17:55:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.322+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7c0683f0 con 0x7fbe7c102780 2024-01-11T17:55:51.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.324+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fbe68022870 con 0x7fbe7c102780 2024-01-11T17:55:51.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.325+0000 7fbe79ffb700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe6c0a1ee0 0x7fbe6c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:51.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.325+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe680ce830 con 0x7fbe7c102780 2024-01-11T17:55:51.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.325+0000 7fbe80d9b700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe6c0a1ee0 0x7fbe6c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:51.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.326+0000 7fbe80d9b700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe6c0a1ee0 0x7fbe6c0a43a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c1025e0 tx=0x7fbe70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:51.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.328+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fbe68097d60 con 0x7fbe7c102780 2024-01-11T17:55:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.587+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe7c10d050 con 0x7fbe7c102780 2024-01-11T17:55:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.588+0000 7fbe79ffb700 1 -- 172.21.15.7:0/1311337175 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5271 (secure 0 0 0) 0x7fbe6809b380 con 0x7fbe7c102780 2024-01-11T17:55:51.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:51.591 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-01-11T17:55:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe6c0a1ee0 msgr2=0x7fbe6c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe6c0a1ee0 0x7fbe6c0a43a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c1025e0 tx=0x7fbe70009260 comp rx=0 tx=0).stop 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 msgr2=0x7fbe7c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c199c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b680 tx=0x7fbe68009580 comp rx=0 tx=0).stop 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 shutdown_connections 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe6c0a1ee0 0x7fbe6c0a43a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe7c103120 0x7fbe7c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe7c101590 0x7fbe7c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 --2- 172.21.15.7:0/1311337175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe7c102780 0x7fbe7c199c30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 >> 172.21.15.7:0/1311337175 conn(0x7fbe7c0fcb00 msgr2=0x7fbe7c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 shutdown_connections 2024-01-11T17:55:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:51.591+0000 7fbe82fff700 1 -- 172.21.15.7:0/1311337175 wait complete. 2024-01-11T17:55:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-01-11T17:55:51.735 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 17:55:51 smithi038 systemd[1]: Starting Ceph mds.c for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:52.276 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 17:55:51 smithi038 bash[80135]: 44b513081536103a65aa258d9aaebca4d7f53ee270201d1c2cde0adf91a1cc87 2024-01-11T17:55:52.277 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 17:55:51 smithi038 systemd[1]: Started Ceph mds.c for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:52.277 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 17:55:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c[80150]: starting mds.c at 2024-01-11T17:55:52.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1311337175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:52.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:52.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:52.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:52.585 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1311337175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:52.585 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.586 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.586 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.586 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:52.586 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:52.586 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1311337175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:52.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:52.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:52.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:52.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:53.496 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 17:55:53 smithi016 systemd[1]: Starting Ceph mds.h for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:53.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-mon[43195]: Deploying daemon mds.h on smithi016 2024-01-11T17:55:53.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6836/2455039829,v1:172.21.15.38:6837/2455039829] up:boot 2024-01-11T17:55:53.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-mon[43195]: fsmap 7 up:standby 2024-01-11T17:55:53.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-11T17:55:53.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-mon[43195]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:53.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:53.681 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:53 smithi007 ceph-mon[33361]: Deploying daemon mds.h on smithi016 2024-01-11T17:55:53.681 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:53 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6836/2455039829,v1:172.21.15.38:6837/2455039829] up:boot 2024-01-11T17:55:53.681 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:53 smithi007 ceph-mon[33361]: fsmap 7 up:standby 2024-01-11T17:55:53.681 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-11T17:55:53.681 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:53 smithi007 ceph-mon[33361]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:53 smithi038 ceph-mon[43582]: Deploying daemon mds.h on smithi016 2024-01-11T17:55:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:53 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6836/2455039829,v1:172.21.15.38:6837/2455039829] up:boot 2024-01-11T17:55:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:53 smithi038 ceph-mon[43582]: fsmap 7 up:standby 2024-01-11T17:55:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-11T17:55:53.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:53 smithi038 ceph-mon[43582]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 -- 172.21.15.7:0/579183178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c103120 msgr2=0x7f447c1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 --2- 172.21.15.7:0/579183178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c103120 0x7f447c1099d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f447400b0d0 tx=0x7f447400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 -- 172.21.15.7:0/579183178 shutdown_connections 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 --2- 172.21.15.7:0/579183178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c101590 0x7f447c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 --2- 172.21.15.7:0/579183178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c103120 0x7f447c1099d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 --2- 172.21.15.7:0/579183178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 -- 172.21.15.7:0/579183178 >> 172.21.15.7:0/579183178 conn(0x7f447c0fcb00 msgr2=0x7f447c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 -- 172.21.15.7:0/579183178 shutdown_connections 2024-01-11T17:55:53.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.828+0000 7f44827ff700 1 -- 172.21.15.7:0/579183178 wait complete. 2024-01-11T17:55:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.829+0000 7f44827ff700 1 Processor -- start 2024-01-11T17:55:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.829+0000 7f44827ff700 1 -- start start 2024-01-11T17:55:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.829+0000 7f44827ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c101590 0x7f447c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.829+0000 7f44827ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.829+0000 7f44827ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c103120 0x7f447c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.829+0000 7f44827ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a1330 con 0x7f447c102780 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f44827ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a1470 con 0x7f447c101590 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f44827ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a1600 con 0x7f447c103120 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c101590 0x7f447c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56636/0 (socket says 172.21.15.7:56636) 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 -- 172.21.15.7:0/2291425101 learned_addr learned my addr 172.21.15.7:0/2291425101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c101590 0x7f447c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55066/0 (socket says 172.21.15.7:55066) 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f4480d9c700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c103120 0x7f447c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 -- 172.21.15.7:0/2291425101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c103120 msgr2=0x7f447c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c103120 0x7f447c1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 -- 172.21.15.7:0/2291425101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c101590 msgr2=0x7f447c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c101590 0x7f447c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:53.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 -- 172.21.15.7:0/2291425101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474009d00 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.830+0000 7f447b7fe700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c199cf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f447000e9a0 tx=0x7f447000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.831+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009790 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.831+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470010430 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.831+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c1a18e0 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.831+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470018680 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.831+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1a1df0 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.832+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c0683f0 con 0x7f447c102780 2024-01-11T17:55:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.834+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f44700105a0 con 0x7f447c102780 2024-01-11T17:55:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.835+0000 7f44797fa700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f44640a1ee0 0x7f44640a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.835+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4470014070 con 0x7f447c102780 2024-01-11T17:55:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.835+0000 7f447bfff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f44640a1ee0 0x7f44640a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.836+0000 7f447bfff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f44640a1ee0 0x7f44640a43a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f447c1025e0 tx=0x7f446c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:53.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:53.839+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f4470052380 con 0x7f447c102780 2024-01-11T17:55:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.118+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f447c1a1f80 con 0x7f447c102780 2024-01-11T17:55:54.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.119+0000 7f44797fa700 1 -- 172.21.15.7:0/2291425101 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6062 (secure 0 0 0) 0x7f4470096be0 con 0x7f447c102780 2024-01-11T17:55:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:54.121 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-01-11T17:55:54.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f44640a1ee0 msgr2=0x7f44640a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f44640a1ee0 0x7f44640a43a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f447c1025e0 tx=0x7f446c009260 comp rx=0 tx=0).stop 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 msgr2=0x7f447c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c199cf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f447000e9a0 tx=0x7f447000ecb0 comp rx=0 tx=0).stop 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 shutdown_connections 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f44640a1ee0 0x7f44640a43a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f447c103120 0x7f447c1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c101590 0x7f447c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 --2- 172.21.15.7:0/2291425101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447c102780 0x7f447c199cf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 >> 172.21.15.7:0/2291425101 conn(0x7f447c0fcb00 msgr2=0x7f447c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 shutdown_connections 2024-01-11T17:55:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:54.122+0000 7f44827ff700 1 -- 172.21.15.7:0/2291425101 wait complete. 2024-01-11T17:55:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T17:55:54.246 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 17:55:53 smithi016 bash[79854]: 13df28825d8e6f20c87735661b492ceedb8bb526ae05b7d5f512cb581890ab11 2024-01-11T17:55:54.246 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 17:55:53 smithi016 systemd[1]: Started Ceph mds.h for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:54.246 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-mds[79892]: -- 172.21.15.16:0/3426183458 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643a1871520 con 0x5643a1847000 2024-01-11T17:55:54.247 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 17:55:53 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h[79869]: starting mds.h at 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: Deploying daemon mds.i on smithi038 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2291425101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6836/2512926203,v1:172.21.15.16:6837/2512926203] up:boot 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: fsmap 8 up:standby 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-11T17:55:55.017 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:54 smithi038 ceph-mon[43582]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: Deploying daemon mds.i on smithi038 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2291425101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6836/2512926203,v1:172.21.15.16:6837/2512926203] up:boot 2024-01-11T17:55:55.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: fsmap 8 up:standby 2024-01-11T17:55:55.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-11T17:55:55.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:54 smithi016 ceph-mon[43195]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:55.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: Deploying daemon mds.i on smithi038 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2291425101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6836/2512926203,v1:172.21.15.16:6837/2512926203] up:boot 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: fsmap 8 up:standby 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-11T17:55:55.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:54 smithi007 ceph-mon[33361]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:55.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:55:55 smithi038 systemd[1]: Starting Ceph mds.i for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:56.026 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:55:55 smithi038 bash[80478]: dbd488d7373ebb24ed5f9a31375251a5315884636cc5db442de127285f5aa866 2024-01-11T17:55:56.027 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:55:55 smithi038 systemd[1]: Started Ceph mds.i for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:56.027 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:55:55 smithi038 ceph-mds[80516]: -- 172.21.15.38:0/4227321048 <== mon.1 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee0f9d3520 con 0x55ee0f9a8800 2024-01-11T17:55:56.027 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:55:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: starting mds.i at 2024-01-11T17:55:56.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.373+0000 7f6d455ff700 1 -- 172.21.15.7:0/1173674045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 msgr2=0x7f6d401019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.373+0000 7f6d455ff700 1 --2- 172.21.15.7:0/1173674045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d401019c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d300099c0 tx=0x7f6d30009cd0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 -- 172.21.15.7:0/1173674045 shutdown_connections 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 --2- 172.21.15.7:0/1173674045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40103130 0x7f6d401099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 --2- 172.21.15.7:0/1173674045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d401019c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 --2- 172.21.15.7:0/1173674045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d40102790 0x7f6d40102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 -- 172.21.15.7:0/1173674045 >> 172.21.15.7:0/1173674045 conn(0x7f6d400fcb30 msgr2=0x7f6d400fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 -- 172.21.15.7:0/1173674045 shutdown_connections 2024-01-11T17:55:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.374+0000 7f6d455ff700 1 -- 172.21.15.7:0/1173674045 wait complete. 2024-01-11T17:55:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.375+0000 7f6d455ff700 1 Processor -- start 2024-01-11T17:55:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.375+0000 7f6d455ff700 1 -- start start 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d455ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d400753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d455ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 0x7f6d40073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d455ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d40103130 0x7f6d40073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d455ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400746e0 con 0x7f6d401015a0 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d455ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40074850 con 0x7f6d40103130 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d455ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d401a1520 con 0x7f6d40102790 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d400753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 0x7f6d40073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d400753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56654/0 (socket says 172.21.15.7:56654) 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 0x7f6d40073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48458/0 (socket says 172.21.15.7:48458) 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3effd700 1 -- 172.21.15.7:0/361996187 learned_addr learned my addr 172.21.15.7:0/361996187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3f7fe700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d40103130 0x7f6d40073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/361996187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d40103130 msgr2=0x7f6d40073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d40103130 0x7f6d40073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 msgr2=0x7f6d400753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d400753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.376+0000 7f6d3e7fc700 1 -- 172.21.15.7:0/361996187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300095f0 con 0x7f6d40102790 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.377+0000 7f6d3e7fc700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 0x7f6d40073a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400e9f0 tx=0x7f6d3400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.377+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400c9c0 con 0x7f6d40102790 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.377+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d34010430 con 0x7f6d40102790 2024-01-11T17:55:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.377+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34018680 con 0x7f6d40102790 2024-01-11T17:55:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.377+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d401a1800 con 0x7f6d40102790 2024-01-11T17:55:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.377+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d401a1cc0 con 0x7f6d40102790 2024-01-11T17:55:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.379+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40108950 con 0x7f6d40102790 2024-01-11T17:55:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.381+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6d34010860 con 0x7f6d40102790 2024-01-11T17:55:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.382+0000 7f6d27fff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6d280a1f60 0x7f6d280a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.382+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d34014070 con 0x7f6d40102790 2024-01-11T17:55:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.382+0000 7f6d3effd700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6d280a1f60 0x7f6d280a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.383+0000 7f6d3effd700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6d280a1f60 0x7f6d280a4420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000c00 tx=0x7f6d30009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.387+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6d34052380 con 0x7f6d40102790 2024-01-11T17:55:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.667+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d40068440 con 0x7f6d40102790 2024-01-11T17:55:56.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.668+0000 7f6d27fff700 1 -- 172.21.15.7:0/361996187 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7643 (secure 0 0 0) 0x7f6d34096ac0 con 0x7f6d40102790 2024-01-11T17:55:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:56.670 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24565,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24577,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-11T17:55:56.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6d280a1f60 msgr2=0x7f6d280a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:56.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6d280a1f60 0x7f6d280a4420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000c00 tx=0x7f6d30009530 comp rx=0 tx=0).stop 2024-01-11T17:55:56.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 msgr2=0x7f6d40073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:56.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 0x7f6d40073a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400e9f0 tx=0x7f6d3400ed00 comp rx=0 tx=0).stop 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 shutdown_connections 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6d280a1f60 0x7f6d280a4420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d40103130 0x7f6d40073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d401015a0 0x7f6d400753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 --2- 172.21.15.7:0/361996187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40102790 0x7f6d40073a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 >> 172.21.15.7:0/361996187 conn(0x7f6d400fcb30 msgr2=0x7f6d40105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 shutdown_connections 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:56.671+0000 7f6d455ff700 1 -- 172.21.15.7:0/361996187 wait complete. 2024-01-11T17:55:56.674 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-01-11T17:55:56.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: Deploying daemon mds.g on smithi007 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:boot 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: fsmap 9 up:standby 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:56.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:56 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/361996187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:57.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:57.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:57.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:57.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:57.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:57.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:57.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: Deploying daemon mds.g on smithi007 2024-01-11T17:55:57.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:boot 2024-01-11T17:55:57.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: fsmap 9 up:standby 2024-01-11T17:55:57.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-11T17:55:57.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:57.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:56 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/361996187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:57.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:57.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:57.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:57.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: Deploying daemon mds.g on smithi007 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:boot 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: fsmap 9 up:standby 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:57.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:56 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/361996187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:58.322 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 17:55:57 smithi007 systemd[1]: Starting Ceph mds.g for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:55:58.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:55:58.759 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 17:55:58 smithi007 bash[80745]: e104acea60b08ee1f8acfd9a5ae7c9410fcd110b4a04f48c2065760467bfdb1f 2024-01-11T17:55:58.759 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 17:55:58 smithi007 systemd[1]: Started Ceph mds.g for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:55:58.759 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 17:55:58 smithi007 ceph-mds[80783]: -- 172.21.15.7:0/1455189562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591ee8f7520 con 0x5591ee8cec00 2024-01-11T17:55:58.759 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 17:55:58 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-g[80760]: starting mds.g at 2024-01-11T17:55:58.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 -- 172.21.15.7:0/1261099239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334103130 msgr2=0x7f63341099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 --2- 172.21.15.7:0/1261099239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334103130 0x7f63341099e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0d0 tx=0x7f632c00b3e0 comp rx=0 tx=0).stop 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 -- 172.21.15.7:0/1261099239 shutdown_connections 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 --2- 172.21.15.7:0/1261099239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334103130 0x7f63341099e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 --2- 172.21.15.7:0/1261099239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63341015a0 0x7f63341019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 --2- 172.21.15.7:0/1261099239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 -- 172.21.15.7:0/1261099239 >> 172.21.15.7:0/1261099239 conn(0x7f63340fcb30 msgr2=0x7f63340fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 -- 172.21.15.7:0/1261099239 shutdown_connections 2024-01-11T17:55:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.905+0000 7f633c7ff700 1 -- 172.21.15.7:0/1261099239 wait complete. 2024-01-11T17:55:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.906+0000 7f633c7ff700 1 Processor -- start 2024-01-11T17:55:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 -- start start 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63341015a0 0x7f63340753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 0x7f6334073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6334103130 0x7f6334073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340746e0 con 0x7f6334102790 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334074850 con 0x7f6334103130 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.907+0000 7f633c7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341a1600 con 0x7f63341015a0 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 0x7f6334073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f633a59b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63341015a0 0x7f63340753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 0x7f6334073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56692/0 (socket says 172.21.15.7:56692) 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 -- 172.21.15.7:0/3160907882 learned_addr learned my addr 172.21.15.7:0/3160907882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f633ad9c700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6334103130 0x7f6334073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 -- 172.21.15.7:0/3160907882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63341015a0 msgr2=0x7f63340753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63341015a0 0x7f63340753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 -- 172.21.15.7:0/3160907882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6334103130 msgr2=0x7f6334073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6334103130 0x7f6334073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 -- 172.21.15.7:0/3160907882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009520 con 0x7f6334102790 2024-01-11T17:55:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f6339d9a700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 0x7f6334073a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f632400eb30 tx=0x7f632400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324009880 con 0x7f6334102790 2024-01-11T17:55:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.908+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63240044e0 con 0x7f6334102790 2024-01-11T17:55:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.909+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c009d00 con 0x7f6334102790 2024-01-11T17:55:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.909+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324003d80 con 0x7f6334102790 2024-01-11T17:55:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.909+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63341a1cb0 con 0x7f6334102790 2024-01-11T17:55:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.910+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6334108950 con 0x7f6334102790 2024-01-11T17:55:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.912+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6324003f20 con 0x7f6334102790 2024-01-11T17:55:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.914+0000 7f632b7fe700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63200a1f60 0x7f63200a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:55:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.914+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6324014070 con 0x7f6334102790 2024-01-11T17:55:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.914+0000 7f633a59b700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63200a1f60 0x7f63200a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:55:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.915+0000 7f633a59b700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63200a1f60 0x7f63200a4420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63341025f0 tx=0x7f6330009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:55:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:58.918+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6324096eb0 con 0x7f6334102790 2024-01-11T17:55:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.193+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63340683f0 con 0x7f6334102790 2024-01-11T17:55:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.194+0000 7f632b7fe700 1 -- 172.21.15.7:0/3160907882 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8429 (secure 0 0 0) 0x7f632409a4d0 con 0x7f6334102790 2024-01-11T17:55:59.196 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:55:59.196 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24565,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24577,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-11T17:55:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63200a1f60 msgr2=0x7f63200a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63200a1f60 0x7f63200a4420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63341025f0 tx=0x7f6330009260 comp rx=0 tx=0).stop 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 msgr2=0x7f6334073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 0x7f6334073a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f632400eb30 tx=0x7f632400ee40 comp rx=0 tx=0).stop 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 shutdown_connections 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63200a1f60 0x7f63200a4420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63341015a0 0x7f63340753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6334103130 0x7f6334073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 --2- 172.21.15.7:0/3160907882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6334102790 0x7f6334073a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 >> 172.21.15.7:0/3160907882 conn(0x7f63340fcb30 msgr2=0x7f6334105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 shutdown_connections 2024-01-11T17:55:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:55:59.197+0000 7f633c7ff700 1 -- 172.21.15.7:0/3160907882 wait complete. 2024-01-11T17:55:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 11 2024-01-11T17:55:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6838/727242421,v1:172.21.15.7:6839/727242421] up:boot 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: fsmap 10 up:standby 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: Deploying daemon mds.a on smithi007 2024-01-11T17:55:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:55:59 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3160907882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:55:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6838/727242421,v1:172.21.15.7:6839/727242421] up:boot 2024-01-11T17:55:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: fsmap 10 up:standby 2024-01-11T17:55:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-11T17:55:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: Deploying daemon mds.a on smithi007 2024-01-11T17:55:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:55:59 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3160907882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6838/727242421,v1:172.21.15.7:6839/727242421] up:boot 2024-01-11T17:56:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: fsmap 10 up:standby 2024-01-11T17:56:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: Deploying daemon mds.a on smithi007 2024-01-11T17:56:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:55:59 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3160907882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:01.071 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:56:00 smithi007 systemd[1]: Starting Ceph mds.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:56:01.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:56:01.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 -- 172.21.15.7:0/663140067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8102780 msgr2=0x7f84d8102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/663140067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8102780 0x7f84d8102be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0099c0 tx=0x7f84cc009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 -- 172.21.15.7:0/663140067 shutdown_connections 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/663140067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84d8101590 0x7f84d81019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/663140067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8103120 0x7f84d81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/663140067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8102780 0x7f84d8102be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 -- 172.21.15.7:0/663140067 >> 172.21.15.7:0/663140067 conn(0x7f84d80fcb00 msgr2=0x7f84d80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 -- 172.21.15.7:0/663140067 shutdown_connections 2024-01-11T17:56:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.441+0000 7f84dd9ff700 1 -- 172.21.15.7:0/663140067 wait complete. 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 Processor -- start 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 -- start start 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 0x7f84d81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84d8102780 0x7f84d8199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8103120 0x7f84d81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81a1240 con 0x7f84d8103120 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81a1380 con 0x7f84d8101590 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.442+0000 7f84dd9ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81a1510 con 0x7f84d8102780 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 0x7f84d81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 0x7f84d81996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55144/0 (socket says 172.21.15.7:55144) 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 -- 172.21.15.7:0/3118874229 learned_addr learned my addr 172.21.15.7:0/3118874229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d77fe700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8103120 0x7f84d81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d67fc700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84d8102780 0x7f84d8199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 -- 172.21.15.7:0/3118874229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84d8102780 msgr2=0x7f84d8199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84d8102780 0x7f84d8199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 -- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8103120 msgr2=0x7f84d81a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8103120 0x7f84d81a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 -- 172.21.15.7:0/3118874229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc0095f0 con 0x7f84d8101590 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d77fe700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8103120 0x7f84d81a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.443+0000 7f84d6ffd700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 0x7f84d81996c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e970 tx=0x7f84c800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.444+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8009790 con 0x7f84d8101590 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.444+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d81a17f0 con 0x7f84d8101590 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.444+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c800c7e0 con 0x7f84d8101590 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.444+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8003c70 con 0x7f84d8101590 2024-01-11T17:56:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.444+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d81a1d50 con 0x7f84d8101590 2024-01-11T17:56:01.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.445+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d80683f0 con 0x7f84d8101590 2024-01-11T17:56:01.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.447+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f84c800cc50 con 0x7f84d8101590 2024-01-11T17:56:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.450+0000 7f84dc9fd700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f84c00a1ee0 0x7f84c00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.450+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f84c8014070 con 0x7f84d8101590 2024-01-11T17:56:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.450+0000 7f84d67fc700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f84c00a1ee0 0x7f84c00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.450+0000 7f84d67fc700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f84c00a1ee0 0x7f84c00a43a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84cc003730 tx=0x7f84cc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:01.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.451+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f84c8096d30 con 0x7f84d8101590 2024-01-11T17:56:01.571 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:56:01 smithi007 bash[81133]: fea8f7d6a088ab3a11bc43c32ca20aa6a9473a9b81ee028c6680e77fadcfa00e 2024-01-11T17:56:01.572 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:56:01 smithi007 systemd[1]: Started Ceph mds.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:56:01.572 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mds[81185]: -- 172.21.15.7:0/2803251743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fb5b121520 con 0x55fb5b0f8800 2024-01-11T17:56:01.572 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: starting mds.a at 2024-01-11T17:56:01.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.711+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84d810ce00 con 0x7f84d8101590 2024-01-11T17:56:01.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.713+0000 7f84dc9fd700 1 -- 172.21.15.7:0/3118874229 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9215 (secure 0 0 0) 0x7f84c80b8e60 con 0x7f84d8101590 2024-01-11T17:56:01.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:01.714 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24565,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24577,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f84c00a1ee0 msgr2=0x7f84c00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f84c00a1ee0 0x7f84c00a43a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84cc003730 tx=0x7f84cc00b540 comp rx=0 tx=0).stop 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 msgr2=0x7f84d81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 0x7f84d81996c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e970 tx=0x7f84c800ec80 comp rx=0 tx=0).stop 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 shutdown_connections 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f84c00a1ee0 0x7f84c00a43a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d8103120 0x7f84d81a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d8101590 0x7f84d81996c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 --2- 172.21.15.7:0/3118874229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84d8102780 0x7f84d8199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 >> 172.21.15.7:0/3118874229 conn(0x7f84d80fcb00 msgr2=0x7f84d8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 shutdown_connections 2024-01-11T17:56:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:01.715+0000 7f84dd9ff700 1 -- 172.21.15.7:0/3118874229 wait complete. 2024-01-11T17:56:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-01-11T17:56:01.899 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:01.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:01.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:01.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:01.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:56:01.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:56:01.900 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:01 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:56:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:56:02.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:01 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:02.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:02.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-11T17:56:02.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-11T17:56:02.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:01 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:02.827 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:02 smithi016 ceph-mon[43195]: Deploying daemon mds.b on smithi016 2024-01-11T17:56:02.827 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:02 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:boot 2024-01-11T17:56:02.827 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:02 smithi016 ceph-mon[43195]: fsmap 11 up:standby 2024-01-11T17:56:02.827 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:02 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-11T17:56:02.827 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:02 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3118874229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:02.828 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:02 smithi016 systemd[1]: Starting Ceph mds.b for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T17:56:03.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:02 smithi038 ceph-mon[43582]: Deploying daemon mds.b on smithi016 2024-01-11T17:56:03.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:02 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:boot 2024-01-11T17:56:03.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:02 smithi038 ceph-mon[43582]: fsmap 11 up:standby 2024-01-11T17:56:03.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:02 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-11T17:56:03.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:02 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3118874229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:03.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:02 smithi007 ceph-mon[33361]: Deploying daemon mds.b on smithi016 2024-01-11T17:56:03.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:02 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:boot 2024-01-11T17:56:03.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:02 smithi007 ceph-mon[33361]: fsmap 11 up:standby 2024-01-11T17:56:03.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:02 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-11T17:56:03.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:02 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3118874229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:03.636 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:03 smithi016 bash[80193]: 0468188a2c19f39cab00754da63c6a75695af8f7daa7a745a67fd086d19c0415 2024-01-11T17:56:03.637 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:03 smithi016 systemd[1]: Started Ceph mds.b for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T17:56:03.637 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mds[80231]: -- 172.21.15.16:0/1383554980 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c98aa9520 con 0x560c98a7ec00 2024-01-11T17:56:03.637 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: starting mds.b at 2024-01-11T17:56:03.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:56:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 -- 172.21.15.7:0/608684021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4075650 msgr2=0x7ff2c4073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/608684021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4075650 0x7ff2c4073aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009990 tx=0x7ff2b4009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 -- 172.21.15.7:0/608684021 shutdown_connections 2024-01-11T17:56:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/608684021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4075650 0x7ff2c4073aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/608684021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c410db60 0x7ff2c410ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/608684021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2c4074070 0x7ff2c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 -- 172.21.15.7:0/608684021 >> 172.21.15.7:0/608684021 conn(0x7ff2c40fcb00 msgr2=0x7ff2c40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 -- 172.21.15.7:0/608684021 shutdown_connections 2024-01-11T17:56:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.943+0000 7ff2c97ff700 1 -- 172.21.15.7:0/608684021 wait complete. 2024-01-11T17:56:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 Processor -- start 2024-01-11T17:56:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 -- start start 2024-01-11T17:56:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 0x7ff2c419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4075650 0x7ff2c419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2c410db60 0x7ff2c41a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41a55b0 con 0x7ff2c4075650 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.945+0000 7ff2c97ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41a56f0 con 0x7ff2c410db60 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c97ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41a5880 con 0x7ff2c4074070 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 0x7ff2c419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 0x7ff2c419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:34524/0 (socket says 172.21.15.7:34524) 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4075650 0x7ff2c419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 -- 172.21.15.7:0/2628947047 learned_addr learned my addr 172.21.15.7:0/2628947047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c37fe700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2c410db60 0x7ff2c41a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 -- 172.21.15.7:0/2628947047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2c410db60 msgr2=0x7ff2c41a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2c410db60 0x7ff2c41a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 -- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4075650 msgr2=0x7ff2c419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4075650 0x7ff2c419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 -- 172.21.15.7:0/2628947047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40095f0 con 0x7ff2c4074070 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.946+0000 7ff2c2ffd700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 0x7ff2c419da80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4005910 tx=0x7ff2b4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.947+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b401d070 con 0x7ff2c4074070 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.947+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c41a5b00 con 0x7ff2c4074070 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.947+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b4021410 con 0x7ff2c4074070 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.947+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4017650 con 0x7ff2c4074070 2024-01-11T17:56:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.947+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c41a5f30 con 0x7ff2c4074070 2024-01-11T17:56:03.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.948+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c4103dc0 con 0x7ff2c4074070 2024-01-11T17:56:03.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.950+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7ff2b4021580 con 0x7ff2c4074070 2024-01-11T17:56:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.951+0000 7ff2abfff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2ac0a1ee0 0x7ff2ac0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.951+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2b40ce950 con 0x7ff2c4074070 2024-01-11T17:56:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.951+0000 7ff2c27fc700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2ac0a1ee0 0x7ff2ac0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.952+0000 7ff2c27fc700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2ac0a1ee0 0x7ff2ac0a43a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4075370 tx=0x7ff2b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:03.954+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff2b4097e80 con 0x7ff2c4074070 2024-01-11T17:56:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:03.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:03 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:56:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:03 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:56:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:03 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:56:04.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.235+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2c40683f0 con 0x7ff2c4074070 2024-01-11T17:56:04.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.236+0000 7ff2abfff700 1 -- 172.21.15.7:0/2628947047 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10007 (secure 0 0 0) 0x7ff2b409b4a0 con 0x7ff2c4074070 2024-01-11T17:56:04.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:04.238 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24565,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24577,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2ac0a1ee0 msgr2=0x7ff2ac0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2ac0a1ee0 0x7ff2ac0a43a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4075370 tx=0x7ff2b8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 msgr2=0x7ff2c419da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 0x7ff2c419da80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4005910 tx=0x7ff2b4009580 comp rx=0 tx=0).stop 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 shutdown_connections 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2ac0a1ee0 0x7ff2ac0a43a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2c4074070 0x7ff2c419da80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2c410db60 0x7ff2c41a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 --2- 172.21.15.7:0/2628947047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c4075650 0x7ff2c419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 >> 172.21.15.7:0/2628947047 conn(0x7ff2c40fcb00 msgr2=0x7ff2c4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:04.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 shutdown_connections 2024-01-11T17:56:04.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.240+0000 7ff2c97ff700 1 -- 172.21.15.7:0/2628947047 wait complete. 2024-01-11T17:56:04.243 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-01-11T17:56:04.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 -- 172.21.15.7:0/2184641518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc1049c0 msgr2=0x7f66fc106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 --2- 172.21.15.7:0/2184641518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc1049c0 0x7f66fc106db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009990 tx=0x7f66ec009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 -- 172.21.15.7:0/2184641518 shutdown_connections 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 --2- 172.21.15.7:0/2184641518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1072f0 0x7f66fc1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 --2- 172.21.15.7:0/2184641518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc102090 0x7f66fc104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 --2- 172.21.15.7:0/2184641518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc1049c0 0x7f66fc106db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.486+0000 7f6703fff700 1 -- 172.21.15.7:0/2184641518 >> 172.21.15.7:0/2184641518 conn(0x7f66fc0fba00 msgr2=0x7f66fc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.487+0000 7f6703fff700 1 -- 172.21.15.7:0/2184641518 shutdown_connections 2024-01-11T17:56:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.487+0000 7f6703fff700 1 -- 172.21.15.7:0/2184641518 wait complete. 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 Processor -- start 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 -- start start 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc102090 0x7f66fc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 0x7f66fc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc1072f0 0x7f66fc1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a1210 con 0x7f66fc1072f0 2024-01-11T17:56:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a1350 con 0x7f66fc102090 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6703fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a14e0 con 0x7f66fc1049c0 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f670159a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 0x7f66fc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f670159a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 0x7f66fc199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:34540/0 (socket says 172.21.15.7:34540) 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f670159a700 1 -- 172.21.15.7:0/3865039422 learned_addr learned my addr 172.21.15.7:0/3865039422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f6701d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc102090 0x7f66fc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.488+0000 7f670259c700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc1072f0 0x7f66fc1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f670159a700 1 -- 172.21.15.7:0/3865039422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc102090 msgr2=0x7f66fc199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f670159a700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc102090 0x7f66fc199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f670159a700 1 -- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc1072f0 msgr2=0x7f66fc1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f670159a700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc1072f0 0x7f66fc1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f670159a700 1 -- 172.21.15.7:0/3865039422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec0095f0 con 0x7f66fc1049c0 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f670159a700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 0x7f66fc199bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b680 tx=0x7f66ec009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01d070 con 0x7f66fc1049c0 2024-01-11T17:56:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ec022470 con 0x7f66fc1049c0 2024-01-11T17:56:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc1a1760 con 0x7f66fc1049c0 2024-01-11T17:56:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.489+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec017650 con 0x7f66fc1049c0 2024-01-11T17:56:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.490+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc1a1b80 con 0x7f66fc1049c0 2024-01-11T17:56:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.491+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc108e70 con 0x7f66fc1049c0 2024-01-11T17:56:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.493+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f66ec0225e0 con 0x7f66fc1049c0 2024-01-11T17:56:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.494+0000 7f66f2ffd700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f66e80a1ee0 0x7f66e80a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.494+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66ec0cedd0 con 0x7f66fc1049c0 2024-01-11T17:56:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.495+0000 7f6701d9b700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f66e80a1ee0 0x7f66e80a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.495+0000 7f6701d9b700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f66e80a1ee0 0x7f66e80a43a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009c60 tx=0x7f66f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.497+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f66ec098300 con 0x7f66fc1049c0 2024-01-11T17:56:04.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.716+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66fc10cde0 con 0x7f66fc1049c0 2024-01-11T17:56:04.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.717+0000 7f66f2ffd700 1 -- 172.21.15.7:0/3865039422 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f66ec09b920 con 0x7f66fc1049c0 2024-01-11T17:56:04.719 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:04.719 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:55:29.931492+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:04.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.719+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f66e80a1ee0 msgr2=0x7f66e80a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.719+0000 7f6703fff700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f66e80a1ee0 0x7f66e80a43a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009c60 tx=0x7f66f8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:04.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.719+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 msgr2=0x7f66fc199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.719+0000 7f6703fff700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 0x7f66fc199bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b680 tx=0x7f66ec009580 comp rx=0 tx=0).stop 2024-01-11T17:56:04.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 shutdown_connections 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f66e80a1ee0 0x7f66e80a43a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc1072f0 0x7f66fc1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc102090 0x7f66fc199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 --2- 172.21.15.7:0/3865039422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66fc1049c0 0x7f66fc199bd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 >> 172.21.15.7:0/3865039422 conn(0x7f66fc0fba00 msgr2=0x7f66fc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 shutdown_connections 2024-01-11T17:56:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.720+0000 7f6703fff700 1 -- 172.21.15.7:0/3865039422 wait complete. 2024-01-11T17:56:04.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-11T17:56:04.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:56:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 -- 172.21.15.7:0/3962676671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 msgr2=0x7fa5ec10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/3962676671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec10b900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400b0a0 tx=0x7fa5e400b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 -- 172.21.15.7:0/3962676671 shutdown_connections 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/3962676671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec10b900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/3962676671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec075650 0x7fa5ec073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/3962676671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5ec074070 0x7fa5ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 -- 172.21.15.7:0/3962676671 >> 172.21.15.7:0/3962676671 conn(0x7fa5ec0fcb00 msgr2=0x7fa5ec0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 -- 172.21.15.7:0/3962676671 shutdown_connections 2024-01-11T17:56:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.987+0000 7fa5f0dff700 1 -- 172.21.15.7:0/3962676671 wait complete. 2024-01-11T17:56:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 Processor -- start 2024-01-11T17:56:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 -- start start 2024-01-11T17:56:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 0x7fa5ec199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5ec075650 0x7fa5ec199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1a12f0 con 0x7fa5ec109470 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1a1430 con 0x7fa5ec074070 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.988+0000 7fa5f0dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1a15c0 con 0x7fa5ec075650 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 0x7fa5ec199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 0x7fa5ec199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46400/0 (socket says 172.21.15.7:46400) 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 -- 172.21.15.7:0/980239166 learned_addr learned my addr 172.21.15.7:0/980239166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5ec075650 0x7fa5ec199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 -- 172.21.15.7:0/980239166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5ec075650 msgr2=0x7fa5ec199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5ec075650 0x7fa5ec199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 -- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 msgr2=0x7fa5ec1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ea59c700 1 -- 172.21.15.7:0/980239166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e4009d00 con 0x7fa5ec074070 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.989+0000 7fa5ead9d700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.990+0000 7fa5ea59c700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 0x7fa5ec199770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e9a0 tx=0x7fa5dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.990+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc009790 con 0x7fa5ec074070 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.990+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5dc010430 con 0x7fa5ec074070 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.990+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc018680 con 0x7fa5ec074070 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.990+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec1a18a0 con 0x7fa5ec074070 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.990+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec1a1e00 con 0x7fa5ec074070 2024-01-11T17:56:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.991+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec108230 con 0x7fa5ec074070 2024-01-11T17:56:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.993+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa5dc0105a0 con 0x7fa5ec074070 2024-01-11T17:56:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.994+0000 7fa5db7fe700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5d40a1f60 0x7fa5d40a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.995+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5dc014070 con 0x7fa5ec074070 2024-01-11T17:56:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.995+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5d40a1f60 0x7fa5d40a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:04.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:04 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] up:boot 2024-01-11T17:56:04.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:04 smithi016 ceph-mon[43195]: fsmap 12 up:standby 2024-01-11T17:56:04.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:04 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-11T17:56:04.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:04 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2628947047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:04.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:04 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.995+0000 7fa5e9d9b700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5d40a1f60 0x7fa5d40a4420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec075370 tx=0x7fa5e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:05.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:04.999+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa5dc052380 con 0x7fa5ec074070 2024-01-11T17:56:05.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:04 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] up:boot 2024-01-11T17:56:05.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:04 smithi038 ceph-mon[43582]: fsmap 12 up:standby 2024-01-11T17:56:05.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:04 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-11T17:56:05.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:04 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2628947047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:05.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:04 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:04 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] up:boot 2024-01-11T17:56:05.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:04 smithi007 ceph-mon[33361]: fsmap 12 up:standby 2024-01-11T17:56:05.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:04 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-11T17:56:05.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:04 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2628947047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:05.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:04 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.224+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5ec02d450 con 0x7fa5ec074070 2024-01-11T17:56:05.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.225+0000 7fa5db7fe700 1 -- 172.21.15.7:0/980239166 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5dc096ba0 con 0x7fa5ec074070 2024-01-11T17:56:05.226 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5d40a1f60 msgr2=0x7fa5d40a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5d40a1f60 0x7fa5d40a4420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec075370 tx=0x7fa5e0009260 comp rx=0 tx=0).stop 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 msgr2=0x7fa5ec199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 0x7fa5ec199770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00e9a0 tx=0x7fa5dc00ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 shutdown_connections 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5d40a1f60 0x7fa5d40a4420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5ec109470 0x7fa5ec1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5ec074070 0x7fa5ec199770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 --2- 172.21.15.7:0/980239166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5ec075650 0x7fa5ec199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.227+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 >> 172.21.15.7:0/980239166 conn(0x7fa5ec0fcb00 msgr2=0x7fa5ec105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.228+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 shutdown_connections 2024-01-11T17:56:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.228+0000 7fa5f0dff700 1 -- 172.21.15.7:0/980239166 wait complete. 2024-01-11T17:56:05.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 -- 172.21.15.7:0/536811037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 msgr2=0x7fa2340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/536811037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa2340744f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa224009990 tx=0x7fa224009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 -- 172.21.15.7:0/536811037 shutdown_connections 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/536811037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa234075650 0x7fa234073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/536811037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa23410db70 0x7fa23410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/536811037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa2340744f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 -- 172.21.15.7:0/536811037 >> 172.21.15.7:0/536811037 conn(0x7fa2340fcb30 msgr2=0x7fa2340fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 -- 172.21.15.7:0/536811037 shutdown_connections 2024-01-11T17:56:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.474+0000 7fa23c7ff700 1 -- 172.21.15.7:0/536811037 wait complete. 2024-01-11T17:56:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.476+0000 7fa23c7ff700 1 Processor -- start 2024-01-11T17:56:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.476+0000 7fa23c7ff700 1 -- start start 2024-01-11T17:56:05.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.476+0000 7fa23c7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa23419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.476+0000 7fa23c7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234075650 0x7fa23419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23c7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23410db70 0x7fa2341a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23c7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341a5630 con 0x7fa234074070 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23c7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341a5770 con 0x7fa234075650 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23c7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341a5900 con 0x7fa23410db70 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa23419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa23419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46220/0 (socket says 172.21.15.7:46220) 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 -- 172.21.15.7:0/277706752 learned_addr learned my addr 172.21.15.7:0/277706752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23ad9c700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23410db70 0x7fa2341a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa239d9a700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234075650 0x7fa23419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 -- 172.21.15.7:0/277706752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23410db70 msgr2=0x7fa2341a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23410db70 0x7fa2341a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 -- 172.21.15.7:0/277706752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234075650 msgr2=0x7fa23419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234075650 0x7fa23419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.477+0000 7fa23a59b700 1 -- 172.21.15.7:0/277706752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240095f0 con 0x7fa234074070 2024-01-11T17:56:05.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.478+0000 7fa23a59b700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa23419dab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa23000e9a0 tx=0x7fa23000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:05.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.478+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230009790 con 0x7fa234074070 2024-01-11T17:56:05.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.478+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa230010430 con 0x7fa234074070 2024-01-11T17:56:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.478+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341a5be0 con 0x7fa234074070 2024-01-11T17:56:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.478+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230018680 con 0x7fa234074070 2024-01-11T17:56:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.478+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341a6140 con 0x7fa234074070 2024-01-11T17:56:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.479+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa234103dd0 con 0x7fa234074070 2024-01-11T17:56:05.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.481+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa2300105a0 con 0x7fa234074070 2024-01-11T17:56:05.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.484+0000 7fa22b7fe700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2200a1f60 0x7fa2200a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.484+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa230014070 con 0x7fa234074070 2024-01-11T17:56:05.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.484+0000 7fa239d9a700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2200a1f60 0x7fa2200a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.485+0000 7fa239d9a700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2200a1f60 0x7fa2200a4420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b680 tx=0x7fa224005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:05.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.487+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa230096b10 con 0x7fa234074070 2024-01-11T17:56:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.712+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa2340683f0 con 0x7fa234074070 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.712+0000 7fa22b7fe700 1 -- 172.21.15.7:0/277706752 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa23009a130 con 0x7fa234074070 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:56:05.714 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2200a1f60 msgr2=0x7fa2200a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2200a1f60 0x7fa2200a4420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b680 tx=0x7fa224005fd0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 msgr2=0x7fa23419dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa23419dab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa23000e9a0 tx=0x7fa23000ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 shutdown_connections 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2200a1f60 0x7fa2200a4420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23410db70 0x7fa2341a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa234074070 0x7fa23419dab0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 --2- 172.21.15.7:0/277706752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa234075650 0x7fa23419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 >> 172.21.15.7:0/277706752 conn(0x7fa2340fcb30 msgr2=0x7fa234100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 shutdown_connections 2024-01-11T17:56:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.715+0000 7fa23c7ff700 1 -- 172.21.15.7:0/277706752 wait complete. 2024-01-11T17:56:05.737 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T17:56:05.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T17:56:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 -- 172.21.15.7:0/33552668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a01007c0 msgr2=0x7f02a0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 --2- 172.21.15.7:0/33552668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a01007c0 0x7f02a0100c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0290009990 tx=0x7f0290009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 -- 172.21.15.7:0/33552668 shutdown_connections 2024-01-11T17:56:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 --2- 172.21.15.7:0/33552668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a0101180 0x7f02a0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 --2- 172.21.15.7:0/33552668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00ffe60 0x7f02a0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 --2- 172.21.15.7:0/33552668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a01007c0 0x7f02a0100c40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 -- 172.21.15.7:0/33552668 >> 172.21.15.7:0/33552668 conn(0x7f02a00fba40 msgr2=0x7f02a00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 -- 172.21.15.7:0/33552668 shutdown_connections 2024-01-11T17:56:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.950+0000 7f02a7dff700 1 -- 172.21.15.7:0/33552668 wait complete. 2024-01-11T17:56:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.951+0000 7f02a7dff700 1 Processor -- start 2024-01-11T17:56:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.951+0000 7f02a7dff700 1 -- start start 2024-01-11T17:56:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.951+0000 7f02a7dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 0x7f02a0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.951+0000 7f02a7dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a01007c0 0x7f02a0199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a7dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a0101180 0x7f02a01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a7dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01a11c0 con 0x7f02a00ffe60 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a7dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01a1300 con 0x7f02a01007c0 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a7dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01a1490 con 0x7f02a0101180 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a5b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 0x7f02a0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a01007c0 0x7f02a0199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a5b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 0x7f02a0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46244/0 (socket says 172.21.15.7:46244) 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a5b9b700 1 -- 172.21.15.7:0/4157890140 learned_addr learned my addr 172.21.15.7:0/4157890140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a639c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a0101180 0x7f02a01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 -- 172.21.15.7:0/4157890140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a0101180 msgr2=0x7f02a01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a0101180 0x7f02a01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 -- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 msgr2=0x7f02a0199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 0x7f02a0199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 -- 172.21.15.7:0/4157890140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900095f0 con 0x7f02a01007c0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a5b9b700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 0x7f02a0199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.952+0000 7f02a539a700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a01007c0 0x7f02a0199b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0290000c00 tx=0x7f0290004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.953+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029001d070 con 0x7f02a01007c0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.953+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0290021410 con 0x7f02a01007c0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.953+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a01a1710 con 0x7f02a01007c0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.953+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290021ea0 con 0x7f02a01007c0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.953+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a01a1b90 con 0x7f02a01007c0 2024-01-11T17:56:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.954+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a010ae30 con 0x7f02a01007c0 2024-01-11T17:56:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.956+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0290021580 con 0x7f02a01007c0 2024-01-11T17:56:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.957+0000 7f0296ffd700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f028c0a1f60 0x7f028c0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.957+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02900ce550 con 0x7f02a01007c0 2024-01-11T17:56:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.958+0000 7f02a5b9b700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f028c0a1f60 0x7f028c0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.958+0000 7f02a5b9b700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f028c0a1f60 0x7f028c0a4420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f029c011550 tx=0x7f029c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:05.960+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0290097a80 con 0x7f02a01007c0 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3865039422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/980239166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:05.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:05.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:05 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3865039422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/980239166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:05 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:06.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3865039422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:06.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/980239166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:06.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:05 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:06.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.179+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f02a010f0f0 con 0x7f02a01007c0 2024-01-11T17:56:06.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.747+0000 7f0296ffd700 1 -- 172.21.15.7:0/4157890140 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f029009b0a0 con 0x7f02a01007c0 2024-01-11T17:56:06.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f028c0a1f60 msgr2=0x7f028c0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:06.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f028c0a1f60 0x7f028c0a4420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f029c011550 tx=0x7f029c009260 comp rx=0 tx=0).stop 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a01007c0 msgr2=0x7f02a0199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a01007c0 0x7f02a0199b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0290000c00 tx=0x7f0290004730 comp rx=0 tx=0).stop 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 shutdown_connections 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f028c0a1f60 0x7f028c0a4420 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f029c011550 tx=0x7f029c009260 comp rx=0 tx=0).stop 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02a0101180 0x7f02a01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a00ffe60 0x7f02a0199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 --2- 172.21.15.7:0/4157890140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a01007c0 0x7f02a0199b80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.753+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 >> 172.21.15.7:0/4157890140 conn(0x7f02a00fba40 msgr2=0x7f02a00fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.754+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 shutdown_connections 2024-01-11T17:56:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:06.754+0000 7f02a7dff700 1 -- 172.21.15.7:0/4157890140 wait complete. 2024-01-11T17:56:06.757 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T17:56:06.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-01-11T17:56:06.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:06 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/277706752' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 -- 172.21.15.7:0/1110190075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 msgr2=0x7f7f6c073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 --2- 172.21.15.7:0/1110190075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c073aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009990 tx=0x7f7f5c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 -- 172.21.15.7:0/1110190075 shutdown_connections 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 --2- 172.21.15.7:0/1110190075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c073aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 --2- 172.21.15.7:0/1110190075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c109470 0x7f7f6c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 --2- 172.21.15.7:0/1110190075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c074070 0x7f7f6c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 -- 172.21.15.7:0/1110190075 >> 172.21.15.7:0/1110190075 conn(0x7f7f6c0fcb00 msgr2=0x7f7f6c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 -- 172.21.15.7:0/1110190075 shutdown_connections 2024-01-11T17:56:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.001+0000 7f7f70fff700 1 -- 172.21.15.7:0/1110190075 wait complete. 2024-01-11T17:56:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.002+0000 7f7f70fff700 1 Processor -- start 2024-01-11T17:56:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 -- start start 2024-01-11T17:56:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c074070 0x7f7f6c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c109470 0x7f7f6c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1a11c0 con 0x7f7f6c074070 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1a1300 con 0x7f7f6c109470 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f70fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1a1490 con 0x7f7f6c075650 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f63fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f63fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:34618/0 (socket says 172.21.15.7:34618) 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f63fff700 1 -- 172.21.15.7:0/61988949 learned_addr learned my addr 172.21.15.7:0/61988949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c074070 0x7f7f6c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.003+0000 7f7f6ad9d700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c109470 0x7f7f6c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f63fff700 1 -- 172.21.15.7:0/61988949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c109470 msgr2=0x7f7f6c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f63fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c109470 0x7f7f6c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f63fff700 1 -- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c074070 msgr2=0x7f7f6c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f63fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c074070 0x7f7f6c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f63fff700 1 -- 172.21.15.7:0/61988949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0095f0 con 0x7f7f6c075650 2024-01-11T17:56:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f63fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c199bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e970 tx=0x7f7f5400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54009790 con 0x7f7f6c075650 2024-01-11T17:56:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f5400c7e0 con 0x7f7f6c075650 2024-01-11T17:56:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c1a1770 con 0x7f7f6c075650 2024-01-11T17:56:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54003c70 con 0x7f7f6c075650 2024-01-11T17:56:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.004+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c1a1cd0 con 0x7f7f6c075650 2024-01-11T17:56:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.005+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c108230 con 0x7f7f6c075650 2024-01-11T17:56:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.008+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f7f5400cc50 con 0x7f7f6c075650 2024-01-11T17:56:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.009+0000 7f7f637fe700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7f580a1f60 0x7f7f580a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.009+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7f54014070 con 0x7f7f6c075650 2024-01-11T17:56:07.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.009+0000 7f7f6a59c700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7f580a1f60 0x7f7f580a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:07.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.010+0000 7f7f6a59c700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7f580a1f60 0x7f7f580a4420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006010 tx=0x7f7f5c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:07.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.011+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f7f54096cc0 con 0x7f7f6c075650 2024-01-11T17:56:07.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:06 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/277706752' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-11T17:56:07.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:06 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/277706752' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-11T17:56:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.231+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f7f6c0683f0 con 0x7f7f6c075650 2024-01-11T17:56:07.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.787+0000 7f7f637fe700 1 -- 172.21.15.7:0/61988949 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) v1 ==== 171+0+0 (secure 0 0 0) 0x7f7f5409a2e0 con 0x7f7f6c075650 2024-01-11T17:56:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.796+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7f580a1f60 msgr2=0x7f7f580a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.796+0000 7f7f70fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7f580a1f60 0x7f7f580a4420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006010 tx=0x7f7f5c00b260 comp rx=0 tx=0).stop 2024-01-11T17:56:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.796+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 msgr2=0x7f7f6c199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.796+0000 7f7f70fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c199bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e970 tx=0x7f7f5400ec80 comp rx=0 tx=0).stop 2024-01-11T17:56:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.796+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 shutdown_connections 2024-01-11T17:56:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7f580a1f60 0x7f7f580a4420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c109470 0x7f7f6c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c074070 0x7f7f6c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 --2- 172.21.15.7:0/61988949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f6c075650 0x7f7f6c199bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 >> 172.21.15.7:0/61988949 conn(0x7f7f6c0fcb00 msgr2=0x7f7f6c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 shutdown_connections 2024-01-11T17:56:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:07.797+0000 7f7f70fff700 1 -- 172.21.15.7:0/61988949 wait complete. 2024-01-11T17:56:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T17:56:07.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T17:56:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4157890140' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-11T17:56:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-11T17:56:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: osdmap e77: 12 total, 12 up, 12 in 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:56:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/61988949' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-01-11T17:56:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 -- 172.21.15.7:0/2170345078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 msgr2=0x7fe6901015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 --2- 172.21.15.7:0/2170345078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901015a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b0a0 tx=0x7fe68800b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 -- 172.21.15.7:0/2170345078 shutdown_connections 2024-01-11T17:56:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 --2- 172.21.15.7:0/2170345078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901015a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 --2- 172.21.15.7:0/2170345078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6900ffe00 0x7fe690100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 --2- 172.21.15.7:0/2170345078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 0x7fe690100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 -- 172.21.15.7:0/2170345078 >> 172.21.15.7:0/2170345078 conn(0x7fe6900fba00 msgr2=0x7fe6900fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 -- 172.21.15.7:0/2170345078 shutdown_connections 2024-01-11T17:56:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.044+0000 7fe6949ff700 1 -- 172.21.15.7:0/2170345078 wait complete. 2024-01-11T17:56:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.045+0000 7fe6949ff700 1 Processor -- start 2024-01-11T17:56:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 -- start start 2024-01-11T17:56:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6900ffe00 0x7fe690199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 0x7fe690199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a12e0 con 0x7fe6900ffe00 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a1420 con 0x7fe690100760 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe6949ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6901a15b0 con 0x7fe690101120 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 0x7fe690199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6900ffe00 0x7fe690199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:34648/0 (socket says 172.21.15.7:34648) 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 0x7fe690199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46494/0 (socket says 172.21.15.7:46494) 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 -- 172.21.15.7:0/865355576 learned_addr learned my addr 172.21.15.7:0/865355576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 -- 172.21.15.7:0/865355576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 msgr2=0x7fe690199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 0x7fe690199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 -- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6900ffe00 msgr2=0x7fe690199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.046+0000 7fe68ed9d700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6900ffe00 0x7fe690199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe68ed9d700 1 -- 172.21.15.7:0/865355576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009d00 con 0x7fe690101120 2024-01-11T17:56:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe68ed9d700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901a0d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe688007a60 tx=0x7fe688007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68800e040 con 0x7fe690101120 2024-01-11T17:56:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe688004c50 con 0x7fe690101120 2024-01-11T17:56:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68801b6c0 con 0x7fe690101120 2024-01-11T17:56:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6901a1830 con 0x7fe690101120 2024-01-11T17:56:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.047+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6901a1c50 con 0x7fe690101120 2024-01-11T17:56:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.048+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69010abe0 con 0x7fe690101120 2024-01-11T17:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.050+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fe68801d470 con 0x7fe690101120 2024-01-11T17:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.051+0000 7fe67f7fe700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe6780a1ee0 0x7fe6780a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.052+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fe6880cecc0 con 0x7fe690101120 2024-01-11T17:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.052+0000 7fe68e59c700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe6780a1ee0 0x7fe6780a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.052+0000 7fe68e59c700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe6780a1ee0 0x7fe6780a43a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe680009c60 tx=0x7fe680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.056+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe688097f00 con 0x7fe690101120 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4157890140' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-11T17:56:08.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: osdmap e77: 12 total, 12 up, 12 in 2024-01-11T17:56:08.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:56:08.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/61988949' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4157890140' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:08.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-11T17:56:08.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: osdmap e77: 12 total, 12 up, 12 in 2024-01-11T17:56:08.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:56:08.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/61988949' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-01-11T17:56:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:08.335+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe69010efc0 con 0x7fe690101120 2024-01-11T17:56:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: osdmap e78: 12 total, 12 up, 12 in 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/865355576' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-11T17:56:09.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:08 smithi038 ceph-mon[43582]: pgmap v240: 97 pgs: 4 creating+peering, 7 active+clean, 86 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-01-11T17:56:09.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: osdmap e78: 12 total, 12 up, 12 in 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/865355576' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-11T17:56:09.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:08 smithi007 ceph-mon[33361]: pgmap v240: 97 pgs: 4 creating+peering, 7 active+clean, 86 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.237+0000 7fe67f7fe700 1 -- 172.21.15.7:0/865355576 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe68809b520 con 0x7fe690101120 2024-01-11T17:56:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.242+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe6780a1ee0 msgr2=0x7fe6780a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.242+0000 7fe6949ff700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe6780a1ee0 0x7fe6780a43a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe680009c60 tx=0x7fe680009260 comp rx=0 tx=0).stop 2024-01-11T17:56:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.242+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 msgr2=0x7fe6901a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.242+0000 7fe6949ff700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901a0d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe688007a60 tx=0x7fe688007930 comp rx=0 tx=0).stop 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.242+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 shutdown_connections 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe6780a1ee0 0x7fe6780a43a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe690101120 0x7fe6901a0d10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6900ffe00 0x7fe690199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 --2- 172.21.15.7:0/865355576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690100760 0x7fe690199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 >> 172.21.15.7:0/865355576 conn(0x7fe6900fba00 msgr2=0x7fe6900fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 shutdown_connections 2024-01-11T17:56:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.243+0000 7fe6949ff700 1 -- 172.21.15.7:0/865355576 wait complete. 2024-01-11T17:56:09.246 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-01-11T17:56:09.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: osdmap e78: 12 total, 12 up, 12 in 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/865355576' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-11T17:56:09.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:08 smithi016 ceph-mon[43195]: pgmap v240: 97 pgs: 4 creating+peering, 7 active+clean, 86 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T17:56:09.265 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-01-11T17:56:09.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 -- 172.21.15.7:0/2438936661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 msgr2=0x7f473c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 --2- 172.21.15.7:0/2438936661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c0744f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4730009990 tx=0x7f4730009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 -- 172.21.15.7:0/2438936661 shutdown_connections 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 --2- 172.21.15.7:0/2438936661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f473c075650 0x7f473c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 --2- 172.21.15.7:0/2438936661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473c109470 0x7f473c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 --2- 172.21.15.7:0/2438936661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c0744f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 -- 172.21.15.7:0/2438936661 >> 172.21.15.7:0/2438936661 conn(0x7f473c0fcb00 msgr2=0x7f473c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 -- 172.21.15.7:0/2438936661 shutdown_connections 2024-01-11T17:56:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.475+0000 7f4741fff700 1 -- 172.21.15.7:0/2438936661 wait complete. 2024-01-11T17:56:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 Processor -- start 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 -- start start 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473c075650 0x7f473c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f473c109470 0x7f473c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1a1270 con 0x7f473c074070 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1a13b0 con 0x7f473c075650 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.476+0000 7f4741fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1a1540 con 0x7f473c109470 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46300/0 (socket says 172.21.15.7:46300) 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 -- 172.21.15.7:0/459784198 learned_addr learned my addr 172.21.15.7:0/459784198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473affd700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473c075650 0x7f473c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473bfff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f473c109470 0x7f473c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 -- 172.21.15.7:0/459784198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f473c109470 msgr2=0x7f473c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f473c109470 0x7f473c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 -- 172.21.15.7:0/459784198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473c075650 msgr2=0x7f473c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473c075650 0x7f473c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 -- 172.21.15.7:0/459784198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300095f0 con 0x7f473c074070 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.477+0000 7f473b7fe700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c199740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f472c00e9a0 tx=0x7f472c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.478+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c009790 con 0x7f473c074070 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.478+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472c00cbd0 con 0x7f473c074070 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.478+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c1a1820 con 0x7f473c074070 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.478+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c018680 con 0x7f473c074070 2024-01-11T17:56:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.478+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c1a1d80 con 0x7f473c074070 2024-01-11T17:56:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.479+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c108230 con 0x7f473c074070 2024-01-11T17:56:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.481+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f472c00cd40 con 0x7f473c074070 2024-01-11T17:56:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.483+0000 7f4738ff9700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47240a1f60 0x7f47240a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.483+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f472c014070 con 0x7f473c074070 2024-01-11T17:56:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.483+0000 7f473affd700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47240a1f60 0x7f47240a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.483+0000 7f473affd700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47240a1f60 0x7f47240a4420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f473000b680 tx=0x7f4730005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.485+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f472c096f60 con 0x7f473c074070 2024-01-11T17:56:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.709+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f473c10f020 con 0x7f473c074070 2024-01-11T17:56:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.781+0000 7f4738ff9700 1 -- 172.21.15.7:0/459784198 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) v1 ==== 157+0+0 (secure 0 0 0) 0x7f472c0260e0 con 0x7f473c074070 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.783+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47240a1f60 msgr2=0x7f47240a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.783+0000 7f4741fff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47240a1f60 0x7f47240a4420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f473000b680 tx=0x7f4730005fd0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.783+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 msgr2=0x7f473c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.783+0000 7f4741fff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c199740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f472c00e9a0 tx=0x7f472c00ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 shutdown_connections 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47240a1f60 0x7f47240a4420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f473c109470 0x7f473c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f473c074070 0x7f473c199740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 --2- 172.21.15.7:0/459784198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473c075650 0x7f473c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 >> 172.21.15.7:0/459784198 conn(0x7f473c0fcb00 msgr2=0x7f473c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 shutdown_connections 2024-01-11T17:56:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:09.784+0000 7f4741fff700 1 -- 172.21.15.7:0/459784198 wait complete. 2024-01-11T17:56:09.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 -- 172.21.15.7:0/3478949603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 msgr2=0x7f340810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3478949603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f340810b8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f340000b0a0 tx=0x7f340000b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 -- 172.21.15.7:0/3478949603 shutdown_connections 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3478949603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3408075650 0x7f3408073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3478949603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f340810b8d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3478949603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f34080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 -- 172.21.15.7:0/3478949603 >> 172.21.15.7:0/3478949603 conn(0x7f34080fcad0 msgr2=0x7f34080fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 -- 172.21.15.7:0/3478949603 shutdown_connections 2024-01-11T17:56:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.015+0000 7f340e7ff700 1 -- 172.21.15.7:0/3478949603 wait complete. 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.016+0000 7f340e7ff700 1 Processor -- start 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.016+0000 7f340e7ff700 1 -- start start 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340e7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f3408199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340e7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3408075650 0x7f3408199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f34081a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34081a12c0 con 0x7f3408109440 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340e7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34081a1400 con 0x7f3408074070 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340e7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34081a1590 con 0x7f3408075650 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f3407fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f3408199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f34081a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f3407fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f3408199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46524/0 (socket says 172.21.15.7:46524) 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f3407fff700 1 -- 172.21.15.7:0/3782878406 learned_addr learned my addr 172.21.15.7:0/3782878406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f34077fe700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3408075650 0x7f3408199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340cd9c700 1 -- 172.21.15.7:0/3782878406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3408075650 msgr2=0x7f3408199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340cd9c700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3408075650 0x7f3408199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340cd9c700 1 -- 172.21.15.7:0/3782878406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 msgr2=0x7f3408199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340cd9c700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f3408199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f340cd9c700 1 -- 172.21.15.7:0/3782878406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400009d00 con 0x7f3408109440 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.017+0000 7f3407fff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f3408199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.018+0000 7f340cd9c700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f34081a0cf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3400007b90 tx=0x7f34000078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.018+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000e040 con 0x7f3408109440 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.018+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f340000be40 con 0x7f3408109440 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.018+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340001b6c0 con 0x7f3408109440 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.018+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34081a1810 con 0x7f3408109440 2024-01-11T17:56:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.018+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34081a1be0 con 0x7f3408109440 2024-01-11T17:56:10.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.019+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3408068430 con 0x7f3408109440 2024-01-11T17:56:10.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.021+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f3400003c00 con 0x7f3408109440 2024-01-11T17:56:10.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.022+0000 7f34057fa700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f33f00a1f60 0x7f33f00a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:10.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.022+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f34000ce7a0 con 0x7f3408109440 2024-01-11T17:56:10.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.023+0000 7f3407fff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f33f00a1f60 0x7f33f00a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:10.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.023+0000 7f3407fff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f33f00a1f60 0x7f33f00a4420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33f8011550 tx=0x7f33f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:10.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.026+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3400097890 con 0x7f3408109440 2024-01-11T17:56:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: osdmap e79: 12 total, 12 up, 12 in 2024-01-11T17:56:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] up:active 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:10.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:09 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/459784198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-01-11T17:56:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:10.241+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f340810ef70 con 0x7f3408109440 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: osdmap e79: 12 total, 12 up, 12 in 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-11T17:56:10.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-11T17:56:10.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] up:active 2024-01-11T17:56:10.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:10.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:09 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/459784198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-01-11T17:56:10.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: osdmap e79: 12 total, 12 up, 12 in 2024-01-11T17:56:10.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:56:10.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] up:active 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:10.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:09 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/459784198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-01-11T17:56:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:10 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/459784198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-01-11T17:56:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:10 smithi007 ceph-mon[33361]: osdmap e80: 12 total, 12 up, 12 in 2024-01-11T17:56:11.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:10 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3782878406' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-11T17:56:11.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:10 smithi007 ceph-mon[33361]: pgmap v243: 97 pgs: 4 creating+peering, 66 active+clean, 27 unknown; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 3 op/s 2024-01-11T17:56:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:10 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/459784198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-01-11T17:56:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:10 smithi016 ceph-mon[43195]: osdmap e80: 12 total, 12 up, 12 in 2024-01-11T17:56:11.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:10 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3782878406' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-11T17:56:11.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:10 smithi016 ceph-mon[43195]: pgmap v243: 97 pgs: 4 creating+peering, 66 active+clean, 27 unknown; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 3 op/s 2024-01-11T17:56:11.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:10 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/459784198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-01-11T17:56:11.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:10 smithi038 ceph-mon[43582]: osdmap e80: 12 total, 12 up, 12 in 2024-01-11T17:56:11.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:10 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3782878406' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-11T17:56:11.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:10 smithi038 ceph-mon[43582]: pgmap v243: 97 pgs: 4 creating+peering, 66 active+clean, 27 unknown; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 3 op/s 2024-01-11T17:56:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.796+0000 7f34057fa700 1 -- 172.21.15.7:0/3782878406 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) v1 ==== 228+0+0 (secure 0 0 0) 0x7f340009aeb0 con 0x7f3408109440 2024-01-11T17:56:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f33f00a1f60 msgr2=0x7f33f00a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f33f00a1f60 0x7f33f00a4420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33f8011550 tx=0x7f33f8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 msgr2=0x7f34081a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f34081a0cf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3400007b90 tx=0x7f34000078e0 comp rx=0 tx=0).stop 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 shutdown_connections 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f33f00a1f60 0x7f33f00a4420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.800+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408109440 0x7f34081a0cf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.801+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408074070 0x7f3408199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.801+0000 7f340e7ff700 1 --2- 172.21.15.7:0/3782878406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3408075650 0x7f3408199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.801+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 >> 172.21.15.7:0/3782878406 conn(0x7f34080fcad0 msgr2=0x7f3408105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.801+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 shutdown_connections 2024-01-11T17:56:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:11.801+0000 7f340e7ff700 1 -- 172.21.15.7:0/3782878406 wait complete. 2024-01-11T17:56:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2024-01-11T17:56:11.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 -- 172.21.15.7:0/536181311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01027a0 msgr2=0x7f78f0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 --2- 172.21.15.7:0/536181311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01027a0 0x7f78f0102c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78d80099c0 tx=0x7f78d8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 -- 172.21.15.7:0/536181311 shutdown_connections 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 --2- 172.21.15.7:0/536181311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f0103140 0x7f78f01099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 --2- 172.21.15.7:0/536181311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01015b0 0x7f78f01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 --2- 172.21.15.7:0/536181311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01027a0 0x7f78f0102c00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 -- 172.21.15.7:0/536181311 >> 172.21.15.7:0/536181311 conn(0x7f78f00fcb00 msgr2=0x7f78f00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 -- 172.21.15.7:0/536181311 shutdown_connections 2024-01-11T17:56:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.043+0000 7f78f4dff700 1 -- 172.21.15.7:0/536181311 wait complete. 2024-01-11T17:56:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.044+0000 7f78f4dff700 1 Processor -- start 2024-01-11T17:56:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.044+0000 7f78f4dff700 1 -- start start 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78f4dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 0x7f78f0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78f4dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01027a0 0x7f78f0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78f4dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f0103140 0x7f78f01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78f4dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01a1200 con 0x7f78f01015b0 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78f4dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01a1340 con 0x7f78f01027a0 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78f4dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01a14d0 con 0x7f78f0103140 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 0x7f78f0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 0x7f78f0199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39914/0 (socket says 172.21.15.7:39914) 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 -- 172.21.15.7:0/4053442074 learned_addr learned my addr 172.21.15.7:0/4053442074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78e7fff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01027a0 0x7f78f0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78eed9d700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f0103140 0x7f78f01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 -- 172.21.15.7:0/4053442074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f0103140 msgr2=0x7f78f01a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f0103140 0x7f78f01a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 -- 172.21.15.7:0/4053442074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01027a0 msgr2=0x7f78f0199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01027a0 0x7f78f0199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78ee59c700 1 -- 172.21.15.7:0/4053442074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80095f0 con 0x7f78f01015b0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.045+0000 7f78e7fff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01027a0 0x7f78f0199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.046+0000 7f78ee59c700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 0x7f78f0199680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e970 tx=0x7f78e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.046+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0009790 con 0x7f78f01015b0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.046+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78e000c7e0 con 0x7f78f01015b0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.046+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f01a17b0 con 0x7f78f01015b0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.046+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003c70 con 0x7f78f01015b0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.046+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f01a1d10 con 0x7f78f01015b0 2024-01-11T17:56:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.047+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f00683f0 con 0x7f78f01015b0 2024-01-11T17:56:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.050+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f78e000cc50 con 0x7f78f01015b0 2024-01-11T17:56:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.051+0000 7f78e77fe700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78dc0a1f60 0x7f78dc0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.051+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f78e0014070 con 0x7f78f01015b0 2024-01-11T17:56:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.051+0000 7f78e7fff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78dc0a1f60 0x7f78dc0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.052+0000 7f78e7fff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78dc0a1f60 0x7f78dc0a4420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78d8003730 tx=0x7f78d801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.054+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f78e0096ce0 con 0x7f78f01015b0 2024-01-11T17:56:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:11 smithi007 ceph-mon[33361]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-11T17:56:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:11 smithi007 ceph-mon[33361]: Cluster is now healthy 2024-01-11T17:56:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:11 smithi007 ceph-mon[33361]: osdmap e81: 12 total, 12 up, 12 in 2024-01-11T17:56:12.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:11 smithi016 ceph-mon[43195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-11T17:56:12.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:11 smithi016 ceph-mon[43195]: Cluster is now healthy 2024-01-11T17:56:12.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:11 smithi016 ceph-mon[43195]: osdmap e81: 12 total, 12 up, 12 in 2024-01-11T17:56:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.267+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f78f01a1fc0 con 0x7f78f01015b0 2024-01-11T17:56:12.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:11 smithi038 ceph-mon[43582]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-11T17:56:12.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:11 smithi038 ceph-mon[43582]: Cluster is now healthy 2024-01-11T17:56:12.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:11 smithi038 ceph-mon[43582]: osdmap e81: 12 total, 12 up, 12 in 2024-01-11T17:56:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.803+0000 7f78e77fe700 1 -- 172.21.15.7:0/4053442074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f78e009a300 con 0x7f78f01015b0 2024-01-11T17:56:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.807+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78dc0a1f60 msgr2=0x7f78dc0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.807+0000 7f78f4dff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78dc0a1f60 0x7f78dc0a4420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78d8003730 tx=0x7f78d801a040 comp rx=0 tx=0).stop 2024-01-11T17:56:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.807+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 msgr2=0x7f78f0199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.807+0000 7f78f4dff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 0x7f78f0199680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e970 tx=0x7f78e000ec80 comp rx=0 tx=0).stop 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 shutdown_connections 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78dc0a1f60 0x7f78dc0a4420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78f0103140 0x7f78f01a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f01015b0 0x7f78f0199680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 --2- 172.21.15.7:0/4053442074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f01027a0 0x7f78f0199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 >> 172.21.15.7:0/4053442074 conn(0x7f78f00fcb00 msgr2=0x7f78f0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 shutdown_connections 2024-01-11T17:56:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:12.808+0000 7f78f4dff700 1 -- 172.21.15.7:0/4053442074 wait complete. 2024-01-11T17:56:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 4 allow_ec_overwrites to true 2024-01-11T17:56:12.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-01-11T17:56:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 -- 172.21.15.7:0/2868914313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8102f90 msgr2=0x7fe1d8109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/2868914313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8102f90 0x7fe1d8109840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000b0d0 tx=0x7fe1d000b3e0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 -- 172.21.15.7:0/2868914313 shutdown_connections 2024-01-11T17:56:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/2868914313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d8101400 0x7fe1d8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/2868914313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8102f90 0x7fe1d8109840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/2868914313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d81025f0 0x7fe1d8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 -- 172.21.15.7:0/2868914313 >> 172.21.15.7:0/2868914313 conn(0x7fe1d80fc970 msgr2=0x7fe1d80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 -- 172.21.15.7:0/2868914313 shutdown_connections 2024-01-11T17:56:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.053+0000 7fe1d7fff700 1 -- 172.21.15.7:0/2868914313 wait complete. 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.055+0000 7fe1d7fff700 1 Processor -- start 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.055+0000 7fe1d7fff700 1 -- start start 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.055+0000 7fe1d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 0x7fe1d8197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.055+0000 7fe1d7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d81025f0 0x7fe1d81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.055+0000 7fe1d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d8102f90 0x7fe1d819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d819ef00 con 0x7fe1d8101400 2024-01-11T17:56:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d7fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d819f040 con 0x7fe1d8102f90 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d7fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d819f1d0 con 0x7fe1d81025f0 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 0x7fe1d8197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 0x7fe1d8197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39928/0 (socket says 172.21.15.7:39928) 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 -- 172.21.15.7:0/672370179 learned_addr learned my addr 172.21.15.7:0/672370179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d67fc700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d81025f0 0x7fe1d81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d77fe700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d8102f90 0x7fe1d819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 -- 172.21.15.7:0/672370179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d81025f0 msgr2=0x7fe1d81978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d81025f0 0x7fe1d81978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 -- 172.21.15.7:0/672370179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d8102f90 msgr2=0x7fe1d819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d8102f90 0x7fe1d819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 -- 172.21.15.7:0/672370179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009d00 con 0x7fe1d8101400 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d77fe700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d8102f90 0x7fe1d819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1d6ffd700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 0x7fe1d8197380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800e970 tx=0x7fe1c800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.056+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8009790 con 0x7fe1d8101400 2024-01-11T17:56:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.057+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1c80044e0 con 0x7fe1d8101400 2024-01-11T17:56:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.057+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d819f4b0 con 0x7fe1d8101400 2024-01-11T17:56:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.057+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8003e30 con 0x7fe1d8101400 2024-01-11T17:56:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.057+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d819f9c0 con 0x7fe1d8101400 2024-01-11T17:56:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.058+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d81086e0 con 0x7fe1d8101400 2024-01-11T17:56:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.060+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fe1c8003f90 con 0x7fe1d8101400 2024-01-11T17:56:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.061+0000 7fe1bffff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1c00a1ec0 0x7fe1c00a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.061+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fe1c8014070 con 0x7fe1d8101400 2024-01-11T17:56:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.061+0000 7fe1d67fc700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1c00a1ec0 0x7fe1c00a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.062+0000 7fe1d67fc700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1c00a1ec0 0x7fe1c00a4380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009bf0 tx=0x7fe1cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.065+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe1c8096d10 con 0x7fe1d8101400 2024-01-11T17:56:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3782878406' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-11T17:56:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:12 smithi007 ceph-mon[33361]: osdmap e82: 12 total, 12 up, 12 in 2024-01-11T17:56:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4053442074' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-01-11T17:56:13.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:12 smithi007 ceph-mon[33361]: pgmap v246: 129 pgs: 32 unknown, 4 creating+peering, 93 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-01-11T17:56:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3782878406' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-11T17:56:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:12 smithi016 ceph-mon[43195]: osdmap e82: 12 total, 12 up, 12 in 2024-01-11T17:56:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4053442074' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-01-11T17:56:13.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:12 smithi016 ceph-mon[43195]: pgmap v246: 129 pgs: 32 unknown, 4 creating+peering, 93 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-01-11T17:56:13.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3782878406' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-11T17:56:13.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:12 smithi038 ceph-mon[43582]: osdmap e82: 12 total, 12 up, 12 in 2024-01-11T17:56:13.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4053442074' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-01-11T17:56:13.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:12 smithi038 ceph-mon[43582]: pgmap v246: 129 pgs: 32 unknown, 4 creating+peering, 93 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-01-11T17:56:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.327+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe1d804f2d0 con 0x7fe1d8101400 2024-01-11T17:56:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.805+0000 7fe1bffff700 1 -- 172.21.15.7:0/672370179 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe1c809a330 con 0x7fe1d8101400 2024-01-11T17:56:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.808+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1c00a1ec0 msgr2=0x7fe1c00a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.808+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1c00a1ec0 0x7fe1c00a4380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009bf0 tx=0x7fe1cc009260 comp rx=0 tx=0).stop 2024-01-11T17:56:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.808+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 msgr2=0x7fe1d8197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.808+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 0x7fe1d8197380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800e970 tx=0x7fe1c800ed30 comp rx=0 tx=0).stop 2024-01-11T17:56:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 shutdown_connections 2024-01-11T17:56:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1c00a1ec0 0x7fe1c00a4380 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d8102f90 0x7fe1d819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1d8101400 0x7fe1d8197380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 --2- 172.21.15.7:0/672370179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1d81025f0 0x7fe1d81978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 >> 172.21.15.7:0/672370179 conn(0x7fe1d80fc970 msgr2=0x7fe1d8105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 shutdown_connections 2024-01-11T17:56:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:13.809+0000 7fe1d7fff700 1 -- 172.21.15.7:0/672370179 wait complete. 2024-01-11T17:56:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 4 to fsmap 2024-01-11T17:56:13.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:56:13.996 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:13 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: 2024-01-11T17:56:13.652+0000 7f7f477aa700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T17:56:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:13 smithi016 ceph-mon[43195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-11T17:56:13.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:13 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4053442074' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-01-11T17:56:13.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:13 smithi016 ceph-mon[43195]: osdmap e83: 12 total, 12 up, 12 in 2024-01-11T17:56:13.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:13 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/672370179' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 -- 172.21.15.7:0/3628720413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568102800 msgr2=0x7fd568102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/3628720413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568102800 0x7fd568102c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0099c0 tx=0x7fd55c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 -- 172.21.15.7:0/3628720413 shutdown_connections 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/3628720413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd568101610 0x7fd568101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/3628720413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5681031a0 0x7fd568109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/3628720413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568102800 0x7fd568102c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 -- 172.21.15.7:0/3628720413 >> 172.21.15.7:0/3628720413 conn(0x7fd5680fcb80 msgr2=0x7fd5680fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 -- 172.21.15.7:0/3628720413 shutdown_connections 2024-01-11T17:56:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.037+0000 7fd56d5ff700 1 -- 172.21.15.7:0/3628720413 wait complete. 2024-01-11T17:56:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.038+0000 7fd56d5ff700 1 Processor -- start 2024-01-11T17:56:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.038+0000 7fd56d5ff700 1 -- start start 2024-01-11T17:56:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd56d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 0x7fd5680753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd56d5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568102800 0x7fd568073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd56d5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 0x7fd568073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd56d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680746e0 con 0x7fd568101610 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd56d5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd568074850 con 0x7fd568102800 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd56d5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5681a14f0 con 0x7fd5681031a0 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd566ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 0x7fd5680753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 0x7fd568073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 0x7fd568073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38774/0 (socket says 172.21.15.7:38774) 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd566ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 0x7fd5680753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39942/0 (socket says 172.21.15.7:39942) 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 -- 172.21.15.7:0/1092855493 learned_addr learned my addr 172.21.15.7:0/1092855493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5667fc700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568102800 0x7fd568073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 -- 172.21.15.7:0/1092855493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568102800 msgr2=0x7fd568073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568102800 0x7fd568073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 -- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 msgr2=0x7fd5680753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 0x7fd5680753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd5677fe700 1 -- 172.21.15.7:0/1092855493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0095f0 con 0x7fd5681031a0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.039+0000 7fd566ffd700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 0x7fd5680753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.040+0000 7fd5677fe700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 0x7fd568073f40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ef30 tx=0x7fd56000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.040+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560018070 con 0x7fd5681031a0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.040+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5681a1770 con 0x7fd5681031a0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.040+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56000f040 con 0x7fd5681031a0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.040+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56000cd00 con 0x7fd5681031a0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.040+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5681a1c80 con 0x7fd5681031a0 2024-01-11T17:56:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.041+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd568108a90 con 0x7fd5681031a0 2024-01-11T17:56:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.043+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fd560003a00 con 0x7fd5681031a0 2024-01-11T17:56:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.045+0000 7fd54ffff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd5500a1ee0 0x7fd5500a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.045+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5600cd8b0 con 0x7fd5681031a0 2024-01-11T17:56:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.045+0000 7fd566ffd700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd5500a1ee0 0x7fd5500a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.045+0000 7fd566ffd700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd5500a1ee0 0x7fd5500a43a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd568102660 tx=0x7fd558009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.047+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd560096790 con 0x7fd5681031a0 2024-01-11T17:56:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:13 smithi007 ceph-mon[33361]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-11T17:56:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4053442074' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-01-11T17:56:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:13 smithi007 ceph-mon[33361]: osdmap e83: 12 total, 12 up, 12 in 2024-01-11T17:56:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/672370179' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-01-11T17:56:14.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:13 smithi038 ceph-mon[43582]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-11T17:56:14.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:13 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4053442074' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-01-11T17:56:14.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:13 smithi038 ceph-mon[43582]: osdmap e83: 12 total, 12 up, 12 in 2024-01-11T17:56:14.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:13 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/672370179' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-01-11T17:56:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.309+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd56810cfa0 con 0x7fd5681031a0 2024-01-11T17:56:14.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.314+0000 7fd54ffff700 1 -- 172.21.15.7:0/1092855493 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11250 (secure 0 0 0) 0x7fd560099db0 con 0x7fd5681031a0 2024-01-11T17:56:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:14.316 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24565,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24577,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:13.329302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-01-11T17:56:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd5500a1ee0 msgr2=0x7fd5500a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd5500a1ee0 0x7fd5500a43a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd568102660 tx=0x7fd558009260 comp rx=0 tx=0).stop 2024-01-11T17:56:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 msgr2=0x7fd568073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 0x7fd568073f40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ef30 tx=0x7fd56000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 shutdown_connections 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd5500a1ee0 0x7fd5500a43a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5681031a0 0x7fd568073f40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd568101610 0x7fd5680753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 --2- 172.21.15.7:0/1092855493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568102800 0x7fd568073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 >> 172.21.15.7:0/1092855493 conn(0x7fd5680fcb80 msgr2=0x7fd568105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 shutdown_connections 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.317+0000 7fd56d5ff700 1 -- 172.21.15.7:0/1092855493 wait complete. 2024-01-11T17:56:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T17:56:14.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 -- 172.21.15.7:0/4138326242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 msgr2=0x7fe214104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 --2- 172.21.15.7:0/4138326242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214104de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe2080099c0 tx=0x7fe208009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 -- 172.21.15.7:0/4138326242 shutdown_connections 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 --2- 172.21.15.7:0/4138326242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214105320 0x7fe21410bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 --2- 172.21.15.7:0/4138326242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214103790 0x7fe214103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 --2- 172.21.15.7:0/4138326242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214104de0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 -- 172.21.15.7:0/4138326242 >> 172.21.15.7:0/4138326242 conn(0x7fe2140fed20 msgr2=0x7fe214101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.541+0000 7fe219bff700 1 -- 172.21.15.7:0/4138326242 shutdown_connections 2024-01-11T17:56:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.542+0000 7fe219bff700 1 -- 172.21.15.7:0/4138326242 wait complete. 2024-01-11T17:56:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.542+0000 7fe219bff700 1 Processor -- start 2024-01-11T17:56:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 -- start start 2024-01-11T17:56:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214103790 0x7fe2141996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214105320 0x7fe2141a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a1230 con 0x7fe214104980 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a1370 con 0x7fe214105320 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe219bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a1500 con 0x7fe214103790 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe212ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe212ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39962/0 (socket says 172.21.15.7:39962) 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe212ffd700 1 -- 172.21.15.7:0/2984419166 learned_addr learned my addr 172.21.15.7:0/2984419166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe213fff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214105320 0x7fe2141a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.543+0000 7fe2137fe700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214103790 0x7fe2141996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe212ffd700 1 -- 172.21.15.7:0/2984419166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214103790 msgr2=0x7fe2141996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe212ffd700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214103790 0x7fe2141996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe212ffd700 1 -- 172.21.15.7:0/2984419166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214105320 msgr2=0x7fe2141a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe212ffd700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214105320 0x7fe2141a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe212ffd700 1 -- 172.21.15.7:0/2984419166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009520 con 0x7fe214104980 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe2137fe700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214103790 0x7fe2141996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe213fff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214105320 0x7fe2141a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe212ffd700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214199bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe208003730 tx=0x7fe20800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20801d070 con 0x7fe214104980 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2080095f0 con 0x7fe214104980 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20800fe70 con 0x7fe214104980 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208017650 con 0x7fe214104980 2024-01-11T17:56:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.544+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a1a70 con 0x7fe214104980 2024-01-11T17:56:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.545+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21410ab60 con 0x7fe214104980 2024-01-11T17:56:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.547+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fe208022470 con 0x7fe214104980 2024-01-11T17:56:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.550+0000 7fe210ff9700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1fc0a1ee0 0x7fe1fc0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.550+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe2080cf230 con 0x7fe214104980 2024-01-11T17:56:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.550+0000 7fe2137fe700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1fc0a1ee0 0x7fe1fc0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.550+0000 7fe2137fe700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1fc0a1ee0 0x7fe1fc0a43a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2141047e0 tx=0x7fe204009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.552+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe208098110 con 0x7fe214104980 2024-01-11T17:56:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.770+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe21410f000 con 0x7fe214104980 2024-01-11T17:56:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.772+0000 7fe210ff9700 1 -- 172.21.15.7:0/2984419166 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23806 (secure 0 0 0) 0x7fe20809b730 con 0x7fe214104980 2024-01-11T17:56:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:14.773 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:13.329124+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.774+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1fc0a1ee0 msgr2=0x7fe1fc0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.774+0000 7fe219bff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1fc0a1ee0 0x7fe1fc0a43a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2141047e0 tx=0x7fe204009260 comp rx=0 tx=0).stop 2024-01-11T17:56:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.774+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 msgr2=0x7fe214199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.774+0000 7fe219bff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214199bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe208003730 tx=0x7fe20800fa00 comp rx=0 tx=0).stop 2024-01-11T17:56:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 shutdown_connections 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe1fc0a1ee0 0x7fe1fc0a43a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe214103790 0x7fe2141996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe214105320 0x7fe2141a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 --2- 172.21.15.7:0/2984419166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214104980 0x7fe214199bf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 >> 172.21.15.7:0/2984419166 conn(0x7fe2140fed20 msgr2=0x7fe214107ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 shutdown_connections 2024-01-11T17:56:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:14.775+0000 7fe219bff700 1 -- 172.21.15.7:0/2984419166 wait complete. 2024-01-11T17:56:14.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 -- 172.21.15.7:0/3439540730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 msgr2=0x7fdba010bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 --2- 172.21.15.7:0/3439540730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba010bbd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b0d0 tx=0x7fdb9800b3e0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 -- 172.21.15.7:0/3439540730 shutdown_connections 2024-01-11T17:56:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 --2- 172.21.15.7:0/3439540730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdba0103790 0x7fdba0103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 --2- 172.21.15.7:0/3439540730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba010bbd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 --2- 172.21.15.7:0/3439540730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba0104980 0x7fdba0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 -- 172.21.15.7:0/3439540730 >> 172.21.15.7:0/3439540730 conn(0x7fdba00fed20 msgr2=0x7fdba0101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:15.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 -- 172.21.15.7:0/3439540730 shutdown_connections 2024-01-11T17:56:15.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.000+0000 7fdba49ff700 1 -- 172.21.15.7:0/3439540730 wait complete. 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.001+0000 7fdba49ff700 1 Processor -- start 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.001+0000 7fdba49ff700 1 -- start start 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdba49ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdba0103790 0x7fdba0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdba49ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba0104980 0x7fdba0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdba49ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdba49ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01a1300 con 0x7fdba0105320 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdba49ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01a1440 con 0x7fdba0104980 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdba49ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01a15d0 con 0x7fdba0103790 2024-01-11T17:56:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdba0103790 0x7fdba0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba01a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39978/0 (socket says 172.21.15.7:39978) 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 -- 172.21.15.7:0/869526085 learned_addr learned my addr 172.21.15.7:0/869526085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9dd9b700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba0104980 0x7fdba0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 -- 172.21.15.7:0/869526085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdba0103790 msgr2=0x7fdba0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdba0103790 0x7fdba0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 -- 172.21.15.7:0/869526085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba0104980 msgr2=0x7fdba0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba0104980 0x7fdba0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.002+0000 7fdb9ed9d700 1 -- 172.21.15.7:0/869526085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb98009d00 con 0x7fdba0105320 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.003+0000 7fdb9ed9d700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba01a0d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb98007b60 tx=0x7fdb98008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.003+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800e040 con 0x7fdba0105320 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.003+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba01a1850 con 0x7fdba0105320 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.003+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb9801d930 con 0x7fdba0105320 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.003+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9801b9f0 con 0x7fdba0105320 2024-01-11T17:56:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.003+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba01a1d20 con 0x7fdba0105320 2024-01-11T17:56:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.004+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba010aae0 con 0x7fdba0105320 2024-01-11T17:56:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.006+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fdb9801daa0 con 0x7fdba0105320 2024-01-11T17:56:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.007+0000 7fdb8f7fe700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb880a1f60 0x7fdb880a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.008+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdb980ceef0 con 0x7fdba0105320 2024-01-11T17:56:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.008+0000 7fdb9e59c700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb880a1f60 0x7fdb880a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.008+0000 7fdb9e59c700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb880a1f60 0x7fdb880a4420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdba01047e0 tx=0x7fdb90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.011+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdb98097dd0 con 0x7fdba0105320 2024-01-11T17:56:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:14 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/672370179' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-01-11T17:56:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:14 smithi007 ceph-mon[33361]: osdmap e84: 12 total, 12 up, 12 in 2024-01-11T17:56:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:14 smithi007 ceph-mon[33361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:14 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1092855493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:14 smithi007 ceph-mon[33361]: pgmap v249: 129 pgs: 20 unknown, 109 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 7 op/s 2024-01-11T17:56:15.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:14 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2984419166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:15.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.225+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdba0068840 con 0x7fdba0105320 2024-01-11T17:56:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.227+0000 7fdb8f7fe700 1 -- 172.21.15.7:0/869526085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23806 (secure 0 0 0) 0x7fdb9809b3f0 con 0x7fdba0105320 2024-01-11T17:56:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:15.228 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:13.329124+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb880a1f60 msgr2=0x7fdb880a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb880a1f60 0x7fdb880a4420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdba01047e0 tx=0x7fdb90009260 comp rx=0 tx=0).stop 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 msgr2=0x7fdba01a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba01a0d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb98007b60 tx=0x7fdb98008e40 comp rx=0 tx=0).stop 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 shutdown_connections 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb880a1f60 0x7fdb880a4420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdba0103790 0x7fdba0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdba0105320 0x7fdba01a0d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 --2- 172.21.15.7:0/869526085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba0104980 0x7fdba0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 >> 172.21.15.7:0/869526085 conn(0x7fdba00fed20 msgr2=0x7fdba0107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 shutdown_connections 2024-01-11T17:56:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.229+0000 7fdba49ff700 1 -- 172.21.15.7:0/869526085 wait complete. 2024-01-11T17:56:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:14 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/672370179' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-01-11T17:56:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:14 smithi016 ceph-mon[43195]: osdmap e84: 12 total, 12 up, 12 in 2024-01-11T17:56:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:14 smithi016 ceph-mon[43195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:15.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:14 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1092855493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:14 smithi016 ceph-mon[43195]: pgmap v249: 129 pgs: 20 unknown, 109 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 7 op/s 2024-01-11T17:56:15.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:14 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2984419166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:15.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:56:15.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:14 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/672370179' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-01-11T17:56:15.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:14 smithi038 ceph-mon[43582]: osdmap e84: 12 total, 12 up, 12 in 2024-01-11T17:56:15.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:14 smithi038 ceph-mon[43582]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:15.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:14 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1092855493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:15.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:14 smithi038 ceph-mon[43582]: pgmap v249: 129 pgs: 20 unknown, 109 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 7 op/s 2024-01-11T17:56:15.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:14 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2984419166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 -- 172.21.15.7:0/3286336480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 msgr2=0x7f0878102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 --2- 172.21.15.7:0/3286336480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 0x7f0878102be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f086c009990 tx=0x7f086c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 -- 172.21.15.7:0/3286336480 shutdown_connections 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 --2- 172.21.15.7:0/3286336480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878103120 0x7f08781099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 --2- 172.21.15.7:0/3286336480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878101590 0x7f08781019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 --2- 172.21.15.7:0/3286336480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 0x7f0878102be0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 -- 172.21.15.7:0/3286336480 >> 172.21.15.7:0/3286336480 conn(0x7f08780fcb00 msgr2=0x7f08780fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 -- 172.21.15.7:0/3286336480 shutdown_connections 2024-01-11T17:56:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.468+0000 7f087dbff700 1 -- 172.21.15.7:0/3286336480 wait complete. 2024-01-11T17:56:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 Processor -- start 2024-01-11T17:56:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 -- start start 2024-01-11T17:56:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101590 0x7f0878197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 0x7f08781979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 0x7f087819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819f010 con 0x7f0878103120 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819f150 con 0x7f0878101590 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.469+0000 7f087dbff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819f2e0 con 0x7f0878102780 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 0x7f087819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0876ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 0x7f08781979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 0x7f087819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39984/0 (socket says 172.21.15.7:39984) 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 -- 172.21.15.7:0/645277457 learned_addr learned my addr 172.21.15.7:0/645277457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f08777fe700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101590 0x7f0878197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 -- 172.21.15.7:0/645277457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 msgr2=0x7f08781979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 0x7f08781979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 -- 172.21.15.7:0/645277457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101590 msgr2=0x7f0878197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101590 0x7f0878197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 -- 172.21.15.7:0/645277457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009520 con 0x7f0878103120 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.470+0000 7f0877fff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 0x7f087819ea40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0870007ee0 tx=0x7f087000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.471+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870017070 con 0x7f0878103120 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.471+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f087000f040 con 0x7f0878103120 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.471+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c0095f0 con 0x7f0878103120 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.471+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870005580 con 0x7f0878103120 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.471+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087819f8f0 con 0x7f0878103120 2024-01-11T17:56:15.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.472+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08781089e0 con 0x7f0878103120 2024-01-11T17:56:15.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.474+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f087000db20 con 0x7f0878103120 2024-01-11T17:56:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.475+0000 7f0874ff9700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08600a1ee0 0x7f08600a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.475+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f08700cd2c0 con 0x7f0878103120 2024-01-11T17:56:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.475+0000 7f08777fe700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08600a1ee0 0x7f08600a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.476+0000 7f08777fe700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08600a1ee0 0x7f08600a43a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08781025e0 tx=0x7f0868009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.479+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0870096170 con 0x7f0878103120 2024-01-11T17:56:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.738+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08780683f0 con 0x7f0878103120 2024-01-11T17:56:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.738+0000 7f0874ff9700 1 -- 172.21.15.7:0/645277457 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11250 (secure 0 0 0) 0x7f0870099790 con 0x7f0878103120 2024-01-11T17:56:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24547,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24565,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24577,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:13.329302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08600a1ee0 msgr2=0x7f08600a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08600a1ee0 0x7f08600a43a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08781025e0 tx=0x7f0868009260 comp rx=0 tx=0).stop 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 msgr2=0x7f087819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 0x7f087819ea40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0870007ee0 tx=0x7f087000d3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 shutdown_connections 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08600a1ee0 0x7f08600a43a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0878103120 0x7f087819ea40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101590 0x7f0878197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 --2- 172.21.15.7:0/645277457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0878102780 0x7f08781979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 >> 172.21.15.7:0/645277457 conn(0x7f08780fcb00 msgr2=0x7f0878105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 shutdown_connections 2024-01-11T17:56:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.741+0000 7f087dbff700 1 -- 172.21.15.7:0/645277457 wait complete. 2024-01-11T17:56:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T17:56:15.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 -- 172.21.15.7:0/3950405670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4102780 msgr2=0x7f3df4102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 --2- 172.21.15.7:0/3950405670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4102780 0x7f3df4102be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3de80099c0 tx=0x7f3de8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 -- 172.21.15.7:0/3950405670 shutdown_connections 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 --2- 172.21.15.7:0/3950405670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3df4103120 0x7f3df41099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 --2- 172.21.15.7:0/3950405670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4101590 0x7f3df41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 --2- 172.21.15.7:0/3950405670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4102780 0x7f3df4102be0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 -- 172.21.15.7:0/3950405670 >> 172.21.15.7:0/3950405670 conn(0x7f3df40fcb00 msgr2=0x7f3df40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 -- 172.21.15.7:0/3950405670 shutdown_connections 2024-01-11T17:56:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.969+0000 7f3df9bff700 1 -- 172.21.15.7:0/3950405670 wait complete. 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.970+0000 7f3df9bff700 1 Processor -- start 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.970+0000 7f3df9bff700 1 -- start start 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df9bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 0x7f3df4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df9bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3df4102780 0x7f3df4199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df9bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4103120 0x7f3df41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df9bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a11f0 con 0x7f3df4101590 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df9bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a1330 con 0x7f3df4103120 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df9bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a14c0 con 0x7f3df4102780 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 0x7f3df4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 0x7f3df4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39994/0 (socket says 172.21.15.7:39994) 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 -- 172.21.15.7:0/4277252605 learned_addr learned my addr 172.21.15.7:0/4277252605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df2ffd700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3df4102780 0x7f3df4199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df3fff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4103120 0x7f3df41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 -- 172.21.15.7:0/4277252605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3df4102780 msgr2=0x7f3df4199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3df4102780 0x7f3df4199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 -- 172.21.15.7:0/4277252605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4103120 msgr2=0x7f3df41a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4103120 0x7f3df41a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 -- 172.21.15.7:0/4277252605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de80095f0 con 0x7f3df4101590 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.971+0000 7f3df37fe700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 0x7f3df4199670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d980 tx=0x7f3de400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.972+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400b950 con 0x7f3df4101590 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.972+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de4007620 con 0x7f3df4101590 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.972+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41a17a0 con 0x7f3df4101590 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.972+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400f3c0 con 0x7f3df4101590 2024-01-11T17:56:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.972+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a1d00 con 0x7f3df4101590 2024-01-11T17:56:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.973+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df40683f0 con 0x7f3df4101590 2024-01-11T17:56:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.975+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f3de4010640 con 0x7f3df4101590 2024-01-11T17:56:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.976+0000 7f3df0ff9700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3ddc0a1f60 0x7f3ddc0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.976+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3de40cda20 con 0x7f3df4101590 2024-01-11T17:56:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.976+0000 7f3df2ffd700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3ddc0a1f60 0x7f3ddc0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.977+0000 7f3df2ffd700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3ddc0a1f60 0x7f3ddc0a4420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009990 tx=0x7f3de8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:15.981+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3de4096870 con 0x7f3df4101590 2024-01-11T17:56:16.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:15 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/869526085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:16.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:15 smithi007 ceph-mon[33361]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-01-11T17:56:16.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:15 smithi007 ceph-mon[33361]: Cluster is now healthy 2024-01-11T17:56:16.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:15 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/645277457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:16.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.195+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3df410cdb0 con 0x7f3df4101590 2024-01-11T17:56:16.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.196+0000 7f3df0ff9700 1 -- 172.21.15.7:0/4277252605 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23806 (secure 0 0 0) 0x7f3de4099e90 con 0x7f3df4101590 2024-01-11T17:56:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:16.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:13.329124+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3ddc0a1f60 msgr2=0x7f3ddc0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3ddc0a1f60 0x7f3ddc0a4420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009990 tx=0x7f3de8000bc0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 msgr2=0x7f3df4199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 0x7f3df4199670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d980 tx=0x7f3de400dc90 comp rx=0 tx=0).stop 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 shutdown_connections 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3ddc0a1f60 0x7f3ddc0a4420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4103120 0x7f3df41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df4101590 0x7f3df4199670 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 --2- 172.21.15.7:0/4277252605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3df4102780 0x7f3df4199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 >> 172.21.15.7:0/4277252605 conn(0x7f3df40fcb00 msgr2=0x7f3df4073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 shutdown_connections 2024-01-11T17:56:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.199+0000 7f3df9bff700 1 -- 172.21.15.7:0/4277252605 wait complete. 2024-01-11T17:56:16.219 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T17:56:08.339093+0000', 'modified': '2024-01-11T17:56:13.329302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24602}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/1892769619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 1892769619}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 1892769619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-01-11T17:56:16.219 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T17:56:16.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-11T17:56:16.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:15 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/869526085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:16.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:15 smithi016 ceph-mon[43195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-01-11T17:56:16.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:15 smithi016 ceph-mon[43195]: Cluster is now healthy 2024-01-11T17:56:16.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:15 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/645277457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:16.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:15 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/869526085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:16.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:15 smithi038 ceph-mon[43582]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-01-11T17:56:16.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:15 smithi038 ceph-mon[43582]: Cluster is now healthy 2024-01-11T17:56:16.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:15 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/645277457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:16.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 -- 172.21.15.7:0/3213255666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 msgr2=0x7f1be81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 --2- 172.21.15.7:0/3213255666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81019d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0099c0 tx=0x7f1bdc009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 -- 172.21.15.7:0/3213255666 shutdown_connections 2024-01-11T17:56:16.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 --2- 172.21.15.7:0/3213255666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8103140 0x7f1be81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 --2- 172.21.15.7:0/3213255666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81019d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 --2- 172.21.15.7:0/3213255666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1be81027a0 0x7f1be8102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 -- 172.21.15.7:0/3213255666 >> 172.21.15.7:0/3213255666 conn(0x7f1be80fcb00 msgr2=0x7f1be80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 -- 172.21.15.7:0/3213255666 shutdown_connections 2024-01-11T17:56:16.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.424+0000 7f1beedff700 1 -- 172.21.15.7:0/3213255666 wait complete. 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 Processor -- start 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 -- start start 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1be81027a0 0x7f1be81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8103140 0x7f1be819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be819f020 con 0x7f1be81015b0 2024-01-11T17:56:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be819f160 con 0x7f1be8103140 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.426+0000 7f1beedff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be819f2f0 con 0x7f1be81027a0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40000/0 (socket says 172.21.15.7:40000) 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 -- 172.21.15.7:0/594167288 learned_addr learned my addr 172.21.15.7:0/594167288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1be7fff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1be81027a0 0x7f1be81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1bed39c700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8103140 0x7f1be819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 -- 172.21.15.7:0/594167288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1be81027a0 msgr2=0x7f1be81979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1be81027a0 0x7f1be81979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 -- 172.21.15.7:0/594167288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8103140 msgr2=0x7f1be819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8103140 0x7f1be819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 -- 172.21.15.7:0/594167288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc0095f0 con 0x7f1be81015b0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.427+0000 7f1becb9b700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81974a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc003730 tx=0x7f1bdc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.428+0000 7f1be5ffb700 1 -- 172.21.15.7:0/594167288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc01d070 con 0x7f1be81015b0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.428+0000 7f1be5ffb700 1 -- 172.21.15.7:0/594167288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc00bda0 con 0x7f1be81015b0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.428+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be819f570 con 0x7f1be81015b0 2024-01-11T17:56:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.428+0000 7f1be5ffb700 1 -- 172.21.15.7:0/594167288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc004b30 con 0x7f1be81015b0 2024-01-11T17:56:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.428+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be819f990 con 0x7f1be81015b0 2024-01-11T17:56:16.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.432+0000 7f1be5ffb700 1 -- 172.21.15.7:0/594167288 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1bdc004c90 con 0x7f1be81015b0 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.433+0000 7f1c02bc3b80 1 Processor -- start 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.433+0000 7f1c02bc3b80 1 -- start start 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c61737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6171f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c61729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c6069040 con 0x5641c6173d30 2024-01-11T17:56:16.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c60ccad0 con 0x5641c6172520 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c60ca3f0 con 0x5641c61733b0 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1be5ffb700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1bd80a1ee0 0x7f1bd80a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1be5ffb700 1 -- 172.21.15.7:0/594167288 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1bdc0ce9f0 con 0x7f1be81015b0 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6171f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1bed39c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c61737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6171f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40004/0 (socket says 172.21.15.7:40004) 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 -- 172.21.15.7:0/2627086502 learned_addr learned my addr 172.21.15.7:0/2627086502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1be7fff700 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c61729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1bed39c700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1bd80a1ee0 0x7f1bd80a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 -- 172.21.15.7:0/2627086502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 msgr2=0x5641c61737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c61737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 -- 172.21.15.7:0/2627086502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 msgr2=0x5641c61729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c61729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1becb9b700 1 -- 172.21.15.7:0/2627086502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c5f3a360 con 0x5641c6173d30 2024-01-11T17:56:16.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.434+0000 7f1be7fff700 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c61729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:16.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1becb9b700 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6171f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000c00 tx=0x7f1bdc0978d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f85f9bcabbdef6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:16.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1bd2ffd700 1 -- 172.21.15.7:0/2627086502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09b0d0 con 0x5641c6173d30 2024-01-11T17:56:16.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1bd2ffd700 1 -- 172.21.15.7:0/2627086502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc09aef0 con 0x5641c6173d30 2024-01-11T17:56:16.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1bd2ffd700 1 -- 172.21.15.7:0/2627086502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09f6e0 con 0x5641c6173d30 2024-01-11T17:56:16.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2627086502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 msgr2=0x5641c6171f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1bed39c700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1bd80a1ee0 0x7f1bd80a43a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5641c6171ed0 tx=0x7f1be000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.435+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6171f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000c00 tx=0x7f1bdc0978d0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2627086502 shutdown_connections 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c61737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c61729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2627086502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6171f40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2627086502 >> 172.21.15.7:0/2627086502 conn(0x5641c60e7750 msgr2=0x5641c616cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2627086502 shutdown_connections 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2627086502 wait complete. 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 -- start start 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.436+0000 7f1c02bc3b80 1 Processor -- start 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c617bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c61733b0 0x5641c617c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c6173d30 0x5641c6183080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c60c5ce0 con 0x5641c6172520 2024-01-11T17:56:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c6034350 con 0x5641c61733b0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c6033ff0 con 0x5641c6173d30 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1c02bc3b80 10 client.0 _ll_register_callbacks cb 0x5641c60666d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c617bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1be7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c61733b0 0x5641c617c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1be7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c61733b0 0x5641c617c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49428/0 (socket says 172.21.15.7:49428) 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1bed39c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c6173d30 0x5641c6183080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1be7fff700 1 -- 172.21.15.7:0/1574299281 learned_addr learned my addr 172.21.15.7:0/1574299281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1becb9b700 1 -- 172.21.15.7:0/1574299281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c6173d30 msgr2=0x5641c6183080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1becb9b700 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c6173d30 0x5641c6183080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1becb9b700 1 -- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c61733b0 msgr2=0x5641c617c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1becb9b700 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c61733b0 0x5641c617c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.437+0000 7f1becb9b700 1 -- 172.21.15.7:0/1574299281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be819f990 con 0x5641c6172520 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1becb9b700 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c617bb00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc09b520 tx=0x7f1bdc09bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33963346c5df3d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1bc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1bc7fff700 1 -- 172.21.15.7:0/1574299281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09b0d0 con 0x5641c6172520 2024-01-11T17:56:16.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641c6164fc0 con 0x5641c6172520 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1bc7fff700 1 -- 172.21.15.7:0/1574299281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1bdc00fdc0 con 0x5641c6172520 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1bc7fff700 1 -- 172.21.15.7:0/1574299281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc064d60 con 0x5641c6172520 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1c02bc3b80 10 client.14721 fetch_fsmap learned FSMap version 16 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x5641c5ee6b10 con 0x5641c6172520 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.438+0000 7f1bc7fff700 1 -- 172.21.15.7:0/1574299281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641c6164fc0 con 0x5641c6172520 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1bc7fff700 1 -- 172.21.15.7:0/1574299281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1bdc0a3bc0 con 0x5641c6172520 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1c02bc3b80 10 client.14721 fetch_fsmap finished waiting for FSMap version 16 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1c02bc3b80 10 client.14721 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1c02bc3b80 20 client.14721 populate_metadata read hostname 'smithi007' 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c6187310 0x5641c6186050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1c02bc3b80 4 client.14721 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641c60beb70 con 0x5641c6187310 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.439+0000 7f1be7fff700 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c6187310 0x5641c6186050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.440+0000 7f1be7fff700 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c6187310 0x5641c6186050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f8c98c090e34d7 server_cookie=91d244433fb8a722 in_seq=0 out_seq=0 2024-01-11T17:56:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.440+0000 7f1bc7fff700 10 client.14721 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.440+0000 7f1bc7fff700 1 -- 172.21.15.7:0/1574299281 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641c60beb70 con 0x5641c6187310 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.440+0000 7f1bc7fff700 10 client.14721 handle_command_reply: tid=0 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.440+0000 7f1c02bc3b80 1 client.14721 shutdown 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 msgr2=0x5641c617bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c617bb00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc09b520 tx=0x7f1bdc09bc90 comp rx=0 tx=0).stop 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 shutdown_connections 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c6187310 0x5641c6186050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c6173d30 0x5641c6183080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c617bb00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1bc7fff700 0 client.14721 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/1574299281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c61733b0 0x5641c617c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 >> 172.21.15.7:0/1574299281 conn(0x5641c60e7750 msgr2=0x5641c6170e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 shutdown_connections 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/1574299281 wait complete. 2024-01-11T17:56:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.441+0000 7f1c02bc3b80 20 client.14721 trim_cache size 0 max 16384 2024-01-11T17:56:16.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 Processor -- start 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 -- start start 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c6179d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c6183040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6173d30 0x5641c61769c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c5f3c650 con 0x5641c6172520 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c60ccad0 con 0x5641c6173d30 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c616d360 con 0x5641c61733b0 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c6179d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c6179d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40032/0 (socket says 172.21.15.7:40032) 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.460+0000 7f1becb9b700 1 -- 172.21.15.7:0/2014113519 learned_addr learned my addr 172.21.15.7:0/2014113519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:16.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1be7fff700 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c6183040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1bed39c700 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6173d30 0x5641c61769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1becb9b700 1 -- 172.21.15.7:0/2014113519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 msgr2=0x5641c6183040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1becb9b700 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c6183040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1becb9b700 1 -- 172.21.15.7:0/2014113519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6173d30 msgr2=0x5641c61769c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1becb9b700 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6173d30 0x5641c61769c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1becb9b700 1 -- 172.21.15.7:0/2014113519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c60871e0 con 0x5641c6172520 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1bed39c700 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6173d30 0x5641c61769c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1becb9b700 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c6179d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009fd0 tx=0x7f1bdc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba417c5bd993d2d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:16.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1bd27fc700 1 -- 172.21.15.7:0/2014113519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09bd00 con 0x5641c6172520 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1bd27fc700 1 -- 172.21.15.7:0/2014113519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc064d60 con 0x5641c6172520 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.461+0000 7f1bd27fc700 1 -- 172.21.15.7:0/2014113519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09f6e0 con 0x5641c6172520 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2014113519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 msgr2=0x5641c6179d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c6179d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009fd0 tx=0x7f1bdc005fd0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2014113519 shutdown_connections 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6173d30 0x5641c61769c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6172520 0x5641c6179d60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/2014113519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c6183040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2014113519 >> 172.21.15.7:0/2014113519 conn(0x5641c60e7750 msgr2=0x5641c6172900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2014113519 shutdown_connections 2024-01-11T17:56:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/2014113519 wait complete. 2024-01-11T17:56:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 -- start start 2024-01-11T17:56:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.462+0000 7f1c02bc3b80 1 Processor -- start 2024-01-11T17:56:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c617f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c617fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6182570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c6182ab0 con 0x5641c6173d30 2024-01-11T17:56:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c6182bf0 con 0x5641c6172520 2024-01-11T17:56:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c6182db0 con 0x5641c61733b0 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1c02bc3b80 10 client.0 _ll_register_callbacks cb 0x5641c60666d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1be7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c617fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c617f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1bed39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6182570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1be7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c617fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38894/0 (socket says 172.21.15.7:38894) 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c617f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49450/0 (socket says 172.21.15.7:49450) 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1be7fff700 1 -- 172.21.15.7:0/417795545 learned_addr learned my addr 172.21.15.7:0/417795545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 -- 172.21.15.7:0/417795545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 msgr2=0x5641c617fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c617fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 -- 172.21.15.7:0/417795545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 msgr2=0x5641c6182570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6182570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1becb9b700 1 -- 172.21.15.7:0/417795545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641c5ee6b10 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.463+0000 7f1be7fff700 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c617fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1becb9b700 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c617f4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009fd0 tx=0x7f1bdc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daaa0e7d3a95d990 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1bc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1bc7fff700 1 -- 172.21.15.7:0/417795545 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09b0d0 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bdc09aef0 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1bc7fff700 1 -- 172.21.15.7:0/417795545 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1bdc09bc90 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1bc7fff700 1 -- 172.21.15.7:0/417795545 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc09f6e0 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1c02bc3b80 10 client.24710 fetch_fsmap learned FSMap version 16 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x5641c5f3ec80 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1bc7fff700 1 -- 172.21.15.7:0/417795545 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc09aef0 con 0x5641c6172520 2024-01-11T17:56:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.464+0000 7f1bc7fff700 1 -- 172.21.15.7:0/417795545 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1bdc0b9a00 con 0x5641c6172520 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1c02bc3b80 10 client.24710 fetch_fsmap finished waiting for FSMap version 16 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1c02bc3b80 10 client.24710 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1c02bc3b80 20 client.24710 populate_metadata read hostname 'smithi007' 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c61e17b0 0x5641c61dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1c02bc3b80 4 client.24710 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5641c60be890 con 0x5641c61e17b0 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1be7fff700 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c61e17b0 0x5641c61dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1be7fff700 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c61e17b0 0x5641c61dde20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a68e46c6f4d501 server_cookie=3e0b86ead56003df in_seq=0 out_seq=0 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.465+0000 7f1bc7fff700 10 client.24710 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1bc7fff700 1 -- 172.21.15.7:0/417795545 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5641c60be890 con 0x5641c61e17b0 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1bc7fff700 10 client.24710 handle_command_reply: tid=0 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 client.24710 shutdown 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 msgr2=0x5641c617f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c617f4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009fd0 tx=0x7f1bdc005830 comp rx=0 tx=0).stop 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 shutdown_connections 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5641c61e17b0 0x5641c61dde20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5641c6173d30 0x5641c6182570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641c6172520 0x5641c617f4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1bc7fff700 0 client.24710 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 --2- 172.21.15.7:0/417795545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5641c61733b0 0x5641c617fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 >> 172.21.15.7:0/417795545 conn(0x5641c60e7750 msgr2=0x5641c61769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 shutdown_connections 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 1 -- 172.21.15.7:0/417795545 wait complete. 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.466+0000 7f1c02bc3b80 20 client.24710 trim_cache size 0 max 16384 2024-01-11T17:56:16.473 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24602, 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.2301215790000004, 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 16, 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 84, 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 83, 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.202655915999999 2024-01-11T17:56:16.474 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1bd80a1ee0 msgr2=0x7f1bd80a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1bd80a1ee0 0x7f1bd80a43a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5641c6171ed0 tx=0x7f1be000b040 comp rx=0 tx=0).stop 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 msgr2=0x7f1be81974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81974a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc003730 tx=0x7f1bdc00fa00 comp rx=0 tx=0).stop 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 shutdown_connections 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1bd80a1ee0 0x7f1bd80a43a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8103140 0x7f1be819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be81015b0 0x7f1be81974a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 --2- 172.21.15.7:0/594167288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1be81027a0 0x7f1be81979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 >> 172.21.15.7:0/594167288 conn(0x7f1be80fcb00 msgr2=0x7f1be8105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 shutdown_connections 2024-01-11T17:56:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:16.467+0000 7f1beedff700 1 -- 172.21.15.7:0/594167288 wait complete. 2024-01-11T17:56:16.486 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-11T17:56:16.486 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:16.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph fs ls 2024-01-11T17:56:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:17.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:16 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4277252605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:17.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:16 smithi007 ceph-mon[33361]: pgmap v250: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 354 B/s wr, 4 op/s 2024-01-11T17:56:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:16 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4277252605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:17.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:16 smithi016 ceph-mon[43195]: pgmap v250: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 354 B/s wr, 4 op/s 2024-01-11T17:56:17.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:16 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4277252605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:17.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:16 smithi038 ceph-mon[43582]: pgmap v250: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 354 B/s wr, 4 op/s 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 -- 172.21.15.7:0/3311306214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434107650 msgr2=0x7f3434107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 --2- 172.21.15.7:0/3311306214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434107650 0x7f3434107a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f3428009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 -- 172.21.15.7:0/3311306214 shutdown_connections 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 --2- 172.21.15.7:0/3311306214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434102020 0x7f343410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 --2- 172.21.15.7:0/3311306214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434107650 0x7f3434107a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 --2- 172.21.15.7:0/3311306214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 -- 172.21.15.7:0/3311306214 >> 172.21.15.7:0/3311306214 conn(0x7f3434078770 msgr2=0x7f3434078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 -- 172.21.15.7:0/3311306214 shutdown_connections 2024-01-11T17:56:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.170+0000 7f343b182700 1 -- 172.21.15.7:0/3311306214 wait complete. 2024-01-11T17:56:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.171+0000 7f343b182700 1 Processor -- start 2024-01-11T17:56:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.171+0000 7f343b182700 1 -- start start 2024-01-11T17:56:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343b182700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434108580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343b182700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 0x7f3434108ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343b182700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434107650 0x7f3434109000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343b182700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34341aa340 con 0x7f3434102020 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343b182700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34341aa480 con 0x7f3434107650 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343b182700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34341aa5c0 con 0x7f3434101660 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3433fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 0x7f3434108ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3433fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 0x7f3434108ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40066/0 (socket says 172.21.15.7:40066) 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3433fff700 1 -- 172.21.15.7:0/964369797 learned_addr learned my addr 172.21.15.7:0/964369797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38916/0 (socket says 172.21.15.7:38916) 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f343971f700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434107650 0x7f3434109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 -- 172.21.15.7:0/964369797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434107650 msgr2=0x7f3434109000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434107650 0x7f3434109000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 -- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 msgr2=0x7f3434108ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 0x7f3434108ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3438f1e700 1 -- 172.21.15.7:0/964369797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280095f0 con 0x7f3434101660 2024-01-11T17:56:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.172+0000 7f3433fff700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 0x7f3434108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.173+0000 7f3438f1e700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434108580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3428000c00 tx=0x7f3428005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.173+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342801d070 con 0x7f3434101660 2024-01-11T17:56:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.173+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34341aa840 con 0x7f3434101660 2024-01-11T17:56:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.173+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3428022470 con 0x7f3434101660 2024-01-11T17:56:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.173+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428017650 con 0x7f3434101660 2024-01-11T17:56:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.173+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34341aacb0 con 0x7f3434101660 2024-01-11T17:56:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.174+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3434102d90 con 0x7f3434101660 2024-01-11T17:56:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.176+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f342800fad0 con 0x7f3434101660 2024-01-11T17:56:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.178+0000 7f3431ffb700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34240a1ee0 0x7f34240a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.178+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34280cf500 con 0x7f3434101660 2024-01-11T17:56:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.178+0000 7f3433fff700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34240a1ee0 0x7f34240a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.179+0000 7f3433fff700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34240a1ee0 0x7f34240a43a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3434109d40 tx=0x7f3420009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.180+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f34280983e0 con 0x7f3434101660 2024-01-11T17:56:18.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.438+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3434111640 con 0x7f3434101660 2024-01-11T17:56:18.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.439+0000 7f3431ffb700 1 -- 172.21.15.7:0/964369797 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7f342809ba00 con 0x7f3434101660 2024-01-11T17:56:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-11T17:56:18.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34240a1ee0 msgr2=0x7f34240a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:18.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34240a1ee0 0x7f34240a43a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3434109d40 tx=0x7f3420009260 comp rx=0 tx=0).stop 2024-01-11T17:56:18.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 msgr2=0x7f3434108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:18.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434108580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3428000c00 tx=0x7f3428005740 comp rx=0 tx=0).stop 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 shutdown_connections 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34240a1ee0 0x7f34240a43a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3434101660 0x7f3434108580 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434107650 0x7f3434109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 --2- 172.21.15.7:0/964369797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3434102020 0x7f3434108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.441+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 >> 172.21.15.7:0/964369797 conn(0x7f3434078770 msgr2=0x7f34340ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.442+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 shutdown_connections 2024-01-11T17:56:18.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:18.442+0000 7f343b182700 1 -- 172.21.15.7:0/964369797 wait complete. 2024-01-11T17:56:18.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:18 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/964369797' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-11T17:56:18.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:18 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/964369797' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-11T17:56:18.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:18 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/964369797' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-11T17:56:18.848 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T17:56:18.848 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T17:56:18.848 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T17:56:18.849 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-01-11T17:56:18.849 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:18.849 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T17:56:18.849 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-11T17:56:18.849 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T17:56:18.849 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-01-11T17:56:18.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:18.849 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T17:56:18.895 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42041sec preferred_lft 42041sec 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T17:56:18.896 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T17:56:18.897 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-11T17:56:18.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:18.897 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:56:18.897 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:56:18.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-01-11T17:56:18.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-01-11T17:56:18.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-01-11T17:56:18.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-11T17:56:18.898 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:56:19.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:19.382 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T17:56:19.453 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T17:56:19.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:19.456 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T17:56:19.518 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T17:56:19.518 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T17:56:19.518 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T17:56:19.518 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T17:56:19.518 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T17:56:19.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:19.519 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:56:19.520 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:56:19.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T17:56:19.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T17:56:19.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T17:56:19.520 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:56:19.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:19 smithi007 ceph-mon[33361]: pgmap v251: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 151 B/s wr, 3 op/s 2024-01-11T17:56:19.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:19 smithi038 ceph-mon[43582]: pgmap v251: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 151 B/s wr, 3 op/s 2024-01-11T17:56:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:19 smithi016 ceph-mon[43195]: pgmap v251: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 151 B/s wr, 3 op/s 2024-01-11T17:56:20.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:20.340 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T17:56:20.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:20.386 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-01-11T17:56:20.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:20.470 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:56:20.470 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:56:20.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:20.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-01-11T17:56:20.470 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:56:20.657 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-01-11T17:56:20.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:20.657 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:56:20.657 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:56:20.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-01-11T17:56:20.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-11T17:56:20.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-01-11T17:56:20.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-01-11T17:56:20.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-01-11T17:56:20.658 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:56:20.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:20.991 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:56:20.992 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:56:20.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T17:56:20.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-01-11T17:56:20.992 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:56:21.170 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-01-11T17:56:21.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T17:56:21.171 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:21.214 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-01-11T17:56:21.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T17:56:21.215 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:21.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-01-11T17:56:21.408 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-01-11T17:56:21.452 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-01-11T17:56:21.452 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-01-11T17:56:21.452 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-01-11T17:56:21.452 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/systemd 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/perf_event 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/hugetlb 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/memory 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/devices 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/blkio 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/cpuset 2024-01-11T17:56:21.453 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/freezer 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/pids 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-01-11T17:56:21.454 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4aa34edbc6b1f6692bf1dd734d2332a806cc62efd19730b92558f6e1b6a6a77a/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/24c492caba31f4f736ce8a9ea91234f395deaed4ec31d26bd98bd45dfc26b297/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4d556f2f30f6d85f6ed961ffb5fadce021856cbaa7a64140cbe115c29d8567b5/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d3102359743f78fabe7b95c64c6f9e7672db3c18179f4479327ed6cef3fbf35a/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d67a1f995691072f157f7513f7f8b23ee7e1bea647dd9523c2bf08ac31ef1572/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/e66f4cad987f927e295c31f2aa3ff2d5a21f3846844acf0b8d5c1497add178b3/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/955a220b4ab39693075b37c675a2d917c9ced1c9dd7b6c39c87134bb67e1e2bc/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/33bbfa977eb92da0ee4d41e79f4b6cd5fb33d58ed03b493cf509ef2ae2a45a4d/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/50072246631df260b9b5bed694e36e3be62a2d98b2f3be076c38371644ab0781/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f878d6dc0c5ac5abc5a76b26cf749a01691338296d8380a566e1db3ecec1de4b/merged 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:21.455 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:21.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:21.456 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-01-11T17:56:21.541 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-11T17:56:21.541 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-01-11T17:56:21.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-01-11T17:56:21.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:21 smithi007 ceph-mon[33361]: pgmap v252: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-01-11T17:56:21.662 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-01-11T17:56:21.707 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-01-11T17:56:21.707 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-01-11T17:56:21.707 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/systemd 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/perf_event 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/hugetlb 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/memory 2024-01-11T17:56:21.708 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/devices 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/blkio 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/cpuset 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/freezer 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/pids 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-01-11T17:56:21.709 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4aa34edbc6b1f6692bf1dd734d2332a806cc62efd19730b92558f6e1b6a6a77a/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/24c492caba31f4f736ce8a9ea91234f395deaed4ec31d26bd98bd45dfc26b297/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4d556f2f30f6d85f6ed961ffb5fadce021856cbaa7a64140cbe115c29d8567b5/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d3102359743f78fabe7b95c64c6f9e7672db3c18179f4479327ed6cef3fbf35a/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d67a1f995691072f157f7513f7f8b23ee7e1bea647dd9523c2bf08ac31ef1572/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/e66f4cad987f927e295c31f2aa3ff2d5a21f3846844acf0b8d5c1497add178b3/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/955a220b4ab39693075b37c675a2d917c9ced1c9dd7b6c39c87134bb67e1e2bc/merged 2024-01-11T17:56:21.710 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/33bbfa977eb92da0ee4d41e79f4b6cd5fb33d58ed03b493cf509ef2ae2a45a4d/merged 2024-01-11T17:56:21.711 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/50072246631df260b9b5bed694e36e3be62a2d98b2f3be076c38371644ab0781/merged 2024-01-11T17:56:21.711 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f878d6dc0c5ac5abc5a76b26cf749a01691338296d8380a566e1db3ecec1de4b/merged 2024-01-11T17:56:21.711 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-01-11T17:56:21.711 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:21.711 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:21.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:21.712 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-01-11T17:56:21.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2024-01-11T17:56:21.779+0000 7f05f45fc5c0 -1 init, newargv = 0x55b11b2da030 newargc=15 2024-01-11T17:56:21.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[83691]: starting ceph client 2024-01-11T17:56:21.790 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[83691]: starting fuse 2024-01-11T17:56:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:21 smithi016 ceph-mon[43195]: pgmap v252: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-01-11T17:56:22.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:21 smithi038 ceph-mon[43582]: pgmap v252: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-01-11T17:56:22.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-01-11T17:56:22.937 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-01-11T17:56:22.983 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/systemd 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/perf_event 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/cpu,cpuacct 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/hugetlb 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/memory 2024-01-11T17:56:22.984 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/devices 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/net_cls,net_prio 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/blkio 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/cpuset 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/freezer 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup/pids 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-01-11T17:56:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4aa34edbc6b1f6692bf1dd734d2332a806cc62efd19730b92558f6e1b6a6a77a/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/24c492caba31f4f736ce8a9ea91234f395deaed4ec31d26bd98bd45dfc26b297/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4d556f2f30f6d85f6ed961ffb5fadce021856cbaa7a64140cbe115c29d8567b5/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d3102359743f78fabe7b95c64c6f9e7672db3c18179f4479327ed6cef3fbf35a/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/d67a1f995691072f157f7513f7f8b23ee7e1bea647dd9523c2bf08ac31ef1572/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/e66f4cad987f927e295c31f2aa3ff2d5a21f3846844acf0b8d5c1497add178b3/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/955a220b4ab39693075b37c675a2d917c9ced1c9dd7b6c39c87134bb67e1e2bc/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/33bbfa977eb92da0ee4d41e79f4b6cd5fb33d58ed03b493cf509ef2ae2a45a4d/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/50072246631df260b9b5bed694e36e3be62a2d98b2f3be076c38371644ab0781/merged 2024-01-11T17:56:22.986 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f878d6dc0c5ac5abc5a76b26cf749a01691338296d8380a566e1db3ecec1de4b/merged 2024-01-11T17:56:22.987 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-01-11T17:56:22.987 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:22.987 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:56:22.987 INFO:teuthology.orchestra.run.smithi007.stdout:/home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:22.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:22.988 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-01-11T17:56:23.072 INFO:teuthology.orchestra.run.smithi007.stdout:121 2024-01-11T17:56:23.073 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-01-11T17:56:23.073 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T17:56:23.074 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2024-01-11T17:56:23.075 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T17:56:23.259 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.83691.asok 2024-01-11T17:56:23.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-11T17:56:23.272 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.83691.asok 2024-01-11T17:56:23.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:23.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.83691.asok status 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "c27531add9155519f390b228e498f0a2c5eae117", 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-446-gc27531ad (c27531add9155519f390b228e498f0a2c5eae117) squid (dev)", 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-01-11T17:56:23.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "83691", 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24739, 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 24739 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1881834179 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:56:23.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1881834179 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.24739 172.21.15.7:0/1881834179", 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "172.21.15.7:0/1881834179", 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 16, 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 84, 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-01-11T17:56:23.506 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:56:23.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:23.519 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-01-11T17:56:23.519 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:23.563 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-01-11T17:56:23.564 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:23.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:23.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:23.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:23.675 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-01-11T17:56:23.808 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2024-01-11T17:56:23.808 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-01-11T17:56:23.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:23.809 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:56:23.809 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-11T17:56:23.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:23 smithi007 ceph-mon[33361]: pgmap v253: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 104 B/s wr, 2 op/s 2024-01-11T17:56:23.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:23 smithi016 ceph-mon[43195]: pgmap v253: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 104 B/s wr, 2 op/s 2024-01-11T17:56:24.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:23 smithi038 ceph-mon[43582]: pgmap v253: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 104 B/s wr, 2 op/s 2024-01-11T17:56:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 -- 172.21.15.7:0/1278918205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 msgr2=0x7ffa5c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1278918205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c1019b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa580099c0 tx=0x7ffa58009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 -- 172.21.15.7:0/1278918205 shutdown_connections 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1278918205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa5c103120 0x7ffa5c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1278918205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c1019b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1278918205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5c102780 0x7ffa5c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.097+0000 7ffa645ff700 1 -- 172.21.15.7:0/1278918205 >> 172.21.15.7:0/1278918205 conn(0x7ffa5c0fcb00 msgr2=0x7ffa5c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.098+0000 7ffa645ff700 1 -- 172.21.15.7:0/1278918205 shutdown_connections 2024-01-11T17:56:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.098+0000 7ffa645ff700 1 -- 172.21.15.7:0/1278918205 wait complete. 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 Processor -- start 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 -- start start 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5c102780 0x7ffa5c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa5c103120 0x7ffa5c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1a1200 con 0x7ffa5c101590 2024-01-11T17:56:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1a1340 con 0x7ffa5c102780 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa645ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1a14d0 con 0x7ffa5c103120 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa6239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa6239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52404/0 (socket says 172.21.15.7:52404) 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.099+0000 7ffa6239b700 1 -- 172.21.15.7:0/1657092980 learned_addr learned my addr 172.21.15.7:0/1657092980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa61b9a700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5c102780 0x7ffa5c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa62b9c700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa5c103120 0x7ffa5c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa6239b700 1 -- 172.21.15.7:0/1657092980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa5c103120 msgr2=0x7ffa5c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa6239b700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa5c103120 0x7ffa5c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa6239b700 1 -- 172.21.15.7:0/1657092980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5c102780 msgr2=0x7ffa5c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa6239b700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5c102780 0x7ffa5c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa6239b700 1 -- 172.21.15.7:0/1657092980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580095f0 con 0x7ffa5c101590 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa6239b700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c199680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa58003730 tx=0x7ffa5800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5801d070 con 0x7ffa5c101590 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.100+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa58005050 con 0x7ffa5c101590 2024-01-11T17:56:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.101+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c1a1750 con 0x7ffa5c101590 2024-01-11T17:56:24.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.101+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58004970 con 0x7ffa5c101590 2024-01-11T17:56:24.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.101+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c1a1c00 con 0x7ffa5c101590 2024-01-11T17:56:24.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.102+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c0683f0 con 0x7ffa5c101590 2024-01-11T17:56:24.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.104+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7ffa58005470 con 0x7ffa5c101590 2024-01-11T17:56:24.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.105+0000 7ffa537fe700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa480a1ee0 0x7ffa480a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.105+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffa580cec20 con 0x7ffa5c101590 2024-01-11T17:56:24.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.105+0000 7ffa61b9a700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa480a1ee0 0x7ffa480a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.106+0000 7ffa61b9a700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa480a1ee0 0x7ffa480a43a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009c60 tx=0x7ffa4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:24.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.107+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ffa58097b00 con 0x7ffa5c101590 2024-01-11T17:56:24.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.321+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa5c10cf80 con 0x7ffa5c101590 2024-01-11T17:56:24.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.322+0000 7ffa537fe700 1 -- 172.21.15.7:0/1657092980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23806 (secure 0 0 0) 0x7ffa5809b120 con 0x7ffa5c101590 2024-01-11T17:56:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:13.329124+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:24.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa480a1ee0 msgr2=0x7ffa480a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa480a1ee0 0x7ffa480a43a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009c60 tx=0x7ffa4c009260 comp rx=0 tx=0).stop 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 msgr2=0x7ffa5c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c199680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa58003730 tx=0x7ffa5800f690 comp rx=0 tx=0).stop 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 shutdown_connections 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa480a1ee0 0x7ffa480a43a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa5c103120 0x7ffa5c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa5c101590 0x7ffa5c199680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 --2- 172.21.15.7:0/1657092980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5c102780 0x7ffa5c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 >> 172.21.15.7:0/1657092980 conn(0x7ffa5c0fcb00 msgr2=0x7ffa5c107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 shutdown_connections 2024-01-11T17:56:24.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.325+0000 7ffa645ff700 1 -- 172.21.15.7:0/1657092980 wait complete. 2024-01-11T17:56:24.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:56:24.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 -- 172.21.15.7:0/3066025754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c107270 msgr2=0x7f452c1096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 --2- 172.21.15.7:0/3066025754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c107270 0x7f452c1096e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f452400b0a0 tx=0x7f452400b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 -- 172.21.15.7:0/3066025754 shutdown_connections 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 --2- 172.21.15.7:0/3066025754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c107270 0x7f452c1096e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 --2- 172.21.15.7:0/3066025754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c102010 0x7f452c104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 --2- 172.21.15.7:0/3066025754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f452c104940 0x7f452c106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 -- 172.21.15.7:0/3066025754 >> 172.21.15.7:0/3066025754 conn(0x7f452c0fba00 msgr2=0x7f452c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 -- 172.21.15.7:0/3066025754 shutdown_connections 2024-01-11T17:56:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.553+0000 7f4531fff700 1 -- 172.21.15.7:0/3066025754 wait complete. 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.554+0000 7f4531fff700 1 Processor -- start 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 -- start start 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 0x7f452c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c104940 0x7f452c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f452c107270 0x7f452c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c19f0d0 con 0x7f452c102010 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c19f210 con 0x7f452c104940 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f4531fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c19f3a0 con 0x7f452c107270 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 0x7f452c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 0x7f452c1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52418/0 (socket says 172.21.15.7:52418) 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 -- 172.21.15.7:0/950527213 learned_addr learned my addr 172.21.15.7:0/950527213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452bfff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f452c107270 0x7f452c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452affd700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c104940 0x7f452c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 -- 172.21.15.7:0/950527213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f452c107270 msgr2=0x7f452c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f452c107270 0x7f452c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 -- 172.21.15.7:0/950527213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c104940 msgr2=0x7f452c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.555+0000 7f452b7fe700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c104940 0x7f452c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f452b7fe700 1 -- 172.21.15.7:0/950527213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009d00 con 0x7f452c102010 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f452b7fe700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 0x7f452c1975a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9a0 tx=0x7f451c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c009790 con 0x7f452c102010 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451c010430 con 0x7f452c102010 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c19f680 con 0x7f452c102010 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c018680 con 0x7f452c102010 2024-01-11T17:56:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.556+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c19fb90 con 0x7f452c102010 2024-01-11T17:56:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.557+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0fd580 con 0x7f452c102010 2024-01-11T17:56:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.559+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f451c0105a0 con 0x7f452c102010 2024-01-11T17:56:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.562+0000 7f4528ff9700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45140a1ee0 0x7f45140a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.562+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f451c014070 con 0x7f452c102010 2024-01-11T17:56:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.562+0000 7f452affd700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45140a1ee0 0x7f45140a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.563+0000 7f452affd700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45140a1ee0 0x7f45140a43a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4520009c60 tx=0x7f4520009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.566+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f451c052380 con 0x7f452c102010 2024-01-11T17:56:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.781+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f452c02d450 con 0x7f452c102010 2024-01-11T17:56:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.781+0000 7f4528ff9700 1 -- 172.21.15.7:0/950527213 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f451c096be0 con 0x7f452c102010 2024-01-11T17:56:24.782 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:56:24.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.783+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45140a1ee0 msgr2=0x7f45140a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.783+0000 7f4531fff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45140a1ee0 0x7f45140a43a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4520009c60 tx=0x7f4520009260 comp rx=0 tx=0).stop 2024-01-11T17:56:24.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.783+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 msgr2=0x7f452c1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.783+0000 7f4531fff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 0x7f452c1975a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9a0 tx=0x7f451c00ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 shutdown_connections 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45140a1ee0 0x7f45140a43a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f452c107270 0x7f452c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102010 0x7f452c1975a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 --2- 172.21.15.7:0/950527213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c104940 0x7f452c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 >> 172.21.15.7:0/950527213 conn(0x7f452c0fba00 msgr2=0x7f452c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 shutdown_connections 2024-01-11T17:56:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:24.784+0000 7f4531fff700 1 -- 172.21.15.7:0/950527213 wait complete. 2024-01-11T17:56:24.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 -- 172.21.15.7:0/1944932944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881015b0 msgr2=0x7fa0881019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 --2- 172.21.15.7:0/1944932944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881015b0 0x7fa0881019d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 -- 172.21.15.7:0/1944932944 shutdown_connections 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 --2- 172.21.15.7:0/1944932944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881015b0 0x7fa0881019d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 --2- 172.21.15.7:0/1944932944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 --2- 172.21.15.7:0/1944932944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0881027a0 0x7fa088102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 -- 172.21.15.7:0/1944932944 >> 172.21.15.7:0/1944932944 conn(0x7fa0880fcb00 msgr2=0x7fa0880fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 -- 172.21.15.7:0/1944932944 shutdown_connections 2024-01-11T17:56:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.012+0000 7fa08ffff700 1 -- 172.21.15.7:0/1944932944 wait complete. 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 Processor -- start 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 -- start start 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0881015b0 0x7fa088199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881027a0 0x7fa088199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0881a1200 con 0x7fa088103140 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0881a1340 con 0x7fa0881015b0 2024-01-11T17:56:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.013+0000 7fa08ffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0881a14d0 con 0x7fa0881027a0 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52432/0 (socket says 172.21.15.7:52432) 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 -- 172.21.15.7:0/427617359 learned_addr learned my addr 172.21.15.7:0/427617359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08dd9b700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0881015b0 0x7fa088199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08d59a700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881027a0 0x7fa088199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 -- 172.21.15.7:0/427617359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881027a0 msgr2=0x7fa088199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881027a0 0x7fa088199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 -- 172.21.15.7:0/427617359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0881015b0 msgr2=0x7fa088199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0881015b0 0x7fa088199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 -- 172.21.15.7:0/427617359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840095f0 con 0x7fa088103140 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.014+0000 7fa08e59c700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881a0c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa08000ef30 tx=0x7fa08000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.015+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080018070 con 0x7fa088103140 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.015+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0881a1750 con 0x7fa088103140 2024-01-11T17:56:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.015+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08000f040 con 0x7fa088103140 2024-01-11T17:56:25.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.015+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08000cd00 con 0x7fa088103140 2024-01-11T17:56:25.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.015+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0881a1c60 con 0x7fa088103140 2024-01-11T17:56:25.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.016+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0880683f0 con 0x7fa088103140 2024-01-11T17:56:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.018+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa080003a00 con 0x7fa088103140 2024-01-11T17:56:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.019+0000 7fa07effd700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0740a1ee0 0x7fa0740a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.019+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa0800cd730 con 0x7fa088103140 2024-01-11T17:56:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.020+0000 7fa08dd9b700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0740a1ee0 0x7fa0740a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.020+0000 7fa08dd9b700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0740a1ee0 0x7fa0740a43a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa08400b680 tx=0x7fa08400b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:25.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.024+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa080096610 con 0x7fa088103140 2024-01-11T17:56:25.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.233+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0881a2040 con 0x7fa088103140 2024-01-11T17:56:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.234+0000 7fa07effd700 1 -- 172.21.15.7:0/427617359 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa080099c30 con 0x7fa088103140 2024-01-11T17:56:25.234 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0740a1ee0 msgr2=0x7fa0740a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0740a1ee0 0x7fa0740a43a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa08400b680 tx=0x7fa08400b750 comp rx=0 tx=0).stop 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 msgr2=0x7fa0881a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881a0c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa08000ef30 tx=0x7fa08000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 shutdown_connections 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0740a1ee0 0x7fa0740a43a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa088103140 0x7fa0881a0c30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0881015b0 0x7fa088199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 --2- 172.21.15.7:0/427617359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0881027a0 0x7fa088199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 >> 172.21.15.7:0/427617359 conn(0x7fa0880fcb00 msgr2=0x7fa088073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 shutdown_connections 2024-01-11T17:56:25.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.236+0000 7fa08ffff700 1 -- 172.21.15.7:0/427617359 wait complete. 2024-01-11T17:56:25.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:56:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 -- 172.21.15.7:0/3142975519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff358074070 msgr2=0x7ff3580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 --2- 172.21.15.7:0/3142975519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff358074070 0x7ff3580744f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff348009990 tx=0x7ff348009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 -- 172.21.15.7:0/3142975519 shutdown_connections 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 --2- 172.21.15.7:0/3142975519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff358075650 0x7ff358073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 --2- 172.21.15.7:0/3142975519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3581092c0 0x7ff35810b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 --2- 172.21.15.7:0/3142975519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff358074070 0x7ff3580744f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 -- 172.21.15.7:0/3142975519 >> 172.21.15.7:0/3142975519 conn(0x7ff3580fc970 msgr2=0x7ff3580fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 -- 172.21.15.7:0/3142975519 shutdown_connections 2024-01-11T17:56:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.468+0000 7ff35e99d700 1 -- 172.21.15.7:0/3142975519 wait complete. 2024-01-11T17:56:25.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.469+0000 7ff35e99d700 1 Processor -- start 2024-01-11T17:56:25.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 -- start start 2024-01-11T17:56:25.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff358074070 0x7ff3581994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff358075650 0x7ff358199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 0x7ff3581a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581a1040 con 0x7ff3581092c0 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581a1180 con 0x7ff358074070 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e99d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581a1310 con 0x7ff358075650 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35d99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff358074070 0x7ff3581994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 0x7ff3581a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35d99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff358074070 0x7ff3581994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50328/0 (socket says 172.21.15.7:50328) 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35e19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 0x7ff3581a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52456/0 (socket says 172.21.15.7:52456) 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35d99b700 1 -- 172.21.15.7:0/898251209 learned_addr learned my addr 172.21.15.7:0/898251209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.470+0000 7ff35d19a700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff358075650 0x7ff358199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e19c700 1 -- 172.21.15.7:0/898251209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff358075650 msgr2=0x7ff358199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e19c700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff358075650 0x7ff358199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e19c700 1 -- 172.21.15.7:0/898251209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff358074070 msgr2=0x7ff3581994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e19c700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff358074070 0x7ff3581994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e19c700 1 -- 172.21.15.7:0/898251209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480095f0 con 0x7ff3581092c0 2024-01-11T17:56:25.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e19c700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 0x7ff3581a0a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff35000efc0 tx=0x7ff35000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350018070 con 0x7ff3581092c0 2024-01-11T17:56:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff35000f040 con 0x7ff3581092c0 2024-01-11T17:56:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3581a1590 con 0x7ff3581092c0 2024-01-11T17:56:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000cd40 con 0x7ff3581092c0 2024-01-11T17:56:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.471+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3581a1a50 con 0x7ff3581092c0 2024-01-11T17:56:25.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.472+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358108080 con 0x7ff3581092c0 2024-01-11T17:56:25.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.475+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7ff350003a00 con 0x7ff3581092c0 2024-01-11T17:56:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.476+0000 7ff34effd700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3440a1ec0 0x7ff3440a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.477+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3500ce230 con 0x7ff3581092c0 2024-01-11T17:56:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.477+0000 7ff35d99b700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3440a1ec0 0x7ff3440a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.477+0000 7ff35d99b700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3440a1ec0 0x7ff3440a4380 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff354009d20 tx=0x7ff354009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:25.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.478+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff350097050 con 0x7ff3581092c0 2024-01-11T17:56:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1657092980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:25.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: pgmap v254: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/950527213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:25.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:25 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/427617359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:56:25.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.692+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff35810ee00 con 0x7ff3581092c0 2024-01-11T17:56:25.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.692+0000 7ff34effd700 1 -- 172.21.15.7:0/898251209 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff35009a670 con 0x7ff3581092c0 2024-01-11T17:56:25.693 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:56:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3440a1ec0 msgr2=0x7ff3440a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3440a1ec0 0x7ff3440a4380 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff354009d20 tx=0x7ff354009260 comp rx=0 tx=0).stop 2024-01-11T17:56:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 msgr2=0x7ff3581a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 0x7ff3581a0a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff35000efc0 tx=0x7ff35000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 shutdown_connections 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3440a1ec0 0x7ff3440a4380 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3581092c0 0x7ff3581a0a70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff358074070 0x7ff3581994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 --2- 172.21.15.7:0/898251209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff358075650 0x7ff358199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 >> 172.21.15.7:0/898251209 conn(0x7ff3580fc970 msgr2=0x7ff3581051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 shutdown_connections 2024-01-11T17:56:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.695+0000 7ff35e99d700 1 -- 172.21.15.7:0/898251209 wait complete. 2024-01-11T17:56:25.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:56:25.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1657092980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:25.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-01-11T17:56:25.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-01-11T17:56:25.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-11T17:56:25.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: pgmap v254: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/950527213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:25 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/427617359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:56:25.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1657092980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:25.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-01-11T17:56:25.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-01-11T17:56:25.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-11T17:56:25.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: pgmap v254: 129 pgs: 129 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/950527213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:25.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:25 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/427617359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 -- 172.21.15.7:0/238565100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1015b0 msgr2=0x7f252c1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 --2- 172.21.15.7:0/238565100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1015b0 0x7f252c1019d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f251c0098f0 tx=0x7f251c009c00 comp rx=0 tx=0).stop 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 -- 172.21.15.7:0/238565100 shutdown_connections 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 --2- 172.21.15.7:0/238565100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1015b0 0x7f252c1019d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 --2- 172.21.15.7:0/238565100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c103140 0x7f252c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 --2- 172.21.15.7:0/238565100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f252c1027a0 0x7f252c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 -- 172.21.15.7:0/238565100 >> 172.21.15.7:0/238565100 conn(0x7f252c0fcb00 msgr2=0x7f252c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 -- 172.21.15.7:0/238565100 shutdown_connections 2024-01-11T17:56:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.921+0000 7f25329ff700 1 -- 172.21.15.7:0/238565100 wait complete. 2024-01-11T17:56:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 Processor -- start 2024-01-11T17:56:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 -- start start 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 0x7f252c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1027a0 0x7f252c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f252c103140 0x7f252c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c1a12e0 con 0x7f252c1015b0 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c1a1420 con 0x7f252c103140 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.922+0000 7f25329ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c1a15b0 con 0x7f252c1027a0 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 0x7f252c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 0x7f252c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52478/0 (socket says 172.21.15.7:52478) 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 -- 172.21.15.7:0/3064719299 learned_addr learned my addr 172.21.15.7:0/3064719299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252b7fe700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1027a0 0x7f252c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f2530f9c700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f252c103140 0x7f252c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 -- 172.21.15.7:0/3064719299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1027a0 msgr2=0x7f252c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1027a0 0x7f252c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 -- 172.21.15.7:0/3064719299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f252c103140 msgr2=0x7f252c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f252c103140 0x7f252c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.923+0000 7f252bfff700 1 -- 172.21.15.7:0/3064719299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009d00 con 0x7f252c1015b0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.924+0000 7f252bfff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 0x7f252c1997b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f251c005e90 tx=0x7f251c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.924+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c01c070 con 0x7f252c1015b0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.924+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f251c005040 con 0x7f252c1015b0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.924+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c003900 con 0x7f252c1015b0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.924+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c009520 con 0x7f252c1015b0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.924+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c1a1a70 con 0x7f252c1015b0 2024-01-11T17:56:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.925+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c068460 con 0x7f252c1015b0 2024-01-11T17:56:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.927+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f251c020030 con 0x7f252c1015b0 2024-01-11T17:56:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.929+0000 7f25297fa700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25140a1ee0 0x7f25140a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.929+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f251c0cec90 con 0x7f252c1015b0 2024-01-11T17:56:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.929+0000 7f252b7fe700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25140a1ee0 0x7f25140a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.930+0000 7f252b7fe700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25140a1ee0 0x7f25140a43a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2520009c60 tx=0x7f2520009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:25.933+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f252c068460 con 0x7f252c1015b0 2024-01-11T17:56:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.144+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f252c0683f0 con 0x7f252c1015b0 2024-01-11T17:56:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.145+0000 7f25297fa700 1 -- 172.21.15.7:0/3064719299 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f251c097a50 con 0x7f252c1015b0 2024-01-11T17:56:26.145 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:56:26.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25140a1ee0 msgr2=0x7f25140a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25140a1ee0 0x7f25140a43a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2520009c60 tx=0x7f2520009260 comp rx=0 tx=0).stop 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 msgr2=0x7f252c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 0x7f252c1997b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f251c005e90 tx=0x7f251c00bf80 comp rx=0 tx=0).stop 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 shutdown_connections 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25140a1ee0 0x7f25140a43a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f252c103140 0x7f252c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c1015b0 0x7f252c1997b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 --2- 172.21.15.7:0/3064719299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f252c1027a0 0x7f252c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 >> 172.21.15.7:0/3064719299 conn(0x7f252c0fcb00 msgr2=0x7f252c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 shutdown_connections 2024-01-11T17:56:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.147+0000 7f25329ff700 1 -- 172.21.15.7:0/3064719299 wait complete. 2024-01-11T17:56:26.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-11T17:56:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/206058251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 msgr2=0x7f2e64103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/206058251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64103c30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009920 tx=0x7f2e60009c30 comp rx=0 tx=0).stop 2024-01-11T17:56:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/206058251 shutdown_connections 2024-01-11T17:56:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/206058251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 0x7f2e6410bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/206058251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64103c30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/206058251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 0x7f2e64104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/206058251 >> 172.21.15.7:0/206058251 conn(0x7f2e640fedc0 msgr2=0x7f2e641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:26.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/206058251 shutdown_connections 2024-01-11T17:56:26.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.379+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/206058251 wait complete. 2024-01-11T17:56:26.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 Processor -- start 2024-01-11T17:56:26.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 -- start start 2024-01-11T17:56:26.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 0x7f2e64199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 0x7f2e641a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641a1280 con 0x7f2e64104a00 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641a13c0 con 0x7f2e64103810 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.381+0000 7f2e6c7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641a1550 con 0x7f2e641053a0 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e69d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 0x7f2e64199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50356/0 (socket says 172.21.15.7:50356) 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 -- 172.21.15.7:0/1201142407 learned_addr learned my addr 172.21.15.7:0/1201142407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6ad9c700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 0x7f2e641a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 -- 172.21.15.7:0/1201142407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 msgr2=0x7f2e641a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 0x7f2e641a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 -- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 msgr2=0x7f2e64199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 0x7f2e64199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 -- 172.21.15.7:0/1201142407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e540095f0 con 0x7f2e64103810 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6ad9c700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 0x7f2e641a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e69d9a700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 0x7f2e64199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.382+0000 7f2e6a59b700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64199700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009230 tx=0x7f2e6000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.383+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6001c070 con 0x7f2e64103810 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.383+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6000bba0 con 0x7f2e64103810 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.383+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e600049c0 con 0x7f2e64103810 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.383+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e60009520 con 0x7f2e64103810 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.383+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e641a1a10 con 0x7f2e64103810 2024-01-11T17:56:26.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.384+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e64068840 con 0x7f2e64103810 2024-01-11T17:56:26.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.386+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f2e60020070 con 0x7f2e64103810 2024-01-11T17:56:26.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.387+0000 7f2e5b7fe700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e500a1ee0 0x7f2e500a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.387+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2e600cea20 con 0x7f2e64103810 2024-01-11T17:56:26.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.387+0000 7f2e69d9a700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e500a1ee0 0x7f2e500a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.388+0000 7f2e69d9a700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e500a1ee0 0x7f2e500a43a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005fd0 tx=0x7f2e54009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:26.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.392+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2e600977e0 con 0x7f2e64103810 2024-01-11T17:56:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-01-11T17:56:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-01-11T17:56:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-11T17:56:26.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: osdmap e85: 12 total, 12 up, 12 in 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/898251209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:56:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:26 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3064719299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:56:26.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.621+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e6410ac90 con 0x7f2e64103810 2024-01-11T17:56:26.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.622+0000 7f2e5b7fe700 1 -- 172.21.15.7:0/1201142407 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+579 (secure 0 0 0) 0x7f2e60027260 con 0x7f2e64103810 2024-01-11T17:56:26.623 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3570192633 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:26.623 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/867047662 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:26.623 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1912965293 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:26.623 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3811163237 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:26.624 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3086228555 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:26.624 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/867047662 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:26.624 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4071850608 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:26.624 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1981932676 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:26.624 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1981932676 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:26.624 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/113093279 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e500a1ee0 msgr2=0x7f2e500a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e500a1ee0 0x7f2e500a43a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005fd0 tx=0x7f2e54009310 comp rx=0 tx=0).stop 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 msgr2=0x7f2e64199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64199700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009230 tx=0x7f2e6000bef0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 shutdown_connections 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e500a1ee0 0x7f2e500a43a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e641053a0 0x7f2e641a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e64103810 0x7f2e64199700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 --2- 172.21.15.7:0/1201142407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e64104a00 0x7f2e64199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.624+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 >> 172.21.15.7:0/1201142407 conn(0x7f2e640fedc0 msgr2=0x7f2e64107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.625+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 shutdown_connections 2024-01-11T17:56:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.625+0000 7f2e6c7ff700 1 -- 172.21.15.7:0/1201142407 wait complete. 2024-01-11T17:56:26.626 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-01-11T17:56:26.643 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:56:26.643 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-11T17:56:26.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-01-11T17:56:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-01-11T17:56:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-11T17:56:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-01-11T17:56:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: osdmap e85: 12 total, 12 up, 12 in 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/898251209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:56:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:26 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3064719299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-01-11T17:56:26.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-01-11T17:56:26.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-01-11T17:56:26.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-01-11T17:56:26.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-01-11T17:56:26.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: osdmap e85: 12 total, 12 up, 12 in 2024-01-11T17:56:26.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/898251209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:56:26.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:26 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3064719299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:56:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/4125015813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44103790 msgr2=0x7fbb44103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/4125015813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44103790 0x7fbb44103bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb340099c0 tx=0x7fbb34009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/4125015813 shutdown_connections 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/4125015813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44105320 0x7fbb4410bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/4125015813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44103790 0x7fbb44103bb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/4125015813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44104980 0x7fbb44104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.931+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/4125015813 >> 172.21.15.7:0/4125015813 conn(0x7fbb440fed20 msgr2=0x7fbb44101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.932+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/4125015813 shutdown_connections 2024-01-11T17:56:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.932+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/4125015813 wait complete. 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 Processor -- start 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 -- start start 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44103790 0x7fbb441996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44104980 0x7fbb44199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 0x7fbb441a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441a1250 con 0x7fbb44105320 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441a1390 con 0x7fbb44104980 2024-01-11T17:56:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441a1520 con 0x7fbb44103790 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4899c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 0x7fbb441a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb43fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44103790 0x7fbb441996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb4899c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 0x7fbb441a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52500/0 (socket says 172.21.15.7:52500) 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 -- 172.21.15.7:0/2951089355 learned_addr learned my addr 172.21.15.7:0/2951089355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.933+0000 7fbb43fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44103790 0x7fbb441996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52836/0 (socket says 172.21.15.7:52836) 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb437fe700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44104980 0x7fbb44199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 -- 172.21.15.7:0/2951089355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44103790 msgr2=0x7fbb441996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44103790 0x7fbb441996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 -- 172.21.15.7:0/2951089355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44104980 msgr2=0x7fbb44199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44104980 0x7fbb44199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 -- 172.21.15.7:0/2951089355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb38009520 con 0x7fbb44105320 2024-01-11T17:56:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4899c700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 0x7fbb441a0c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ef30 tx=0x7fbb3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c018070 con 0x7fbb44105320 2024-01-11T17:56:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3c00f040 con 0x7fbb44105320 2024-01-11T17:56:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00cd00 con 0x7fbb44105320 2024-01-11T17:56:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.934+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb340095f0 con 0x7fbb44105320 2024-01-11T17:56:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.935+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb441a1ad0 con 0x7fbb44105320 2024-01-11T17:56:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.936+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4410ac10 con 0x7fbb44105320 2024-01-11T17:56:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.937+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fbb3c003a00 con 0x7fbb44105320 2024-01-11T17:56:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.939+0000 7fbb417fa700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbb2c0a1f60 0x7fbb2c0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.939+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbb3c0cd7c0 con 0x7fbb44105320 2024-01-11T17:56:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.939+0000 7fbb43fff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbb2c0a1f60 0x7fbb2c0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.940+0000 7fbb43fff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbb2c0a1f60 0x7fbb2c0a4420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb34003730 tx=0x7fbb34000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:26.943+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fbb3c096580 con 0x7fbb44105320 2024-01-11T17:56:27.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.159+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb44068840 con 0x7fbb44105320 2024-01-11T17:56:27.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.161+0000 7fbb417fa700 1 -- 172.21.15.7:0/2951089355 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7fbb3c099ba0 con 0x7fbb44105320 2024-01-11T17:56:27.162 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:27.162 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbb2c0a1f60 msgr2=0x7fbb2c0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbb2c0a1f60 0x7fbb2c0a4420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb34003730 tx=0x7fbb34000bc0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 msgr2=0x7fbb441a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 0x7fbb441a0c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ef30 tx=0x7fbb3c00c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 shutdown_connections 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbb2c0a1f60 0x7fbb2c0a4420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb44103790 0x7fbb441996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb44105320 0x7fbb441a0c80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 --2- 172.21.15.7:0/2951089355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44104980 0x7fbb44199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 >> 172.21.15.7:0/2951089355 conn(0x7fbb440fed20 msgr2=0x7fbb44107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 shutdown_connections 2024-01-11T17:56:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.164+0000 7fbb4a3ff700 1 -- 172.21.15.7:0/2951089355 wait complete. 2024-01-11T17:56:27.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:56:27.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:56:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.391+0000 7f200a7ff700 1 -- 172.21.15.7:0/731254004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20041027a0 msgr2=0x7f2004102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.391+0000 7f200a7ff700 1 --2- 172.21.15.7:0/731254004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20041027a0 0x7f2004102c00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80099c0 tx=0x7f1ff8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 -- 172.21.15.7:0/731254004 shutdown_connections 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 --2- 172.21.15.7:0/731254004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2004103140 0x7f20041099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 --2- 172.21.15.7:0/731254004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041015b0 0x7f20041019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 --2- 172.21.15.7:0/731254004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20041027a0 0x7f2004102c00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 -- 172.21.15.7:0/731254004 >> 172.21.15.7:0/731254004 conn(0x7f20040fcb00 msgr2=0x7f20040fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 -- 172.21.15.7:0/731254004 shutdown_connections 2024-01-11T17:56:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.392+0000 7f200a7ff700 1 -- 172.21.15.7:0/731254004 wait complete. 2024-01-11T17:56:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 Processor -- start 2024-01-11T17:56:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 -- start start 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20041015b0 0x7f2004197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041027a0 0x7f2004197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 0x7f200419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200419efc0 con 0x7f2004103140 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200419f100 con 0x7f20041027a0 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f200a7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200419f290 con 0x7f20041015b0 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f2008d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 0x7f200419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f20037fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041027a0 0x7f2004197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f2008d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 0x7f200419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52516/0 (socket says 172.21.15.7:52516) 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f2008d9c700 1 -- 172.21.15.7:0/3694431243 learned_addr learned my addr 172.21.15.7:0/3694431243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.393+0000 7f20037fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041027a0 0x7f2004197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50392/0 (socket says 172.21.15.7:50392) 2024-01-11T17:56:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2003fff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20041015b0 0x7f2004197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2008d9c700 1 -- 172.21.15.7:0/3694431243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20041015b0 msgr2=0x7f2004197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2008d9c700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20041015b0 0x7f2004197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2008d9c700 1 -- 172.21.15.7:0/3694431243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041027a0 msgr2=0x7f2004197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2008d9c700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041027a0 0x7f2004197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2008d9c700 1 -- 172.21.15.7:0/3694431243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80095f0 con 0x7f2004103140 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f2008d9c700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 0x7f200419e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00ef30 tx=0x7f1ffc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc018070 con 0x7f2004103140 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ffc00f040 con 0x7f2004103140 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00cd00 con 0x7f2004103140 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.394+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200419f510 con 0x7f2004103140 2024-01-11T17:56:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.395+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200419fa20 con 0x7f2004103140 2024-01-11T17:56:27.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.396+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2004108900 con 0x7f2004103140 2024-01-11T17:56:27.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.399+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1ffc003a00 con 0x7f2004103140 2024-01-11T17:56:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.400+0000 7f20017fa700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1fec0a1f60 0x7f1fec0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.401+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ffc0cdc00 con 0x7f2004103140 2024-01-11T17:56:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.401+0000 7f2003fff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1fec0a1f60 0x7f1fec0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.401+0000 7f2003fff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1fec0a1f60 0x7f1fec0a4420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2004102600 tx=0x7f1ff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.403+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1ffc0969c0 con 0x7f2004103140 2024-01-11T17:56:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:27 smithi007 ceph-mon[33361]: osdmap e86: 12 total, 12 up, 12 in 2024-01-11T17:56:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:27 smithi007 ceph-mon[33361]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-01-11T17:56:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:27 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1201142407' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T17:56:27.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:27 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2951089355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.616+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f200410cfb0 con 0x7f2004103140 2024-01-11T17:56:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.616+0000 7f20017fa700 1 -- 172.21.15.7:0/3694431243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ffc099fe0 con 0x7f2004103140 2024-01-11T17:56:27.617 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:56:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1fec0a1f60 msgr2=0x7f1fec0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1fec0a1f60 0x7f1fec0a4420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2004102600 tx=0x7f1ff4009260 comp rx=0 tx=0).stop 2024-01-11T17:56:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 msgr2=0x7f200419e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 0x7f200419e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00ef30 tx=0x7f1ffc00c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 shutdown_connections 2024-01-11T17:56:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1fec0a1f60 0x7f1fec0a4420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20041015b0 0x7f2004197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2004103140 0x7f200419e9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 --2- 172.21.15.7:0/3694431243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20041027a0 0x7f2004197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.618+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 >> 172.21.15.7:0/3694431243 conn(0x7f20040fcb00 msgr2=0x7f2004105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.619+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 shutdown_connections 2024-01-11T17:56:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.619+0000 7f200a7ff700 1 -- 172.21.15.7:0/3694431243 wait complete. 2024-01-11T17:56:27.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:56:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:27 smithi016 ceph-mon[43195]: osdmap e86: 12 total, 12 up, 12 in 2024-01-11T17:56:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:27 smithi016 ceph-mon[43195]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-01-11T17:56:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:27 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1201142407' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T17:56:27.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:27 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2951089355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:27 smithi038 ceph-mon[43582]: osdmap e86: 12 total, 12 up, 12 in 2024-01-11T17:56:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:27 smithi038 ceph-mon[43582]: pgmap v257: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-01-11T17:56:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:27 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1201142407' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T17:56:27.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:27 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2951089355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 -- 172.21.15.7:0/2884707084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc105340 msgr2=0x7f9ddc10bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 --2- 172.21.15.7:0/2884707084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc105340 0x7f9ddc10bbf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400b0d0 tx=0x7f9dd400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 -- 172.21.15.7:0/2884707084 shutdown_connections 2024-01-11T17:56:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 --2- 172.21.15.7:0/2884707084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc105340 0x7f9ddc10bbf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 --2- 172.21.15.7:0/2884707084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 --2- 172.21.15.7:0/2884707084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ddc1049a0 0x7f9ddc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 -- 172.21.15.7:0/2884707084 >> 172.21.15.7:0/2884707084 conn(0x7f9ddc0fed20 msgr2=0x7f9ddc101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 -- 172.21.15.7:0/2884707084 shutdown_connections 2024-01-11T17:56:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.844+0000 7f9de15ff700 1 -- 172.21.15.7:0/2884707084 wait complete. 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.845+0000 7f9de15ff700 1 Processor -- start 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.845+0000 7f9de15ff700 1 -- start start 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9de15ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9de15ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ddc1049a0 0x7f9ddc199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9de15ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ddc105340 0x7f9ddc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9de15ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1a12e0 con 0x7f9ddc1037b0 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9de15ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1a1420 con 0x7f9ddc105340 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9de15ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1a15b0 con 0x7f9ddc1049a0 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52538/0 (socket says 172.21.15.7:52538) 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 -- 172.21.15.7:0/810243045 learned_addr learned my addr 172.21.15.7:0/810243045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddb7fe700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ddc105340 0x7f9ddc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9dda7fc700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ddc1049a0 0x7f9ddc199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 -- 172.21.15.7:0/810243045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ddc1049a0 msgr2=0x7f9ddc199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ddc1049a0 0x7f9ddc199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 -- 172.21.15.7:0/810243045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ddc105340 msgr2=0x7f9ddc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ddc105340 0x7f9ddc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.846+0000 7f9ddaffd700 1 -- 172.21.15.7:0/810243045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4009d00 con 0x7f9ddc1037b0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.847+0000 7f9ddaffd700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc199760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e970 tx=0x7f9dcc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.847+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc009790 con 0x7f9ddc1037b0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.847+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dcc00c7e0 con 0x7f9ddc1037b0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.847+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc1a1890 con 0x7f9ddc1037b0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.847+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc003c70 con 0x7f9ddc1037b0 2024-01-11T17:56:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.847+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc1a1df0 con 0x7f9ddc1037b0 2024-01-11T17:56:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.848+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc10ab00 con 0x7f9ddc1037b0 2024-01-11T17:56:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.850+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f9dcc00cc50 con 0x7f9ddc1037b0 2024-01-11T17:56:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.851+0000 7f9dc3fff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9dc40a1f60 0x7f9dc40a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.852+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9dcc014070 con 0x7f9ddc1037b0 2024-01-11T17:56:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.852+0000 7f9dda7fc700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9dc40a1f60 0x7f9dc40a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.852+0000 7f9dda7fc700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9dc40a1f60 0x7f9dc40a4420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009c60 tx=0x7f9dd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:27.855+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f9dcc096b60 con 0x7f9ddc1037b0 2024-01-11T17:56:28.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.064+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ddc02d420 con 0x7f9ddc1037b0 2024-01-11T17:56:28.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.065+0000 7f9dc3fff700 1 -- 172.21.15.7:0/810243045 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9dcc003dd0 con 0x7f9ddc1037b0 2024-01-11T17:56:28.066 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:56:28.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9dc40a1f60 msgr2=0x7f9dc40a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9dc40a1f60 0x7f9dc40a4420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009c60 tx=0x7f9dd0009260 comp rx=0 tx=0).stop 2024-01-11T17:56:28.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 msgr2=0x7f9ddc199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc199760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e970 tx=0x7f9dcc00ec80 comp rx=0 tx=0).stop 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 shutdown_connections 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9dc40a1f60 0x7f9dc40a4420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ddc105340 0x7f9ddc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc1037b0 0x7f9ddc199760 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 --2- 172.21.15.7:0/810243045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ddc1049a0 0x7f9ddc199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 >> 172.21.15.7:0/810243045 conn(0x7f9ddc0fed20 msgr2=0x7f9ddc107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 shutdown_connections 2024-01-11T17:56:28.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.068+0000 7f9de15ff700 1 -- 172.21.15.7:0/810243045 wait complete. 2024-01-11T17:56:28.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 -- 172.21.15.7:0/3192666302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901015b0 msgr2=0x7f6f901019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3192666302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901015b0 0x7f6f901019d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0099c0 tx=0x7f6f8c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 -- 172.21.15.7:0/3192666302 shutdown_connections 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3192666302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f90103140 0x7f6f901099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3192666302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901015b0 0x7f6f901019d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3192666302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f901027a0 0x7f6f90102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 -- 172.21.15.7:0/3192666302 >> 172.21.15.7:0/3192666302 conn(0x7f6f900fcb00 msgr2=0x7f6f900fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 -- 172.21.15.7:0/3192666302 shutdown_connections 2024-01-11T17:56:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.292+0000 7f6f979ff700 1 -- 172.21.15.7:0/3192666302 wait complete. 2024-01-11T17:56:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.293+0000 7f6f979ff700 1 Processor -- start 2024-01-11T17:56:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.293+0000 7f6f979ff700 1 -- start start 2024-01-11T17:56:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f979ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f901015b0 0x7f6f90199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f979ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 0x7f6f90199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f979ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f90103140 0x7f6f901a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f979ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901a1200 con 0x7f6f901027a0 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f979ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901a1340 con 0x7f6f90103140 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f979ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901a14d0 con 0x7f6f901015b0 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 0x7f6f90199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f9579b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f901015b0 0x7f6f90199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 0x7f6f90199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52544/0 (socket says 172.21.15.7:52544) 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 -- 172.21.15.7:0/3209839529 learned_addr learned my addr 172.21.15.7:0/3209839529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f95f9c700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f90103140 0x7f6f901a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 -- 172.21.15.7:0/3209839529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f901015b0 msgr2=0x7f6f90199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f901015b0 0x7f6f90199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 -- 172.21.15.7:0/3209839529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f90103140 msgr2=0x7f6f901a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f90103140 0x7f6f901a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.294+0000 7f6f94f9a700 1 -- 172.21.15.7:0/3209839529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0095f0 con 0x7f6f901027a0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.295+0000 7f6f94f9a700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 0x7f6f90199bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000e9a0 tx=0x7f6f8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.295+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80009790 con 0x7f6f901027a0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.295+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f80010430 con 0x7f6f901027a0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.295+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f901a17b0 con 0x7f6f901027a0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.295+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80018680 con 0x7f6f901027a0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.295+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f901a1d10 con 0x7f6f901027a0 2024-01-11T17:56:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.296+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f900683f0 con 0x7f6f901027a0 2024-01-11T17:56:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.298+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6f800105a0 con 0x7f6f901027a0 2024-01-11T17:56:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.300+0000 7f6f867fc700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f7c0a1f60 0x7f6f7c0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.300+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6f80014070 con 0x7f6f901027a0 2024-01-11T17:56:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.300+0000 7f6f9579b700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f7c0a1f60 0x7f6f7c0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.301+0000 7f6f9579b700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f7c0a1f60 0x7f6f7c0a4420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009990 tx=0x7f6f8c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.304+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6f80052380 con 0x7f6f901027a0 2024-01-11T17:56:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.515+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f901a2140 con 0x7f6f901027a0 2024-01-11T17:56:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.516+0000 7f6f867fc700 1 -- 172.21.15.7:0/3209839529 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f800d3020 con 0x7f6f901027a0 2024-01-11T17:56:28.517 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f7c0a1f60 msgr2=0x7f6f7c0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f7c0a1f60 0x7f6f7c0a4420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009990 tx=0x7f6f8c000bc0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 msgr2=0x7f6f90199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 0x7f6f90199bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000e9a0 tx=0x7f6f8000ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 shutdown_connections 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f7c0a1f60 0x7f6f7c0a4420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f901015b0 0x7f6f90199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f90103140 0x7f6f901a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 --2- 172.21.15.7:0/3209839529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f901027a0 0x7f6f90199bc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 >> 172.21.15.7:0/3209839529 conn(0x7f6f900fcb00 msgr2=0x7f6f90107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 shutdown_connections 2024-01-11T17:56:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.518+0000 7f6f979ff700 1 -- 172.21.15.7:0/3209839529 wait complete. 2024-01-11T17:56:28.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:56:28.600 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:28 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3694431243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:28.600 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:28 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/810243045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 -- 172.21.15.7:0/1275878815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 msgr2=0x7f508c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 --2- 172.21.15.7:0/1275878815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c1099f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f508400b0d0 tx=0x7f508400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 -- 172.21.15.7:0/1275878815 shutdown_connections 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 --2- 172.21.15.7:0/1275878815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c1015b0 0x7f508c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 --2- 172.21.15.7:0/1275878815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c1099f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 --2- 172.21.15.7:0/1275878815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 -- 172.21.15.7:0/1275878815 >> 172.21.15.7:0/1275878815 conn(0x7f508c0fcb00 msgr2=0x7f508c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:28.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 -- 172.21.15.7:0/1275878815 shutdown_connections 2024-01-11T17:56:28.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.744+0000 7f5093bff700 1 -- 172.21.15.7:0/1275878815 wait complete. 2024-01-11T17:56:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:28 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3694431243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:28 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/810243045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:56:28.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.745+0000 7f5093bff700 1 Processor -- start 2024-01-11T17:56:28.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.745+0000 7f5093bff700 1 -- start start 2024-01-11T17:56:28.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f5093bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c1015b0 0x7f508c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f5093bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f5093bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f5093bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c19f120 con 0x7f508c103140 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f5093bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c19f260 con 0x7f508c1027a0 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f5093bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c19f3f0 con 0x7f508c1015b0 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509119a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509219c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509119a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50464/0 (socket says 172.21.15.7:50464) 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509119a700 1 -- 172.21.15.7:0/1671244040 learned_addr learned my addr 172.21.15.7:0/1671244040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509199b700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c1015b0 0x7f508c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509219c700 1 -- 172.21.15.7:0/1671244040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c1015b0 msgr2=0x7f508c1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509219c700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c1015b0 0x7f508c1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509219c700 1 -- 172.21.15.7:0/1671244040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 msgr2=0x7f508c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509219c700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509219c700 1 -- 172.21.15.7:0/1671244040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009d00 con 0x7f508c103140 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.746+0000 7f509119a700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.747+0000 7f509219c700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c19eb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5084007b60 tx=0x7f5084008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.747+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508400e040 con 0x7f508c103140 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.747+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5084005070 con 0x7f508c103140 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.747+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c19f670 con 0x7f508c103140 2024-01-11T17:56:28.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.747+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084006000 con 0x7f508c103140 2024-01-11T17:56:28.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.747+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c19fa90 con 0x7f508c103140 2024-01-11T17:56:28.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.748+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c108900 con 0x7f508c103140 2024-01-11T17:56:28.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.750+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f50840051e0 con 0x7f508c103140 2024-01-11T17:56:28.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.751+0000 7f5082ffd700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50780a1ee0 0x7f50780a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:28.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.751+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50840cea40 con 0x7f508c103140 2024-01-11T17:56:28.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.752+0000 7f509199b700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50780a1ee0 0x7f50780a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:28.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.752+0000 7f509199b700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50780a1ee0 0x7f50780a43a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f508c102600 tx=0x7f5088009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:28.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.755+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f5084097890 con 0x7f508c103140 2024-01-11T17:56:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:28 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3694431243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:56:28.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:28 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/810243045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:56:28.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.971+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f508c02d490 con 0x7f508c103140 2024-01-11T17:56:28.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.971+0000 7f5082ffd700 1 -- 172.21.15.7:0/1671244040 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f508409aeb0 con 0x7f508c103140 2024-01-11T17:56:28.972 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50780a1ee0 msgr2=0x7f50780a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50780a1ee0 0x7f50780a43a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f508c102600 tx=0x7f5088009260 comp rx=0 tx=0).stop 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 msgr2=0x7f508c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c19eb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5084007b60 tx=0x7f5084008ec0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 shutdown_connections 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50780a1ee0 0x7f50780a43a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c1015b0 0x7f508c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508c103140 0x7f508c19eb50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 --2- 172.21.15.7:0/1671244040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c1027a0 0x7f508c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 >> 172.21.15.7:0/1671244040 conn(0x7f508c0fcb00 msgr2=0x7f508c105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.973+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 shutdown_connections 2024-01-11T17:56:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:28.974+0000 7f5093bff700 1 -- 172.21.15.7:0/1671244040 wait complete. 2024-01-11T17:56:28.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-11T17:56:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.198+0000 7f7e807ff700 1 -- 172.21.15.7:0/2782020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 msgr2=0x7f7e781019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.198+0000 7f7e807ff700 1 --2- 172.21.15.7:0/2782020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781019d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e740099c0 tx=0x7f7e74009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.198+0000 7f7e807ff700 1 -- 172.21.15.7:0/2782020196 shutdown_connections 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.199+0000 7f7e807ff700 1 --2- 172.21.15.7:0/2782020196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.199+0000 7f7e807ff700 1 --2- 172.21.15.7:0/2782020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781019d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.199+0000 7f7e807ff700 1 --2- 172.21.15.7:0/2782020196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 0x7f7e78102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.199+0000 7f7e807ff700 1 -- 172.21.15.7:0/2782020196 >> 172.21.15.7:0/2782020196 conn(0x7f7e780fcb00 msgr2=0x7f7e780fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.199+0000 7f7e807ff700 1 -- 172.21.15.7:0/2782020196 shutdown_connections 2024-01-11T17:56:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.199+0000 7f7e807ff700 1 -- 172.21.15.7:0/2782020196 wait complete. 2024-01-11T17:56:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 Processor -- start 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 -- start start 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 0x7f7e78199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a1200 con 0x7f7e78103140 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a1340 con 0x7f7e781015b0 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e807ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a14d0 con 0x7f7e781027a0 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e7ed9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e7e59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e7ed9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52568/0 (socket says 172.21.15.7:52568) 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e7e59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50480/0 (socket says 172.21.15.7:50480) 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.200+0000 7f7e7ed9c700 1 -- 172.21.15.7:0/1975919414 learned_addr learned my addr 172.21.15.7:0/1975919414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7dd9a700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 0x7f7e78199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7ed9c700 1 -- 172.21.15.7:0/1975919414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 msgr2=0x7f7e78199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7ed9c700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 0x7f7e78199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7ed9c700 1 -- 172.21.15.7:0/1975919414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 msgr2=0x7f7e781996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7ed9c700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7ed9c700 1 -- 172.21.15.7:0/1975919414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740095f0 con 0x7f7e78103140 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7dd9a700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 0x7f7e78199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e7ed9c700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781a0c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000ef30 tx=0x7f7e7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70018070 con 0x7f7e78103140 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e7000f040 con 0x7f7e78103140 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e781a1750 con 0x7f7e78103140 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.201+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000cdb0 con 0x7f7e78103140 2024-01-11T17:56:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.202+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e781a1c60 con 0x7f7e78103140 2024-01-11T17:56:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.203+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e780683f0 con 0x7f7e78103140 2024-01-11T17:56:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.204+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f7e70003a00 con 0x7f7e78103140 2024-01-11T17:56:29.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.206+0000 7f7e6f7fe700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7e640a1f60 0x7f7e640a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.206+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7e700cda20 con 0x7f7e78103140 2024-01-11T17:56:29.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.206+0000 7f7e7e59b700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7e640a1f60 0x7f7e640a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.207+0000 7f7e7e59b700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7e640a1f60 0x7f7e640a4420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009990 tx=0x7f7e74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.208+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f7e700967e0 con 0x7f7e78103140 2024-01-11T17:56:29.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.441+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e7810cfa0 con 0x7f7e78103140 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.442+0000 7f7e6f7fe700 1 -- 172.21.15.7:0/1975919414 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+579 (secure 0 0 0) 0x7f7e70099e00 con 0x7f7e78103140 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3570192633 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/867047662 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1912965293 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3811163237 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3086228555 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/867047662 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4071850608 2024-01-12T17:50:24.497330+0000 2024-01-11T17:56:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1981932676 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1981932676 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/113093279 2024-01-12T17:49:57.975302+0000 2024-01-11T17:56:29.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7e640a1f60 msgr2=0x7f7e640a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7e640a1f60 0x7f7e640a4420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009990 tx=0x7f7e74000bc0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 msgr2=0x7f7e781a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781a0c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000ef30 tx=0x7f7e7000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 shutdown_connections 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f7e640a1f60 0x7f7e640a4420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78103140 0x7f7e781a0c30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781015b0 0x7f7e781996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 --2- 172.21.15.7:0/1975919414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e781027a0 0x7f7e78199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 >> 172.21.15.7:0/1975919414 conn(0x7f7e780fcb00 msgr2=0x7f7e78073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 shutdown_connections 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.444+0000 7f7e807ff700 1 -- 172.21.15.7:0/1975919414 wait complete. 2024-01-11T17:56:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-01-11T17:56:29.463 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-01-11T17:56:29.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:29.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-01-11T17:56:29.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[83691]: fuse finished with error 0 and tester_r 0 2024-01-11T17:56:29.534 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T17:56:29.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:56:29.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:56:29.555 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-01-11T17:56:29.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:29.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:29 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3209839529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:56:29.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:29 smithi007 ceph-mon[33361]: pgmap v258: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-01-11T17:56:29.713 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:29 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1671244040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:56:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:29 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3209839529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:56:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:29 smithi016 ceph-mon[43195]: pgmap v258: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-01-11T17:56:29.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:29 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1671244040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:56:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:29 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3209839529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:56:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:29 smithi038 ceph-mon[43582]: pgmap v258: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-01-11T17:56:29.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:29 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1671244040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:56:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 -- 172.21.15.7:0/1607956671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 msgr2=0x7fc15c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 --2- 172.21.15.7:0/1607956671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c1019c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1500099c0 tx=0x7fc150009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 -- 172.21.15.7:0/1607956671 shutdown_connections 2024-01-11T17:56:29.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 --2- 172.21.15.7:0/1607956671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc15c103130 0x7fc15c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 --2- 172.21.15.7:0/1607956671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c1019c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 --2- 172.21.15.7:0/1607956671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c102790 0x7fc15c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 -- 172.21.15.7:0/1607956671 >> 172.21.15.7:0/1607956671 conn(0x7fc15c0fcb30 msgr2=0x7fc15c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 -- 172.21.15.7:0/1607956671 shutdown_connections 2024-01-11T17:56:29.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.857+0000 7fc1631ff700 1 -- 172.21.15.7:0/1607956671 wait complete. 2024-01-11T17:56:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.858+0000 7fc1631ff700 1 Processor -- start 2024-01-11T17:56:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.858+0000 7fc1631ff700 1 -- start start 2024-01-11T17:56:29.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc1631ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc1631ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c102790 0x7fc15c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc1631ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc15c103130 0x7fc15c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc1631ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c1a1210 con 0x7fc15c1015a0 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc1631ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c1a1350 con 0x7fc15c102790 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc1631ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c1a14e0 con 0x7fc15c103130 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52582/0 (socket says 172.21.15.7:52582) 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 -- 172.21.15.7:0/284602172 learned_addr learned my addr 172.21.15.7:0/284602172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc16179c700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc15c103130 0x7fc15c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc15bfff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c102790 0x7fc15c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 -- 172.21.15.7:0/284602172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc15c103130 msgr2=0x7fc15c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc15c103130 0x7fc15c1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 -- 172.21.15.7:0/284602172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c102790 msgr2=0x7fc15c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c102790 0x7fc15c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.859+0000 7fc160f9b700 1 -- 172.21.15.7:0/284602172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009520 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.860+0000 7fc160f9b700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c199690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc150003730 tx=0x7fc15000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.860+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15001d070 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.860+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc150005050 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.860+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1500095f0 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.860+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150004970 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.860+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c1a1a80 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.861+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c0683f0 con 0x7fc15c1015a0 2024-01-11T17:56:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.863+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc150005470 con 0x7fc15c1015a0 2024-01-11T17:56:29.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.864+0000 7fc159ffb700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc14c0a1ee0 0x7fc14c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:29.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.865+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1500ced40 con 0x7fc15c1015a0 2024-01-11T17:56:29.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.865+0000 7fc15bfff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc14c0a1ee0 0x7fc14c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:29.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.865+0000 7fc15bfff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc14c0a1ee0 0x7fc14c0a43a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc148009df0 tx=0x7fc148009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:29.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:29.867+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc150097b00 con 0x7fc15c1015a0 2024-01-11T17:56:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.080+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc15c10cfa0 con 0x7fc15c1015a0 2024-01-11T17:56:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.082+0000 7fc159ffb700 1 -- 172.21.15.7:0/284602172 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7fc15009b120 con 0x7fc15c1015a0 2024-01-11T17:56:30.083 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:30.083 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc14c0a1ee0 msgr2=0x7fc14c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc14c0a1ee0 0x7fc14c0a43a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc148009df0 tx=0x7fc148009260 comp rx=0 tx=0).stop 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 msgr2=0x7fc15c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c199690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc150003730 tx=0x7fc15000f690 comp rx=0 tx=0).stop 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 shutdown_connections 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc14c0a1ee0 0x7fc14c0a43a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc15c103130 0x7fc15c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c1015a0 0x7fc15c199690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 --2- 172.21.15.7:0/284602172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c102790 0x7fc15c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 >> 172.21.15.7:0/284602172 conn(0x7fc15c0fcb30 msgr2=0x7fc15c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 shutdown_connections 2024-01-11T17:56:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.085+0000 7fc1631ff700 1 -- 172.21.15.7:0/284602172 wait complete. 2024-01-11T17:56:30.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 -- 172.21.15.7:0/1840596074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0101590 msgr2=0x7fa4a01019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/1840596074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0101590 0x7fa4a01019b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa4940099c0 tx=0x7fa494009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 -- 172.21.15.7:0/1840596074 shutdown_connections 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/1840596074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0103120 0x7fa4a01099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/1840596074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0101590 0x7fa4a01019b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/1840596074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0102780 0x7fa4a0102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 -- 172.21.15.7:0/1840596074 >> 172.21.15.7:0/1840596074 conn(0x7fa4a00fcb00 msgr2=0x7fa4a00fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.310+0000 7fa4a6bff700 1 -- 172.21.15.7:0/1840596074 shutdown_connections 2024-01-11T17:56:30.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.311+0000 7fa4a6bff700 1 -- 172.21.15.7:0/1840596074 wait complete. 2024-01-11T17:56:30.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.312+0000 7fa4a6bff700 1 Processor -- start 2024-01-11T17:56:30.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.312+0000 7fa4a6bff700 1 -- start start 2024-01-11T17:56:30.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a6bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 0x7fa4a0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a6bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0102780 0x7fa4a0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a6bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0103120 0x7fa4a01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a6bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a12e0 con 0x7fa4a0103120 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a6bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a1420 con 0x7fa4a0102780 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a6bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a15b0 con 0x7fa4a0101590 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 0x7fa4a0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0103120 0x7fa4a01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 0x7fa4a0199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52994/0 (socket says 172.21.15.7:52994) 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 -- 172.21.15.7:0/2655247505 learned_addr learned my addr 172.21.15.7:0/2655247505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa49ffff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0102780 0x7fa4a0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 -- 172.21.15.7:0/2655247505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0102780 msgr2=0x7fa4a0199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0102780 0x7fa4a0199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 -- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0103120 msgr2=0x7fa4a01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0103120 0x7fa4a01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a499b700 1 -- 172.21.15.7:0/2655247505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4940095f0 con 0x7fa4a0101590 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa49ffff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0102780 0x7fa4a0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.313+0000 7fa4a519c700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0103120 0x7fa4a01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.314+0000 7fa4a499b700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 0x7fa4a0199760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa494003730 tx=0x7fa49400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.314+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49401d070 con 0x7fa4a0101590 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.314+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a01a1830 con 0x7fa4a0101590 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.314+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49400fe70 con 0x7fa4a0101590 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.314+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494004b30 con 0x7fa4a0101590 2024-01-11T17:56:30.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.314+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01a1cb0 con 0x7fa4a0101590 2024-01-11T17:56:30.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.315+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a00683f0 con 0x7fa4a0101590 2024-01-11T17:56:30.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.317+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa494004cb0 con 0x7fa4a0101590 2024-01-11T17:56:30.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.319+0000 7fa49dffb700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4900a1f60 0x7fa4900a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.319+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa4940ceed0 con 0x7fa4a0101590 2024-01-11T17:56:30.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.319+0000 7fa49ffff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4900a1f60 0x7fa4900a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.320+0000 7fa49ffff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4900a1f60 0x7fa4900a4420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009cb0 tx=0x7fa48c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.321+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa494097db0 con 0x7fa4a0101590 2024-01-11T17:56:30.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.540+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4a010cfe0 con 0x7fa4a0101590 2024-01-11T17:56:30.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.542+0000 7fa49dffb700 1 -- 172.21.15.7:0/2655247505 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7fa494026020 con 0x7fa4a0101590 2024-01-11T17:56:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:30.543 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4900a1f60 msgr2=0x7fa4900a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4900a1f60 0x7fa4900a4420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009cb0 tx=0x7fa48c009260 comp rx=0 tx=0).stop 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 msgr2=0x7fa4a0199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 0x7fa4a0199760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa494003730 tx=0x7fa49400fa00 comp rx=0 tx=0).stop 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 shutdown_connections 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4900a1f60 0x7fa4900a4420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a0101590 0x7fa4a0199760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4a0103120 0x7fa4a01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 --2- 172.21.15.7:0/2655247505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a0102780 0x7fa4a0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 >> 172.21.15.7:0/2655247505 conn(0x7fa4a00fcb00 msgr2=0x7fa4a0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.544+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 shutdown_connections 2024-01-11T17:56:30.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.545+0000 7fa4a6bff700 1 -- 172.21.15.7:0/2655247505 wait complete. 2024-01-11T17:56:30.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T17:56:30.624 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:30 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1975919414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T17:56:30.624 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:30 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/284602172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:30 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1975919414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T17:56:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:30 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/284602172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2735322935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998075650 msgr2=0x7fa998073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2735322935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998075650 0x7fa998073aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098f0 tx=0x7fa98c009c00 comp rx=0 tx=0).stop 2024-01-11T17:56:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2735322935 shutdown_connections 2024-01-11T17:56:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2735322935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998109470 0x7fa99810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2735322935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998075650 0x7fa998073aa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2735322935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 0x7fa9980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2735322935 >> 172.21.15.7:0/2735322935 conn(0x7fa9980fcb00 msgr2=0x7fa9980fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2735322935 shutdown_connections 2024-01-11T17:56:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.772+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2735322935 wait complete. 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.773+0000 7fa99f1ff700 1 Processor -- start 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.773+0000 7fa99f1ff700 1 -- start start 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99f1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 0x7fa9981996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 0x7fa998199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99f1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998109470 0x7fa9981a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a1270 con 0x7fa998075650 2024-01-11T17:56:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99f1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a13b0 con 0x7fa998109470 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99f1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a1540 con 0x7fa998074070 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 0x7fa998199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99cf9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 0x7fa9981996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 0x7fa998199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52616/0 (socket says 172.21.15.7:52616) 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa997fff700 1 -- 172.21.15.7:0/2784170429 learned_addr learned my addr 172.21.15.7:0/2784170429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99d79c700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998109470 0x7fa9981a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99cf9b700 1 -- 172.21.15.7:0/2784170429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998109470 msgr2=0x7fa9981a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99cf9b700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998109470 0x7fa9981a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99cf9b700 1 -- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 msgr2=0x7fa998199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99cf9b700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 0x7fa998199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa99cf9b700 1 -- 172.21.15.7:0/2784170429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009520 con 0x7fa998074070 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.774+0000 7fa997fff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 0x7fa998199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.775+0000 7fa99cf9b700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 0x7fa9981996f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009230 tx=0x7fa98c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.775+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c01c070 con 0x7fa998074070 2024-01-11T17:56:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.775+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9981a17c0 con 0x7fa998074070 2024-01-11T17:56:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.775+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa98c00f7d0 con 0x7fa998074070 2024-01-11T17:56:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.775+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0038e0 con 0x7fa998074070 2024-01-11T17:56:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.775+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a1c40 con 0x7fa998074070 2024-01-11T17:56:30.778 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:30 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1975919414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T17:56:30.779 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:30 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/284602172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.776+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998108230 con 0x7fa998074070 2024-01-11T17:56:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.778+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa98c020030 con 0x7fa998074070 2024-01-11T17:56:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.780+0000 7fa995ffb700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9880a1ee0 0x7fa9880a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.780+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa98c0cecf0 con 0x7fa998074070 2024-01-11T17:56:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.780+0000 7fa997fff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9880a1ee0 0x7fa9880a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.780+0000 7fa997fff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9880a1ee0 0x7fa9880a43a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa984009fd0 tx=0x7fa984009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:30.782+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa98c097bd0 con 0x7fa998074070 2024-01-11T17:56:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.044+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa9980683f0 con 0x7fa998074070 2024-01-11T17:56:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.365+0000 7fa995ffb700 1 -- 172.21.15.7:0/2784170429 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa98c09b1f0 con 0x7fa998074070 2024-01-11T17:56:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9880a1ee0 msgr2=0x7fa9880a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9880a1ee0 0x7fa9880a43a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa984009fd0 tx=0x7fa984009310 comp rx=0 tx=0).stop 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 msgr2=0x7fa9981996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 0x7fa9981996f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009230 tx=0x7fa98c00b790 comp rx=0 tx=0).stop 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 shutdown_connections 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9880a1ee0 0x7fa9880a43a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa998074070 0x7fa9981996f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa998109470 0x7fa9981a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 --2- 172.21.15.7:0/2784170429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa998075650 0x7fa998199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 >> 172.21.15.7:0/2784170429 conn(0x7fa9980fcb00 msgr2=0x7fa998105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.368+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 shutdown_connections 2024-01-11T17:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.369+0000 7fa99f1ff700 1 -- 172.21.15.7:0/2784170429 wait complete. 2024-01-11T17:56:31.389 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-01-11T17:56:31.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-01-11T17:56:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.597+0000 7f625bdff700 1 -- 172.21.15.7:0/738235868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 msgr2=0x7f625410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.597+0000 7f625bdff700 1 --2- 172.21.15.7:0/738235868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f625410b900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b0a0 tx=0x7f624c00b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 -- 172.21.15.7:0/738235868 shutdown_connections 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 --2- 172.21.15.7:0/738235868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6254075650 0x7f6254073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 --2- 172.21.15.7:0/738235868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f625410b900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 --2- 172.21.15.7:0/738235868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6254074070 0x7f62540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 -- 172.21.15.7:0/738235868 >> 172.21.15.7:0/738235868 conn(0x7f62540fcb00 msgr2=0x7f62540fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 -- 172.21.15.7:0/738235868 shutdown_connections 2024-01-11T17:56:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.598+0000 7f625bdff700 1 -- 172.21.15.7:0/738235868 wait complete. 2024-01-11T17:56:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 Processor -- start 2024-01-11T17:56:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 -- start start 2024-01-11T17:56:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6254074070 0x7f6254199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6254075650 0x7f6254199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f62541a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62541a1300 con 0x7f6254109470 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62541a1440 con 0x7f6254075650 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.599+0000 7f625bdff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62541a15d0 con 0x7f6254074070 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f62541a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f62541a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52632/0 (socket says 172.21.15.7:52632) 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 -- 172.21.15.7:0/682031169 learned_addr learned my addr 172.21.15.7:0/682031169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625939a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6254075650 0x7f6254199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f6259b9b700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6254074070 0x7f6254199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 -- 172.21.15.7:0/682031169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6254074070 msgr2=0x7f6254199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6254074070 0x7f6254199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 -- 172.21.15.7:0/682031169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6254075650 msgr2=0x7f6254199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6254075650 0x7f6254199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 -- 172.21.15.7:0/682031169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244009520 con 0x7f6254109470 2024-01-11T17:56:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.600+0000 7f625a39c700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f62541a0d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f624c007a60 tx=0x7f624c007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.601+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00e040 con 0x7f6254109470 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.601+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624c004c50 con 0x7f6254109470 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.601+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c009d00 con 0x7f6254109470 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.601+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c01b6c0 con 0x7f6254109470 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.601+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541a1ae0 con 0x7f6254109470 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.602+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254108230 con 0x7f6254109470 2024-01-11T17:56:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.604+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f624c01d470 con 0x7f6254109470 2024-01-11T17:56:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.605+0000 7f624affd700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f62400a1ee0 0x7f62400a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.605+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f624c0ced10 con 0x7f6254109470 2024-01-11T17:56:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.605+0000 7f6259b9b700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f62400a1ee0 0x7f62400a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.606+0000 7f6259b9b700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f62400a1ee0 0x7f62400a43a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6250009c60 tx=0x7f6250009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.609+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f624c0d4020 con 0x7f6254109470 2024-01-11T17:56:31.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:31 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2655247505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:31.760 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:31 smithi016 ceph-mon[43195]: pgmap v259: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:31.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:31 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2784170429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-11T17:56:31.761 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:31 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-11T17:56:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:31 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2655247505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:31 smithi038 ceph-mon[43582]: pgmap v259: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:31 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2784170429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-11T17:56:31.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:31 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-11T17:56:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:31 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2655247505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:31 smithi007 ceph-mon[33361]: pgmap v259: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:31 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2784170429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-11T17:56:31.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:31 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-11T17:56:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:31.872+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f625402d490 con 0x7f6254109470 2024-01-11T17:56:32.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.367+0000 7f624affd700 1 -- 172.21.15.7:0/682031169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f624c097ad0 con 0x7f6254109470 2024-01-11T17:56:32.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f62400a1ee0 msgr2=0x7f62400a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:32.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f62400a1ee0 0x7f62400a43a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6250009c60 tx=0x7f6250009260 comp rx=0 tx=0).stop 2024-01-11T17:56:32.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 msgr2=0x7f62541a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:32.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f62541a0d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f624c007a60 tx=0x7f624c007930 comp rx=0 tx=0).stop 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 shutdown_connections 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f62400a1ee0 0x7f62400a43a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6254074070 0x7f6254199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6254109470 0x7f62541a0d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 --2- 172.21.15.7:0/682031169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6254075650 0x7f6254199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.371+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 >> 172.21.15.7:0/682031169 conn(0x7f62540fcb00 msgr2=0x7f6254105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.372+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 shutdown_connections 2024-01-11T17:56:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.372+0000 7f625bdff700 1 -- 172.21.15.7:0/682031169 wait complete. 2024-01-11T17:56:32.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-01-11T17:56:32.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 -- 172.21.15.7:0/3878253427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1015b0 msgr2=0x7f90cc1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:32.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 --2- 172.21.15.7:0/3878253427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1015b0 0x7f90cc1019d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099c0 tx=0x7f90c8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 -- 172.21.15.7:0/3878253427 shutdown_connections 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 --2- 172.21.15.7:0/3878253427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90cc103140 0x7f90cc1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 --2- 172.21.15.7:0/3878253427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1015b0 0x7f90cc1019d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 --2- 172.21.15.7:0/3878253427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1027a0 0x7f90cc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.607+0000 7f90d43ff700 1 -- 172.21.15.7:0/3878253427 >> 172.21.15.7:0/3878253427 conn(0x7f90cc0fcb00 msgr2=0x7f90cc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.608+0000 7f90d43ff700 1 -- 172.21.15.7:0/3878253427 shutdown_connections 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.608+0000 7f90d43ff700 1 -- 172.21.15.7:0/3878253427 wait complete. 2024-01-11T17:56:32.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.609+0000 7f90d43ff700 1 Processor -- start 2024-01-11T17:56:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.609+0000 7f90d43ff700 1 -- start start 2024-01-11T17:56:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d43ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 0x7f90cc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d43ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 0x7f90cc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d43ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90cc103140 0x7f90cc1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d43ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc1a1200 con 0x7f90cc1027a0 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d43ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc1a1340 con 0x7f90cc1015b0 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d43ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc1a14d0 con 0x7f90cc103140 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 0x7f90cc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d219b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 0x7f90cc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d219b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 0x7f90cc199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37742/0 (socket says 172.21.15.7:37742) 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d219b700 1 -- 172.21.15.7:0/4285326846 learned_addr learned my addr 172.21.15.7:0/4285326846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 0x7f90cc199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44182/0 (socket says 172.21.15.7:44182) 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d299c700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90cc103140 0x7f90cc1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 -- 172.21.15.7:0/4285326846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90cc103140 msgr2=0x7f90cc1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90cc103140 0x7f90cc1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 -- 172.21.15.7:0/4285326846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 msgr2=0x7f90cc199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 0x7f90cc199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d199a700 1 -- 172.21.15.7:0/4285326846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80095f0 con 0x7f90cc1027a0 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.610+0000 7f90d219b700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 0x7f90cc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.611+0000 7f90d199a700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 0x7f90cc199c60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d750 tx=0x7f90bc00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.611+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc004020 con 0x7f90cc1027a0 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.611+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90bc004830 con 0x7f90cc1027a0 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.611+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc1a17b0 con 0x7f90cc1027a0 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.611+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0054c0 con 0x7f90cc1027a0 2024-01-11T17:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.611+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc1a1d10 con 0x7f90cc1027a0 2024-01-11T17:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.612+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc0683f0 con 0x7f90cc1027a0 2024-01-11T17:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.614+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f90bc0049a0 con 0x7f90cc1027a0 2024-01-11T17:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.615+0000 7f90c37fe700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90b80a1f60 0x7f90b80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.615+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90bc0cd5e0 con 0x7f90cc1027a0 2024-01-11T17:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.616+0000 7f90d219b700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90b80a1f60 0x7f90b80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.616+0000 7f90d219b700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90b80a1f60 0x7f90b80a4420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004800 tx=0x7f90c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.619+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f90bc096310 con 0x7f90cc1027a0 2024-01-11T17:56:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:32 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-11T17:56:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:32 smithi016 ceph-mon[43195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:32 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/682031169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-11T17:56:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:32 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-11T17:56:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:32 smithi038 ceph-mon[43582]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:32.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:32 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/682031169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-11T17:56:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:32 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-11T17:56:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:32 smithi007 ceph-mon[33361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:32.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:32 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/682031169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-11T17:56:32.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:32.883+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f90cc10cde0 con 0x7f90cc1027a0 2024-01-11T17:56:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.379+0000 7f90c37fe700 1 -- 172.21.15.7:0/4285326846 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f90bc099930 con 0x7f90cc1027a0 2024-01-11T17:56:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.381+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90b80a1f60 msgr2=0x7f90b80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.381+0000 7f90d43ff700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90b80a1f60 0x7f90b80a4420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004800 tx=0x7f90c801a040 comp rx=0 tx=0).stop 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.381+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 msgr2=0x7f90cc199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.381+0000 7f90d43ff700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 0x7f90cc199c60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d750 tx=0x7f90bc00da60 comp rx=0 tx=0).stop 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 shutdown_connections 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90b80a1f60 0x7f90b80a4420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90cc103140 0x7f90cc1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90cc1015b0 0x7f90cc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 --2- 172.21.15.7:0/4285326846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc1027a0 0x7f90cc199c60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 >> 172.21.15.7:0/4285326846 conn(0x7f90cc0fcb00 msgr2=0x7f90cc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 shutdown_connections 2024-01-11T17:56:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.382+0000 7f90d43ff700 1 -- 172.21.15.7:0/4285326846 wait complete. 2024-01-11T17:56:33.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 -- 172.21.15.7:0/871239853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 msgr2=0x7fdf3c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 --2- 172.21.15.7:0/871239853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c102c00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009920 tx=0x7fdf30009c30 comp rx=0 tx=0).stop 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 -- 172.21.15.7:0/871239853 shutdown_connections 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 --2- 172.21.15.7:0/871239853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf3c103140 0x7fdf3c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 --2- 172.21.15.7:0/871239853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 0x7fdf3c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 --2- 172.21.15.7:0/871239853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c102c00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 -- 172.21.15.7:0/871239853 >> 172.21.15.7:0/871239853 conn(0x7fdf3c0fcb00 msgr2=0x7fdf3c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 -- 172.21.15.7:0/871239853 shutdown_connections 2024-01-11T17:56:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.614+0000 7fdf411ff700 1 -- 172.21.15.7:0/871239853 wait complete. 2024-01-11T17:56:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.615+0000 7fdf411ff700 1 Processor -- start 2024-01-11T17:56:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.615+0000 7fdf411ff700 1 -- start start 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.615+0000 7fdf411ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 0x7fdf3c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.615+0000 7fdf411ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.615+0000 7fdf411ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf3c103140 0x7fdf3c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.615+0000 7fdf411ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1a12a0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf411ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1a13e0 con 0x7fdf3c1015b0 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf411ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1a1570 con 0x7fdf3c103140 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44196/0 (socket says 172.21.15.7:44196) 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 -- 172.21.15.7:0/695801337 learned_addr learned my addr 172.21.15.7:0/695801337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3ad9d700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 0x7fdf3c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3b59e700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf3c103140 0x7fdf3c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 -- 172.21.15.7:0/695801337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf3c103140 msgr2=0x7fdf3c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf3c103140 0x7fdf3c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 -- 172.21.15.7:0/695801337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 msgr2=0x7fdf3c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 0x7fdf3c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 -- 172.21.15.7:0/695801337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0095f0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3ad9d700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 0x7fdf3c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.616+0000 7fdf3a59c700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c199c60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf30005e90 tx=0x7fdf3000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.617+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3001c070 con 0x7fdf3c1027a0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.617+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf3000f7f0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.617+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003cc0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.617+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf34009c40 con 0x7fdf3c1027a0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.617+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf30009520 con 0x7fdf3c1027a0 2024-01-11T17:56:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.618+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c0683f0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.620+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fdf30005590 con 0x7fdf3c1027a0 2024-01-11T17:56:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.622+0000 7fdf23fff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf240a1f60 0x7fdf240a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.623+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdf300ced00 con 0x7fdf3c1027a0 2024-01-11T17:56:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.623+0000 7fdf3ad9d700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf240a1f60 0x7fdf240a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.623+0000 7fdf3ad9d700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf240a1f60 0x7fdf240a4420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c102600 tx=0x7fdf2c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.625+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdf30097ac0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/682031169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: pgmap v260: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:33.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4285326846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-11T17:56:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-11T17:56:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/682031169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-11T17:56:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-11T17:56:33.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:33.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-01-11T17:56:33.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-01-11T17:56:33.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: pgmap v260: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:33.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:33 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4285326846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/682031169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-01-11T17:56:33.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: pgmap v260: 129 pgs: 129 active+clean; 581 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:33.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:33 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4285326846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-11T17:56:33.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:33.888+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fdf3c10cdd0 con 0x7fdf3c1027a0 2024-01-11T17:56:33.996 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:33 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: 2024-01-11T17:56:33.654+0000 7f7f477aa700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-01-11T17:56:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.316+0000 7fdf23fff700 1 -- 172.21.15.7:0/695801337 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fdf3009b0e0 con 0x7fdf3c1027a0 2024-01-11T17:56:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.318+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf240a1f60 msgr2=0x7fdf240a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.318+0000 7fdf411ff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf240a1f60 0x7fdf240a4420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c102600 tx=0x7fdf2c009310 comp rx=0 tx=0).stop 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.318+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 msgr2=0x7fdf3c199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.318+0000 7fdf411ff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c199c60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf30005e90 tx=0x7fdf3000b790 comp rx=0 tx=0).stop 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 shutdown_connections 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf240a1f60 0x7fdf240a4420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf3c103140 0x7fdf3c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3c1015b0 0x7fdf3c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 --2- 172.21.15.7:0/695801337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1027a0 0x7fdf3c199c60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 >> 172.21.15.7:0/695801337 conn(0x7fdf3c0fcb00 msgr2=0x7fdf3c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 shutdown_connections 2024-01-11T17:56:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.319+0000 7fdf411ff700 1 -- 172.21.15.7:0/695801337 wait complete. 2024-01-11T17:56:34.341 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-01-11T17:56:34.341 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 -- 172.21.15.7:0/1791170628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a81015b0 msgr2=0x7fd3a81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/1791170628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a81015b0 0x7fd3a81019d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40099c0 tx=0x7fd3a4009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 -- 172.21.15.7:0/1791170628 shutdown_connections 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/1791170628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a8103140 0x7fd3a81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/1791170628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a81015b0 0x7fd3a81019d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/1791170628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 0x7fd3a8102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 -- 172.21.15.7:0/1791170628 >> 172.21.15.7:0/1791170628 conn(0x7fd3a80fcb00 msgr2=0x7fd3a80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 -- 172.21.15.7:0/1791170628 shutdown_connections 2024-01-11T17:56:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.551+0000 7fd3b07ff700 1 -- 172.21.15.7:0/1791170628 wait complete. 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.552+0000 7fd3b07ff700 1 Processor -- start 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.552+0000 7fd3b07ff700 1 -- start start 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3b07ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a81015b0 0x7fd3a81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3b07ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 0x7fd3a8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3b07ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 0x7fd3a81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3b07ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81a1260 con 0x7fd3a8103140 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3b07ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81a13a0 con 0x7fd3a81027a0 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3b07ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81a1530 con 0x7fd3a81015b0 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 0x7fd3a81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 0x7fd3a81a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44208/0 (socket says 172.21.15.7:44208) 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 -- 172.21.15.7:0/305243703 learned_addr learned my addr 172.21.15.7:0/305243703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3ae59b700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a81015b0 0x7fd3a81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3add9a700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 0x7fd3a8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 -- 172.21.15.7:0/305243703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a81015b0 msgr2=0x7fd3a81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a81015b0 0x7fd3a81996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 -- 172.21.15.7:0/305243703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 msgr2=0x7fd3a8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 0x7fd3a8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3aed9c700 1 -- 172.21.15.7:0/305243703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40095f0 con 0x7fd3a8103140 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.553+0000 7fd3add9a700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 0x7fd3a8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.554+0000 7fd3aed9c700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 0x7fd3a81a0c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000ef30 tx=0x7fd3a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.554+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0018070 con 0x7fd3a8103140 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.554+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a000f040 con 0x7fd3a8103140 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.554+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a81a17b0 con 0x7fd3a8103140 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.554+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000cdb0 con 0x7fd3a8103140 2024-01-11T17:56:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.554+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a81a1c70 con 0x7fd3a8103140 2024-01-11T17:56:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.555+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a80683f0 con 0x7fd3a8103140 2024-01-11T17:56:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.557+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fd3a0003a00 con 0x7fd3a8103140 2024-01-11T17:56:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.558+0000 7fd39f7fe700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd3940a1ee0 0x7fd3940a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.559+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd3a00cd900 con 0x7fd3a8103140 2024-01-11T17:56:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.559+0000 7fd3ae59b700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd3940a1ee0 0x7fd3940a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:34.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.559+0000 7fd3ae59b700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd3940a1ee0 0x7fd3940a43a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009990 tx=0x7fd3a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:34.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.563+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd3a00966c0 con 0x7fd3a8103140 2024-01-11T17:56:34.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4285326846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-11T17:56:34.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:active 2024-01-11T17:56:34.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: Cluster is now healthy 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/695801337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/695801337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:active 2024-01-11T17:56:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:34 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:56:34.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4285326846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-11T17:56:34.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:active 2024-01-11T17:56:34.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: Cluster is now healthy 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/695801337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/695801337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:active 2024-01-11T17:56:34.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:34 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:56:34.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.812+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3a8074f60 con 0x7fd3940a1ee0 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4285326846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:active 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: Cluster is now healthy 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-01-11T17:56:34.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/695801337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-11T17:56:34.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/695801337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-11T17:56:34.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:active 2024-01-11T17:56:34.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:56:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.845+0000 7fd39f7fe700 1 -- 172.21.15.7:0/305243703 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd3a8074f60 con 0x7fd3940a1ee0 2024-01-11T17:56:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd3940a1ee0 msgr2=0x7fd3940a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd3940a1ee0 0x7fd3940a43a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009990 tx=0x7fd3a4000bc0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 msgr2=0x7fd3a81a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 0x7fd3a81a0c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000ef30 tx=0x7fd3a000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 shutdown_connections 2024-01-11T17:56:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd3940a1ee0 0x7fd3940a43a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3a81015b0 0x7fd3a81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3a8103140 0x7fd3a81a0c90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.847+0000 7fd3b07ff700 1 --2- 172.21.15.7:0/305243703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a81027a0 0x7fd3a8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.848+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 >> 172.21.15.7:0/305243703 conn(0x7fd3a80fcb00 msgr2=0x7fd3a8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.848+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 shutdown_connections 2024-01-11T17:56:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:34.848+0000 7fd3b07ff700 1 -- 172.21.15.7:0/305243703 wait complete. 2024-01-11T17:56:34.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-01-11T17:56:35.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:56:34.819+0000 7fdbdb29e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-11T17:56:35.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:56:34.819+0000 7fdbdb29e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-11T17:56:35.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:56:34.819+0000 7fdbdb29e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-11T17:56:35.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:56:34.819+0000 7fdbdb29e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-11T17:56:35.072 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 17:56:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T17:56:34.819+0000 7fdbdb29e700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 -- 172.21.15.7:0/2595351201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401015b0 msgr2=0x7f25401019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 --2- 172.21.15.7:0/2595351201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401015b0 0x7f25401019d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2530009990 tx=0x7f2530009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 -- 172.21.15.7:0/2595351201 shutdown_connections 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 --2- 172.21.15.7:0/2595351201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401015b0 0x7f25401019d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 --2- 172.21.15.7:0/2595351201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 --2- 172.21.15.7:0/2595351201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25401027a0 0x7f2540102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 -- 172.21.15.7:0/2595351201 >> 172.21.15.7:0/2595351201 conn(0x7f25400fcb00 msgr2=0x7f25400fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 -- 172.21.15.7:0/2595351201 shutdown_connections 2024-01-11T17:56:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.075+0000 7f2545dff700 1 -- 172.21.15.7:0/2595351201 wait complete. 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.076+0000 7f2545dff700 1 Processor -- start 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.076+0000 7f2545dff700 1 -- start start 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f2545dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25401015b0 0x7f25401996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f2545dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401027a0 0x7f2540199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f2545dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f2545dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401a1250 con 0x7f2540103140 2024-01-11T17:56:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f2545dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401a1390 con 0x7f25401015b0 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f2545dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401a1520 con 0x7f25401027a0 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401027a0 0x7f2540199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44264/0 (socket says 172.21.15.7:44264) 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25401015b0 0x7f25401996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 -- 172.21.15.7:0/277482587 learned_addr learned my addr 172.21.15.7:0/277482587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401027a0 0x7f2540199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:60814/0 (socket says 172.21.15.7:60814) 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 -- 172.21.15.7:0/277482587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401027a0 msgr2=0x7f2540199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401027a0 0x7f2540199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 -- 172.21.15.7:0/277482587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25401015b0 msgr2=0x7f25401996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25401015b0 0x7f25401996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.077+0000 7f253ffff700 1 -- 172.21.15.7:0/277482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300095f0 con 0x7f2540103140 2024-01-11T17:56:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.078+0000 7f253ffff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401a0c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f253800ef30 tx=0x7f253800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.078+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538018070 con 0x7f2540103140 2024-01-11T17:56:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.078+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f253800f040 con 0x7f2540103140 2024-01-11T17:56:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.078+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800cd00 con 0x7f2540103140 2024-01-11T17:56:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.078+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25401a17a0 con 0x7f2540103140 2024-01-11T17:56:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.078+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25401a1cb0 con 0x7f2540103140 2024-01-11T17:56:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.079+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25400683f0 con 0x7f2540103140 2024-01-11T17:56:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.081+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f2538003a00 con 0x7f2540103140 2024-01-11T17:56:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.082+0000 7f253cff9700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25280a1f60 0x7f25280a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.083+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f25380cd7c0 con 0x7f2540103140 2024-01-11T17:56:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.083+0000 7f253f7fe700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25280a1f60 0x7f25280a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.083+0000 7f253f7fe700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25280a1f60 0x7f25280a4420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f253000b680 tx=0x7f253000b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:35.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.086+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2538096580 con 0x7f2540103140 2024-01-11T17:56:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.329+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2540109690 con 0x7f25280a1f60 2024-01-11T17:56:35.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.355+0000 7f253cff9700 1 -- 172.21.15.7:0/277482587 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2540109690 con 0x7f25280a1f60 2024-01-11T17:56:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.357+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25280a1f60 msgr2=0x7f25280a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.357+0000 7f2545dff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25280a1f60 0x7f25280a4420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f253000b680 tx=0x7f253000b750 comp rx=0 tx=0).stop 2024-01-11T17:56:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.357+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 msgr2=0x7f25401a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.357+0000 7f2545dff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401a0c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f253800ef30 tx=0x7f253800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 shutdown_connections 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f25280a1f60 0x7f25280a4420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2540103140 0x7f25401a0c80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25401015b0 0x7f25401996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 --2- 172.21.15.7:0/277482587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25401027a0 0x7f2540199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 >> 172.21.15.7:0/277482587 conn(0x7f25400fcb00 msgr2=0x7f2540107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 shutdown_connections 2024-01-11T17:56:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.358+0000 7f2545dff700 1 -- 172.21.15.7:0/277482587 wait complete. 2024-01-11T17:56:35.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/1922455872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 msgr2=0x7f1d040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/1922455872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d040744f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009990 tx=0x7f1cf8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/1922455872 shutdown_connections 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/1922455872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 0x7f1d0410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/1922455872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d04075650 0x7f1d04073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/1922455872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d040744f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/1922455872 >> 172.21.15.7:0/1922455872 conn(0x7f1d040fcb00 msgr2=0x7f1d040fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:35.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/1922455872 shutdown_connections 2024-01-11T17:56:35.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.587+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/1922455872 wait complete. 2024-01-11T17:56:35.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.588+0000 7f1d0a3ff700 1 Processor -- start 2024-01-11T17:56:35.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 -- start start 2024-01-11T17:56:35.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d041996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d04075650 0x7f1d04199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 0x7f1d041a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041a1270 con 0x7f1d04109470 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041a13b0 con 0x7f1d04075650 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0a3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041a1540 con 0x7f1d04074070 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d041996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d0899c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 0x7f1d041a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d041996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:60818/0 (socket says 172.21.15.7:60818) 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d03fff700 1 -- 172.21.15.7:0/74435272 learned_addr learned my addr 172.21.15.7:0/74435272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.589+0000 7f1d037fe700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d04075650 0x7f1d04199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0899c700 1 -- 172.21.15.7:0/74435272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 msgr2=0x7f1d041996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0899c700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d041996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0899c700 1 -- 172.21.15.7:0/74435272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d04075650 msgr2=0x7f1d04199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0899c700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d04075650 0x7f1d04199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0899c700 1 -- 172.21.15.7:0/74435272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80095f0 con 0x7f1d04109470 2024-01-11T17:56:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d03fff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d041996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0899c700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 0x7f1d041a0ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00efc0 tx=0x7f1cfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc018070 con 0x7f1d04109470 2024-01-11T17:56:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d041a17c0 con 0x7f1d04109470 2024-01-11T17:56:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cfc00f040 con 0x7f1d04109470 2024-01-11T17:56:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.590+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00cd40 con 0x7f1d04109470 2024-01-11T17:56:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.591+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d041a1c80 con 0x7f1d04109470 2024-01-11T17:56:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.591+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d04108230 con 0x7f1d04109470 2024-01-11T17:56:35.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.593+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1cfc003a00 con 0x7f1d04109470 2024-01-11T17:56:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.595+0000 7f1d017fa700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1cec0a1ee0 0x7f1cec0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.595+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1cfc0cd8d0 con 0x7f1d04109470 2024-01-11T17:56:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.595+0000 7f1d03fff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1cec0a1ee0 0x7f1cec0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.596+0000 7f1d03fff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1cec0a1ee0 0x7f1cec0a43a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4011550 tx=0x7f1cf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:35.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.599+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1cfc096690 con 0x7f1d04109470 2024-01-11T17:56:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:35 smithi016 ceph-mon[43195]: pgmap v261: 129 pgs: 129 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 443 B/s rd, 775 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:35.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:35 smithi038 ceph-mon[43582]: pgmap v261: 129 pgs: 129 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 443 B/s rd, 775 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:35.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:35 smithi007 ceph-mon[33361]: pgmap v261: 129 pgs: 129 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 443 B/s rd, 775 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.852+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d0402d460 con 0x7f1cec0a1ee0 2024-01-11T17:56:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.857+0000 7f1d017fa700 1 -- 172.21.15.7:0/74435272 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f1d0402d460 con 0x7f1cec0a1ee0 2024-01-11T17:56:35.857 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1cec0a1ee0 msgr2=0x7f1cec0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1cec0a1ee0 0x7f1cec0a43a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4011550 tx=0x7f1cf4009260 comp rx=0 tx=0).stop 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 msgr2=0x7f1d041a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 0x7f1d041a0ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00efc0 tx=0x7f1cfc00c3a0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 shutdown_connections 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1cec0a1ee0 0x7f1cec0a43a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d04074070 0x7f1d041996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d04109470 0x7f1d041a0ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 --2- 172.21.15.7:0/74435272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d04075650 0x7f1d04199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 >> 172.21.15.7:0/74435272 conn(0x7f1d040fcb00 msgr2=0x7f1d04105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 shutdown_connections 2024-01-11T17:56:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:35.859+0000 7f1d0a3ff700 1 -- 172.21.15.7:0/74435272 wait complete. 2024-01-11T17:56:35.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 -- 172.21.15.7:0/2241872085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 msgr2=0x7f1698109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 --2- 172.21.15.7:0/2241872085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 0x7f1698109700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1690009f90 tx=0x7f169000b330 comp rx=0 tx=0).stop 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 -- 172.21.15.7:0/2241872085 shutdown_connections 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 --2- 172.21.15.7:0/2241872085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 0x7f1698109700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 --2- 172.21.15.7:0/2241872085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981020b0 0x7f16981044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 --2- 172.21.15.7:0/2241872085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981049e0 0x7f1698106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 -- 172.21.15.7:0/2241872085 >> 172.21.15.7:0/2241872085 conn(0x7f16980fba00 msgr2=0x7f16980fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 -- 172.21.15.7:0/2241872085 shutdown_connections 2024-01-11T17:56:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.087+0000 7f16a05ff700 1 -- 172.21.15.7:0/2241872085 wait complete. 2024-01-11T17:56:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 Processor -- start 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 -- start start 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981020b0 0x7f1698199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 0x7f1698199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 0x7f16981a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16981a12b0 con 0x7f16981049e0 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16981a13f0 con 0x7f16981020b0 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.088+0000 7f16a05ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16981a1580 con 0x7f1698107310 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 0x7f1698199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 0x7f1698199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44292/0 (socket says 172.21.15.7:44292) 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 -- 172.21.15.7:0/1755006674 learned_addr learned my addr 172.21.15.7:0/1755006674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169e39b700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981020b0 0x7f1698199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169eb9c700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 0x7f16981a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 -- 172.21.15.7:0/1755006674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 msgr2=0x7f16981a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 0x7f16981a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 -- 172.21.15.7:0/1755006674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981020b0 msgr2=0x7f1698199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981020b0 0x7f1698199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 -- 172.21.15.7:0/1755006674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009c40 con 0x7f16981049e0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169e39b700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981020b0 0x7f1698199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.089+0000 7f169db9a700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 0x7f1698199c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f168800e9a0 tx=0x7f168800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.090+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688009790 con 0x7f16981049e0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.090+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f168800ce80 con 0x7f16981049e0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.090+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688018680 con 0x7f16981049e0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.090+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16981a1860 con 0x7f16981049e0 2024-01-11T17:56:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.090+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16981a1dc0 con 0x7f16981049e0 2024-01-11T17:56:36.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.091+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1698108e90 con 0x7f16981049e0 2024-01-11T17:56:36.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.093+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1688010430 con 0x7f16981049e0 2024-01-11T17:56:36.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.094+0000 7f168f7fe700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16840a1f60 0x7f16840a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.094+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1688014070 con 0x7f16981049e0 2024-01-11T17:56:36.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.095+0000 7f169e39b700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16840a1f60 0x7f16840a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.095+0000 7f169e39b700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16840a1f60 0x7f16840a4420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1694005fd0 tx=0x7f1694009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.099+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1688052380 con 0x7f16981049e0 2024-01-11T17:56:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.345+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f169802d460 con 0x7f16840a1f60 2024-01-11T17:56:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.370+0000 7f168f7fe700 1 -- 172.21.15.7:0/1755006674 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f169802d460 con 0x7f16840a1f60 2024-01-11T17:56:36.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16840a1f60 msgr2=0x7f16840a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16840a1f60 0x7f16840a4420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1694005fd0 tx=0x7f1694009310 comp rx=0 tx=0).stop 2024-01-11T17:56:36.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 msgr2=0x7f1698199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 0x7f1698199c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f168800e9a0 tx=0x7f168800ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 shutdown_connections 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16840a1f60 0x7f16840a4420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.372+0000 7f16a05ff700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1698107310 0x7f16981a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.373+0000 7f16a05ff700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16981020b0 0x7f1698199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.373+0000 7f16a05ff700 1 --2- 172.21.15.7:0/1755006674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16981049e0 0x7f1698199c70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.373+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 >> 172.21.15.7:0/1755006674 conn(0x7f16980fba00 msgr2=0x7f1698101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.373+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 shutdown_connections 2024-01-11T17:56:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.373+0000 7f16a05ff700 1 -- 172.21.15.7:0/1755006674 wait complete. 2024-01-11T17:56:36.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 -- 172.21.15.7:0/3982500468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 msgr2=0x7f76bc1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 --2- 172.21.15.7:0/3982500468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1019d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76b80099c0 tx=0x7f76b8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 -- 172.21.15.7:0/3982500468 shutdown_connections 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 --2- 172.21.15.7:0/3982500468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc103140 0x7f76bc1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 --2- 172.21.15.7:0/3982500468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1019d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 --2- 172.21.15.7:0/3982500468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76bc1027a0 0x7f76bc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 -- 172.21.15.7:0/3982500468 >> 172.21.15.7:0/3982500468 conn(0x7f76bc0fcb00 msgr2=0x7f76bc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 -- 172.21.15.7:0/3982500468 shutdown_connections 2024-01-11T17:56:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.603+0000 7f76c45ff700 1 -- 172.21.15.7:0/3982500468 wait complete. 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 Processor -- start 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 -- start start 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 0x7f76bc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76bc103140 0x7f76bc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc19f060 con 0x7f76bc1015b0 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc19f1a0 con 0x7f76bc103140 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.604+0000 7f76c45ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc19f330 con 0x7f76bc1027a0 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c1b9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 0x7f76bc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44308/0 (socket says 172.21.15.7:44308) 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c1b9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 0x7f76bc197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:60856/0 (socket says 172.21.15.7:60856) 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 -- 172.21.15.7:0/2523992215 learned_addr learned my addr 172.21.15.7:0/2523992215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c2b9c700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76bc103140 0x7f76bc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 -- 172.21.15.7:0/2523992215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 msgr2=0x7f76bc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 0x7f76bc197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 -- 172.21.15.7:0/2523992215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76bc103140 msgr2=0x7f76bc19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76bc103140 0x7f76bc19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c239b700 1 -- 172.21.15.7:0/2523992215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80095f0 con 0x7f76bc1015b0 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.605+0000 7f76c1b9a700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 0x7f76bc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.606+0000 7f76c239b700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1974e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f76b8003730 tx=0x7f76b800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.606+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b801d070 con 0x7f76bc1015b0 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.606+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76b800bda0 con 0x7f76bc1015b0 2024-01-11T17:56:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.606+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b80173f0 con 0x7f76bc1015b0 2024-01-11T17:56:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.606+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc19f5b0 con 0x7f76bc1015b0 2024-01-11T17:56:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.606+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc19f980 con 0x7f76bc1015b0 2024-01-11T17:56:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.607+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc068430 con 0x7f76bc1015b0 2024-01-11T17:56:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.609+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f76b8022470 con 0x7f76bc1015b0 2024-01-11T17:56:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.611+0000 7f76b37fe700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f76a80a1ee0 0x7f76a80a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.611+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76b80ced70 con 0x7f76bc1015b0 2024-01-11T17:56:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.611+0000 7f76c1b9a700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f76a80a1ee0 0x7f76a80a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.612+0000 7f76c1b9a700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f76a80a1ee0 0x7f76a80a43a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005950 tx=0x7f76ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.615+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f76b8097b30 con 0x7f76bc1015b0 2024-01-11T17:56:36.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:36 smithi016 ceph-mon[43195]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:36.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:36 smithi016 ceph-mon[43195]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:36.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:36 smithi038 ceph-mon[43582]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:36.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:36 smithi038 ceph-mon[43582]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:36.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:36 smithi007 ceph-mon[33361]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:36.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:36 smithi007 ceph-mon[33361]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.866+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f76bc02d630 con 0x7f76a80a1ee0 2024-01-11T17:56:36.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.870+0000 7f76b37fe700 1 -- 172.21.15.7:0/2523992215 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f76bc02d630 con 0x7f76a80a1ee0 2024-01-11T17:56:36.871 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f76a80a1ee0 msgr2=0x7f76a80a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f76a80a1ee0 0x7f76a80a43a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005950 tx=0x7f76ac00b410 comp rx=0 tx=0).stop 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 msgr2=0x7f76bc1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1974e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f76b8003730 tx=0x7f76b800fa00 comp rx=0 tx=0).stop 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 shutdown_connections 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f76a80a1ee0 0x7f76a80a43a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76bc103140 0x7f76bc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76bc1015b0 0x7f76bc1974e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 --2- 172.21.15.7:0/2523992215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc1027a0 0x7f76bc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 >> 172.21.15.7:0/2523992215 conn(0x7f76bc0fcb00 msgr2=0x7f76bc105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 shutdown_connections 2024-01-11T17:56:36.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:36.873+0000 7f76c45ff700 1 -- 172.21.15.7:0/2523992215 wait complete. 2024-01-11T17:56:36.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:56:37.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 -- 172.21.15.7:0/968649235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854102780 msgr2=0x7f4854102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 --2- 172.21.15.7:0/968649235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854102780 0x7f4854102be0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f483c0099c0 tx=0x7f483c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 -- 172.21.15.7:0/968649235 shutdown_connections 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 --2- 172.21.15.7:0/968649235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854103120 0x7f48541099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 --2- 172.21.15.7:0/968649235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4854101590 0x7f48541019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 --2- 172.21.15.7:0/968649235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854102780 0x7f4854102be0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 -- 172.21.15.7:0/968649235 >> 172.21.15.7:0/968649235 conn(0x7f48540fcb00 msgr2=0x7f48540fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 -- 172.21.15.7:0/968649235 shutdown_connections 2024-01-11T17:56:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.099+0000 7f48593ff700 1 -- 172.21.15.7:0/968649235 wait complete. 2024-01-11T17:56:37.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 Processor -- start 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 -- start start 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 0x7f48541996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854102780 0x7f4854199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4854103120 0x7f48541a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48541a11f0 con 0x7f4854101590 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48541a1330 con 0x7f4854103120 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.100+0000 7f48593ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48541a14c0 con 0x7f4854102780 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 0x7f48541996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f484bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854102780 0x7f4854199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 0x7f48541996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44322/0 (socket says 172.21.15.7:44322) 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 -- 172.21.15.7:0/3758298765 learned_addr learned my addr 172.21.15.7:0/3758298765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f48537fe700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4854103120 0x7f48541a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 -- 172.21.15.7:0/3758298765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854102780 msgr2=0x7f4854199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854102780 0x7f4854199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 -- 172.21.15.7:0/3758298765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4854103120 msgr2=0x7f48541a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4854103120 0x7f48541a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 -- 172.21.15.7:0/3758298765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0095f0 con 0x7f4854101590 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f484bfff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854102780 0x7f4854199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.101+0000 7f4852ffd700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 0x7f48541996c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f484400e970 tx=0x7f484400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.102+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844009790 con 0x7f4854101590 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.102+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48541a17a0 con 0x7f4854101590 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.102+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484400c7e0 con 0x7f4854101590 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.102+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844003c70 con 0x7f4854101590 2024-01-11T17:56:37.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.102+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48541a1cb0 con 0x7f4854101590 2024-01-11T17:56:37.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.103+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48540683f0 con 0x7f4854101590 2024-01-11T17:56:37.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.105+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f484400cc50 con 0x7f4854101590 2024-01-11T17:56:37.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.106+0000 7f4850ff9700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48400a1ee0 0x7f48400a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.106+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4844014070 con 0x7f4854101590 2024-01-11T17:56:37.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.106+0000 7f484bfff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48400a1ee0 0x7f48400a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.107+0000 7f484bfff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48400a1ee0 0x7f48400a43a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f483c009fd0 tx=0x7f483c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.110+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f4844096bd0 con 0x7f4854101590 2024-01-11T17:56:37.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.373+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48541a1f30 con 0x7f4854101590 2024-01-11T17:56:37.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.374+0000 7f4850ff9700 1 -- 172.21.15.7:0/3758298765 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11309 (secure 0 0 0) 0x7f4844003dd0 con 0x7f4854101590 2024-01-11T17:56:37.375 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:37.375 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:34.314730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:56:37.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48400a1ee0 msgr2=0x7f48400a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48400a1ee0 0x7f48400a43a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f483c009fd0 tx=0x7f483c00b540 comp rx=0 tx=0).stop 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 msgr2=0x7f48541996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 0x7f48541996c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f484400e970 tx=0x7f484400ec80 comp rx=0 tx=0).stop 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 shutdown_connections 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48400a1ee0 0x7f48400a43a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4854103120 0x7f48541a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4854101590 0x7f48541996c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 --2- 172.21.15.7:0/3758298765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4854102780 0x7f4854199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 >> 172.21.15.7:0/3758298765 conn(0x7f48540fcb00 msgr2=0x7f4854073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 shutdown_connections 2024-01-11T17:56:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.377+0000 7f48593ff700 1 -- 172.21.15.7:0/3758298765 wait complete. 2024-01-11T17:56:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:56:37.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:56:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.604+0000 7fb6035ff700 1 -- 172.21.15.7:0/1672823492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc103120 msgr2=0x7fb5fc1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.604+0000 7fb6035ff700 1 --2- 172.21.15.7:0/1672823492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc103120 0x7fb5fc1099d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b0d0 tx=0x7fb5f400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 -- 172.21.15.7:0/1672823492 shutdown_connections 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 --2- 172.21.15.7:0/1672823492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc101590 0x7fb5fc1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 --2- 172.21.15.7:0/1672823492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc103120 0x7fb5fc1099d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 --2- 172.21.15.7:0/1672823492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc102780 0x7fb5fc102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 -- 172.21.15.7:0/1672823492 >> 172.21.15.7:0/1672823492 conn(0x7fb5fc0fcb00 msgr2=0x7fb5fc0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 -- 172.21.15.7:0/1672823492 shutdown_connections 2024-01-11T17:56:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.605+0000 7fb6035ff700 1 -- 172.21.15.7:0/1672823492 wait complete. 2024-01-11T17:56:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 Processor -- start 2024-01-11T17:56:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 -- start start 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc101590 0x7fb5fc1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 0x7fb5fc199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc103120 0x7fb5fc1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1a1330 con 0x7fb5fc102780 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1a1470 con 0x7fb5fc103120 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb6035ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1a1600 con 0x7fb5fc101590 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb600b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 0x7fb5fc199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb600b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 0x7fb5fc199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44340/0 (socket says 172.21.15.7:44340) 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb600b9a700 1 -- 172.21.15.7:0/2966243198 learned_addr learned my addr 172.21.15.7:0/2966243198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb60139b700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc101590 0x7fb5fc1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.606+0000 7fb601b9c700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc103120 0x7fb5fc1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb601b9c700 1 -- 172.21.15.7:0/2966243198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc101590 msgr2=0x7fb5fc1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb601b9c700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc101590 0x7fb5fc1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb601b9c700 1 -- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 msgr2=0x7fb5fc199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb601b9c700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 0x7fb5fc199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb601b9c700 1 -- 172.21.15.7:0/2966243198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009d00 con 0x7fb5fc103120 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb600b9a700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 0x7fb5fc199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb601b9c700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc103120 0x7fb5fc1a0d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4007b60 tx=0x7fb5f4008ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.607+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400e040 con 0x7fb5fc103120 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.608+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc1a1880 con 0x7fb5fc103120 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.608+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f4005070 con 0x7fb5fc103120 2024-01-11T17:56:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.608+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4006000 con 0x7fb5fc103120 2024-01-11T17:56:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.608+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc1a1d00 con 0x7fb5fc103120 2024-01-11T17:56:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.608+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc0683f0 con 0x7fb5fc103120 2024-01-11T17:56:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.611+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb5f40051e0 con 0x7fb5fc103120 2024-01-11T17:56:37.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.612+0000 7fb5f27fc700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1f70 0x7fb5e80a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:37.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.612+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb5f40cf1f0 con 0x7fb5fc103120 2024-01-11T17:56:37.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.612+0000 7fb60139b700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1f70 0x7fb5e80a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:37.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.613+0000 7fb60139b700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1f70 0x7fb5e80a4430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc1025e0 tx=0x7fb5f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.614+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb5f4097ee0 con 0x7fb5fc103120 2024-01-11T17:56:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:37 smithi016 ceph-mon[43195]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:37 smithi016 ceph-mon[43195]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:37 smithi016 ceph-mon[43195]: pgmap v262: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:37 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3758298765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:37 smithi038 ceph-mon[43582]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:37 smithi038 ceph-mon[43582]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:37 smithi038 ceph-mon[43582]: pgmap v262: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:37 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3758298765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:37.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:37 smithi007 ceph-mon[33361]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:37.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:37 smithi007 ceph-mon[33361]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:37.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:37 smithi007 ceph-mon[33361]: pgmap v262: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:37.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:37 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3758298765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:56:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.827+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5fc10d000 con 0x7fb5fc103120 2024-01-11T17:56:37.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.829+0000 7fb5f27fc700 1 -- 172.21.15.7:0/2966243198 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7fb5f409b500 con 0x7fb5fc103120 2024-01-11T17:56:37.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:37.830 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.831+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1f70 msgr2=0x7fb5e80a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.831+0000 7fb6035ff700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1f70 0x7fb5e80a4430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc1025e0 tx=0x7fb5f8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.831+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc103120 msgr2=0x7fb5fc1a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.831+0000 7fb6035ff700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc103120 0x7fb5fc1a0d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4007b60 tx=0x7fb5f4008ec0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 shutdown_connections 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1f70 0x7fb5e80a4430 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5fc101590 0x7fb5fc1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5fc103120 0x7fb5fc1a0d60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 --2- 172.21.15.7:0/2966243198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5fc102780 0x7fb5fc199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 >> 172.21.15.7:0/2966243198 conn(0x7fb5fc0fcb00 msgr2=0x7fb5fc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 shutdown_connections 2024-01-11T17:56:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:37.832+0000 7fb6035ff700 1 -- 172.21.15.7:0/2966243198 wait complete. 2024-01-11T17:56:37.852 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T17:56:37.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 -- 172.21.15.7:0/3823223042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68103120 msgr2=0x7fed681099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 --2- 172.21.15.7:0/3823223042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68103120 0x7fed681099d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b0a0 tx=0x7fed6000b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 -- 172.21.15.7:0/3823223042 shutdown_connections 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 --2- 172.21.15.7:0/3823223042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68103120 0x7fed681099d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 --2- 172.21.15.7:0/3823223042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed681019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 --2- 172.21.15.7:0/3823223042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102780 0x7fed68102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 -- 172.21.15.7:0/3823223042 >> 172.21.15.7:0/3823223042 conn(0x7fed680fcb00 msgr2=0x7fed680fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 -- 172.21.15.7:0/3823223042 shutdown_connections 2024-01-11T17:56:38.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.056+0000 7fed6ebff700 1 -- 172.21.15.7:0/3823223042 wait complete. 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 Processor -- start 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 -- start start 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed68199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68102780 0x7fed68199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68103120 0x7fed681a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a12b0 con 0x7fed68101590 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a13f0 con 0x7fed68103120 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.057+0000 7fed6ebff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a1580 con 0x7fed68102780 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed68199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed68199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44370/0 (socket says 172.21.15.7:44370) 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 -- 172.21.15.7:0/300229435 learned_addr learned my addr 172.21.15.7:0/300229435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed67fff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68102780 0x7fed68199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6d19c700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68103120 0x7fed681a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 -- 172.21.15.7:0/300229435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68102780 msgr2=0x7fed68199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68102780 0x7fed68199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 -- 172.21.15.7:0/300229435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68103120 msgr2=0x7fed681a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68103120 0x7fed681a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.058+0000 7fed6c99b700 1 -- 172.21.15.7:0/300229435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60009d00 con 0x7fed68101590 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.059+0000 7fed6c99b700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed68199730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00e970 tx=0x7fed5c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.059+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c009790 con 0x7fed68101590 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.059+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c00c7e0 con 0x7fed68101590 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.059+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed681a1860 con 0x7fed68101590 2024-01-11T17:56:38.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.059+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c003c70 con 0x7fed68101590 2024-01-11T17:56:38.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.059+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed681a1d70 con 0x7fed68101590 2024-01-11T17:56:38.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.060+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed680683f0 con 0x7fed68101590 2024-01-11T17:56:38.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.062+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fed5c00cc50 con 0x7fed68101590 2024-01-11T17:56:38.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.064+0000 7fed65ffb700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fed580a1f70 0x7fed580a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.064+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed5c014070 con 0x7fed68101590 2024-01-11T17:56:38.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.065+0000 7fed67fff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fed580a1f70 0x7fed580a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.065+0000 7fed67fff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fed580a1f70 0x7fed580a4430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed54009c60 tx=0x7fed54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:38.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.069+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fed5c096c70 con 0x7fed68101590 2024-01-11T17:56:38.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.278+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed68074f60 con 0x7fed580a1f70 2024-01-11T17:56:38.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.294+0000 7fed65ffb700 1 -- 172.21.15.7:0/300229435 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422523 (secure 0 0 0) 0x7fed68074f60 con 0x7fed580a1f70 2024-01-11T17:56:38.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:38.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fed580a1f70 msgr2=0x7fed580a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fed580a1f70 0x7fed580a4430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed54009c60 tx=0x7fed54009260 comp rx=0 tx=0).stop 2024-01-11T17:56:38.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 msgr2=0x7fed68199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed68199730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00e970 tx=0x7fed5c00ec80 comp rx=0 tx=0).stop 2024-01-11T17:56:38.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 shutdown_connections 2024-01-11T17:56:38.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fed580a1f70 0x7fed580a4430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68103120 0x7fed681a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed68101590 0x7fed68199730 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 --2- 172.21.15.7:0/300229435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed68102780 0x7fed68199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 >> 172.21.15.7:0/300229435 conn(0x7fed680fcb00 msgr2=0x7fed68073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 shutdown_connections 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.300+0000 7fed6ebff700 1 -- 172.21.15.7:0/300229435 wait complete. 2024-01-11T17:56:38.307 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:56:38.320 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-01-11T17:56:36.572331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598097,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":148,"num_read_kb":130,"num_write":302,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385396,"kb_used_data":11284,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556428,"statfs":{"total":1151940427776,"available":1151545782272,"internally_reserved":0,"allocated":11554816,"data_stored":7425470,"data_compressed":320463,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3231,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":74,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.241040"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338271+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:56:25.338271+0000","last_peered":"2024-01-11T17:56:25.338271+0000","last_clean":"2024-01-11T17:56:25.338271+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:56:25.338271+0000","last_undegraded":"2024-01-11T17:56:25.338271+0000","last_fullsized":"2024-01-11T17:56:25.338271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336434+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:56:25.336434+0000","last_peered":"2024-01-11T17:56:25.336434+0000","last_clean":"2024-01-11T17:56:25.336434+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:56:25.336434+0000","last_undegraded":"2024-01-11T17:56:25.336434+0000","last_fullsized":"2024-01-11T17:56:25.336434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336251+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:56:25.336251+0000","last_peered":"2024-01-11T17:56:25.336251+0000","last_clean":"2024-01-11T17:56:25.336251+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:56:25.336251+0000","last_undegraded":"2024-01-11T17:56:25.336251+0000","last_fullsized":"2024-01-11T17:56:25.336251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332674+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:56:25.332674+0000","last_peered":"2024-01-11T17:56:25.332674+0000","last_clean":"2024-01-11T17:56:25.332674+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:56:25.332674+0000","last_undegraded":"2024-01-11T17:56:25.332674+0000","last_fullsized":"2024-01-11T17:56:25.332674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355950+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:56:26.355950+0000","last_peered":"2024-01-11T17:56:26.355950+0000","last_clean":"2024-01-11T17:56:26.355950+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:56:26.355950+0000","last_undegraded":"2024-01-11T17:56:26.355950+0000","last_fullsized":"2024-01-11T17:56:26.355950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355667+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:56:26.355667+0000","last_peered":"2024-01-11T17:56:26.355667+0000","last_clean":"2024-01-11T17:56:26.355667+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:56:26.355667+0000","last_undegraded":"2024-01-11T17:56:26.355667+0000","last_fullsized":"2024-01-11T17:56:26.355667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341507+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:56:25.341507+0000","last_peered":"2024-01-11T17:56:25.341507+0000","last_clean":"2024-01-11T17:56:25.341507+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:56:25.341507+0000","last_undegraded":"2024-01-11T17:56:25.341507+0000","last_fullsized":"2024-01-11T17:56:25.341507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338751+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:56:25.338751+0000","last_peered":"2024-01-11T17:56:25.338751+0000","last_clean":"2024-01-11T17:56:25.338751+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:56:25.338751+0000","last_undegraded":"2024-01-11T17:56:25.338751+0000","last_fullsized":"2024-01-11T17:56:25.338751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338413+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:56:25.338413+0000","last_peered":"2024-01-11T17:56:25.338413+0000","last_clean":"2024-01-11T17:56:25.338413+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:56:25.338413+0000","last_undegraded":"2024-01-11T17:56:25.338413+0000","last_fullsized":"2024-01-11T17:56:25.338413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335441+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:56:25.335441+0000","last_peered":"2024-01-11T17:56:25.335441+0000","last_clean":"2024-01-11T17:56:25.335441+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:56:25.335441+0000","last_undegraded":"2024-01-11T17:56:25.335441+0000","last_fullsized":"2024-01-11T17:56:25.335441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341624+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:56:25.341624+0000","last_peered":"2024-01-11T17:56:25.341624+0000","last_clean":"2024-01-11T17:56:25.341624+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:56:25.341624+0000","last_undegraded":"2024-01-11T17:56:25.341624+0000","last_fullsized":"2024-01-11T17:56:25.341624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332662+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:56:25.332662+0000","last_peered":"2024-01-11T17:56:25.332662+0000","last_clean":"2024-01-11T17:56:25.332662+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:56:25.332662+0000","last_undegraded":"2024-01-11T17:56:25.332662+0000","last_fullsized":"2024-01-11T17:56:25.332662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332139+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:56:25.332139+0000","last_peered":"2024-01-11T17:56:25.332139+0000","last_clean":"2024-01-11T17:56:25.332139+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:56:25.332139+0000","last_undegraded":"2024-01-11T17:56:25.332139+0000","last_fullsized":"2024-01-11T17:56:25.332139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335487+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:56:25.335487+0000","last_peered":"2024-01-11T17:56:25.335487+0000","last_clean":"2024-01-11T17:56:25.335487+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:56:25.335487+0000","last_undegraded":"2024-01-11T17:56:25.335487+0000","last_fullsized":"2024-01-11T17:56:25.335487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355728+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:56:26.355728+0000","last_peered":"2024-01-11T17:56:26.355728+0000","last_clean":"2024-01-11T17:56:26.355728+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:56:26.355728+0000","last_undegraded":"2024-01-11T17:56:26.355728+0000","last_fullsized":"2024-01-11T17:56:26.355728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337407+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:56:25.337407+0000","last_peered":"2024-01-11T17:56:25.337407+0000","last_clean":"2024-01-11T17:56:25.337407+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:56:25.337407+0000","last_undegraded":"2024-01-11T17:56:25.337407+0000","last_fullsized":"2024-01-11T17:56:25.337407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337914+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:56:25.337914+0000","last_peered":"2024-01-11T17:56:25.337914+0000","last_clean":"2024-01-11T17:56:25.337914+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:56:25.337914+0000","last_undegraded":"2024-01-11T17:56:25.337914+0000","last_fullsized":"2024-01-11T17:56:25.337914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336121+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:56:25.336121+0000","last_peered":"2024-01-11T17:56:25.336121+0000","last_clean":"2024-01-11T17:56:25.336121+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:56:25.336121+0000","last_undegraded":"2024-01-11T17:56:25.336121+0000","last_fullsized":"2024-01-11T17:56:25.336121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338989+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:56:25.338989+0000","last_peered":"2024-01-11T17:56:25.338989+0000","last_clean":"2024-01-11T17:56:25.338989+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:56:25.338989+0000","last_undegraded":"2024-01-11T17:56:25.338989+0000","last_fullsized":"2024-01-11T17:56:25.338989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340885+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:56:25.340885+0000","last_peered":"2024-01-11T17:56:25.340885+0000","last_clean":"2024-01-11T17:56:25.340885+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:56:25.340885+0000","last_undegraded":"2024-01-11T17:56:25.340885+0000","last_fullsized":"2024-01-11T17:56:25.340885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397873+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:56:33.397873+0000","last_peered":"2024-01-11T17:56:33.397873+0000","last_clean":"2024-01-11T17:56:33.397873+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:56:33.397873+0000","last_undegraded":"2024-01-11T17:56:33.397873+0000","last_fullsized":"2024-01-11T17:56:33.397873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339633+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:56:25.339633+0000","last_peered":"2024-01-11T17:56:25.339633+0000","last_clean":"2024-01-11T17:56:25.339633+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:56:25.339633+0000","last_undegraded":"2024-01-11T17:56:25.339633+0000","last_fullsized":"2024-01-11T17:56:25.339633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331322+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:56:25.331322+0000","last_peered":"2024-01-11T17:56:25.331322+0000","last_clean":"2024-01-11T17:56:25.331322+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:56:25.331322+0000","last_undegraded":"2024-01-11T17:56:25.331322+0000","last_fullsized":"2024-01-11T17:56:25.331322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.333494+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:56:26.333494+0000","last_peered":"2024-01-11T17:56:26.333494+0000","last_clean":"2024-01-11T17:56:26.333494+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:56:26.333494+0000","last_undegraded":"2024-01-11T17:56:26.333494+0000","last_fullsized":"2024-01-11T17:56:26.333494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336511+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:56:25.336511+0000","last_peered":"2024-01-11T17:56:25.336511+0000","last_clean":"2024-01-11T17:56:25.336511+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:56:25.336511+0000","last_undegraded":"2024-01-11T17:56:25.336511+0000","last_fullsized":"2024-01-11T17:56:25.336511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331455+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:56:25.331455+0000","last_peered":"2024-01-11T17:56:25.331455+0000","last_clean":"2024-01-11T17:56:25.331455+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:56:25.331455+0000","last_undegraded":"2024-01-11T17:56:25.331455+0000","last_fullsized":"2024-01-11T17:56:25.331455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336368+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:56:25.336368+0000","last_peered":"2024-01-11T17:56:25.336368+0000","last_clean":"2024-01-11T17:56:25.336368+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:56:25.336368+0000","last_undegraded":"2024-01-11T17:56:25.336368+0000","last_fullsized":"2024-01-11T17:56:25.336368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337771+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:56:25.337771+0000","last_peered":"2024-01-11T17:56:25.337771+0000","last_clean":"2024-01-11T17:56:25.337771+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:56:25.337771+0000","last_undegraded":"2024-01-11T17:56:25.337771+0000","last_fullsized":"2024-01-11T17:56:25.337771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331648+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:56:25.331648+0000","last_peered":"2024-01-11T17:56:25.331648+0000","last_clean":"2024-01-11T17:56:25.331648+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:56:25.331648+0000","last_undegraded":"2024-01-11T17:56:25.331648+0000","last_fullsized":"2024-01-11T17:56:25.331648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.351148+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:56:26.351148+0000","last_peered":"2024-01-11T17:56:26.351148+0000","last_clean":"2024-01-11T17:56:26.351148+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:56:26.351148+0000","last_undegraded":"2024-01-11T17:56:26.351148+0000","last_fullsized":"2024-01-11T17:56:26.351148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335605+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:56:25.335605+0000","last_peered":"2024-01-11T17:56:25.335605+0000","last_clean":"2024-01-11T17:56:25.335605+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:56:25.335605+0000","last_undegraded":"2024-01-11T17:56:25.335605+0000","last_fullsized":"2024-01-11T17:56:25.335605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338781+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:56:25.338781+0000","last_peered":"2024-01-11T17:56:25.338781+0000","last_clean":"2024-01-11T17:56:25.338781+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:56:25.338781+0000","last_undegraded":"2024-01-11T17:56:25.338781+0000","last_fullsized":"2024-01-11T17:56:25.338781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.326764+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:56:34.326764+0000","last_peered":"2024-01-11T17:56:34.326764+0000","last_clean":"2024-01-11T17:56:34.326764+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:56:34.326764+0000","last_undegraded":"2024-01-11T17:56:34.326764+0000","last_fullsized":"2024-01-11T17:56:34.326764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335788+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:56:25.335788+0000","last_peered":"2024-01-11T17:56:25.335788+0000","last_clean":"2024-01-11T17:56:25.335788+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:56:25.335788+0000","last_undegraded":"2024-01-11T17:56:25.335788+0000","last_fullsized":"2024-01-11T17:56:25.335788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337755+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:56:25.337755+0000","last_peered":"2024-01-11T17:56:25.337755+0000","last_clean":"2024-01-11T17:56:25.337755+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:56:25.337755+0000","last_undegraded":"2024-01-11T17:56:25.337755+0000","last_fullsized":"2024-01-11T17:56:25.337755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395256+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:56:33.395256+0000","last_peered":"2024-01-11T17:56:33.395256+0000","last_clean":"2024-01-11T17:56:33.395256+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:56:33.395256+0000","last_undegraded":"2024-01-11T17:56:33.395256+0000","last_fullsized":"2024-01-11T17:56:33.395256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337679+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:56:25.337679+0000","last_peered":"2024-01-11T17:56:25.337679+0000","last_clean":"2024-01-11T17:56:25.337679+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:56:25.337679+0000","last_undegraded":"2024-01-11T17:56:25.337679+0000","last_fullsized":"2024-01-11T17:56:25.337679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337811+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:56:25.337811+0000","last_peered":"2024-01-11T17:56:25.337811+0000","last_clean":"2024-01-11T17:56:25.337811+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:56:25.337811+0000","last_undegraded":"2024-01-11T17:56:25.337811+0000","last_fullsized":"2024-01-11T17:56:25.337811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397732+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:56:33.397732+0000","last_peered":"2024-01-11T17:56:33.397732+0000","last_clean":"2024-01-11T17:56:33.397732+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:56:33.397732+0000","last_undegraded":"2024-01-11T17:56:33.397732+0000","last_fullsized":"2024-01-11T17:56:33.397732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338662+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:56:25.338662+0000","last_peered":"2024-01-11T17:56:25.338662+0000","last_clean":"2024-01-11T17:56:25.338662+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:56:25.338662+0000","last_undegraded":"2024-01-11T17:56:25.338662+0000","last_fullsized":"2024-01-11T17:56:25.338662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336435+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:56:25.336435+0000","last_peered":"2024-01-11T17:56:25.336435+0000","last_clean":"2024-01-11T17:56:25.336435+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:56:25.336435+0000","last_undegraded":"2024-01-11T17:56:25.336435+0000","last_fullsized":"2024-01-11T17:56:25.336435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.344358+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:56:26.344358+0000","last_peered":"2024-01-11T17:56:26.344358+0000","last_clean":"2024-01-11T17:56:26.344358+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:56:26.344358+0000","last_undegraded":"2024-01-11T17:56:26.344358+0000","last_fullsized":"2024-01-11T17:56:26.344358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335428+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:56:25.335428+0000","last_peered":"2024-01-11T17:56:25.335428+0000","last_clean":"2024-01-11T17:56:25.335428+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:56:25.335428+0000","last_undegraded":"2024-01-11T17:56:25.335428+0000","last_fullsized":"2024-01-11T17:56:25.335428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338974+0000","last_change":"2024-01-11T17:56:12.824145+0000","last_active":"2024-01-11T17:56:25.338974+0000","last_peered":"2024-01-11T17:56:25.338974+0000","last_clean":"2024-01-11T17:56:25.338974+0000","last_became_active":"2024-01-11T17:56:12.823443+0000","last_became_peered":"2024-01-11T17:56:12.823443+0000","last_unstale":"2024-01-11T17:56:25.338974+0000","last_undegraded":"2024-01-11T17:56:25.338974+0000","last_fullsized":"2024-01-11T17:56:25.338974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:34:19.317304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.390278+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:56:32.390278+0000","last_peered":"2024-01-11T17:56:32.390278+0000","last_clean":"2024-01-11T17:56:32.390278+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:56:32.390278+0000","last_undegraded":"2024-01-11T17:56:32.390278+0000","last_fullsized":"2024-01-11T17:56:32.390278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338217+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:56:25.338217+0000","last_peered":"2024-01-11T17:56:25.338217+0000","last_clean":"2024-01-11T17:56:25.338217+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:56:25.338217+0000","last_undegraded":"2024-01-11T17:56:25.338217+0000","last_fullsized":"2024-01-11T17:56:25.338217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334836+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:56:25.334836+0000","last_peered":"2024-01-11T17:56:25.334836+0000","last_clean":"2024-01-11T17:56:25.334836+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:56:25.334836+0000","last_undegraded":"2024-01-11T17:56:25.334836+0000","last_fullsized":"2024-01-11T17:56:25.334836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388337+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:56:32.388337+0000","last_peered":"2024-01-11T17:56:32.388337+0000","last_clean":"2024-01-11T17:56:32.388337+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:56:32.388337+0000","last_undegraded":"2024-01-11T17:56:32.388337+0000","last_fullsized":"2024-01-11T17:56:32.388337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336605+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:56:25.336605+0000","last_peered":"2024-01-11T17:56:25.336605+0000","last_clean":"2024-01-11T17:56:25.336605+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:56:25.336605+0000","last_undegraded":"2024-01-11T17:56:25.336605+0000","last_fullsized":"2024-01-11T17:56:25.336605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331894+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:56:25.331894+0000","last_peered":"2024-01-11T17:56:25.331894+0000","last_clean":"2024-01-11T17:56:25.331894+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:56:25.331894+0000","last_undegraded":"2024-01-11T17:56:25.331894+0000","last_fullsized":"2024-01-11T17:56:25.331894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391761+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:56:32.391761+0000","last_peered":"2024-01-11T17:56:32.391761+0000","last_clean":"2024-01-11T17:56:32.391761+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:56:32.391761+0000","last_undegraded":"2024-01-11T17:56:32.391761+0000","last_fullsized":"2024-01-11T17:56:32.391761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337845+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:56:25.337845+0000","last_peered":"2024-01-11T17:56:25.337845+0000","last_clean":"2024-01-11T17:56:25.337845+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:56:25.337845+0000","last_undegraded":"2024-01-11T17:56:25.337845+0000","last_fullsized":"2024-01-11T17:56:25.337845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336863+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:56:25.336863+0000","last_peered":"2024-01-11T17:56:25.336863+0000","last_clean":"2024-01-11T17:56:25.336863+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:56:25.336863+0000","last_undegraded":"2024-01-11T17:56:25.336863+0000","last_fullsized":"2024-01-11T17:56:25.336863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.327572+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:56:34.327572+0000","last_peered":"2024-01-11T17:56:34.327572+0000","last_clean":"2024-01-11T17:56:34.327572+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:56:34.327572+0000","last_undegraded":"2024-01-11T17:56:34.327572+0000","last_fullsized":"2024-01-11T17:56:34.327572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336092+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:56:25.336092+0000","last_peered":"2024-01-11T17:56:25.336092+0000","last_clean":"2024-01-11T17:56:25.336092+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:56:25.336092+0000","last_undegraded":"2024-01-11T17:56:25.336092+0000","last_fullsized":"2024-01-11T17:56:25.336092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395453+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:56:33.395453+0000","last_peered":"2024-01-11T17:56:33.395453+0000","last_clean":"2024-01-11T17:56:33.395453+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:56:33.395453+0000","last_undegraded":"2024-01-11T17:56:33.395453+0000","last_fullsized":"2024-01-11T17:56:33.395453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331833+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:56:25.331833+0000","last_peered":"2024-01-11T17:56:25.331833+0000","last_clean":"2024-01-11T17:56:25.331833+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:56:25.331833+0000","last_undegraded":"2024-01-11T17:56:25.331833+0000","last_fullsized":"2024-01-11T17:56:25.331833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337590+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:56:25.337590+0000","last_peered":"2024-01-11T17:56:25.337590+0000","last_clean":"2024-01-11T17:56:25.337590+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:56:25.337590+0000","last_undegraded":"2024-01-11T17:56:25.337590+0000","last_fullsized":"2024-01-11T17:56:25.337590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391050+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:56:32.391050+0000","last_peered":"2024-01-11T17:56:32.391050+0000","last_clean":"2024-01-11T17:56:32.391050+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:56:32.391050+0000","last_undegraded":"2024-01-11T17:56:32.391050+0000","last_fullsized":"2024-01-11T17:56:32.391050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341538+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:56:25.341538+0000","last_peered":"2024-01-11T17:56:25.341538+0000","last_clean":"2024-01-11T17:56:25.341538+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:56:25.341538+0000","last_undegraded":"2024-01-11T17:56:25.341538+0000","last_fullsized":"2024-01-11T17:56:25.341538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.343158+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:56:26.343158+0000","last_peered":"2024-01-11T17:56:26.343158+0000","last_clean":"2024-01-11T17:56:26.343158+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:56:26.343158+0000","last_undegraded":"2024-01-11T17:56:26.343158+0000","last_fullsized":"2024-01-11T17:56:26.343158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:35.331005+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:56:35.331005+0000","last_peered":"2024-01-11T17:56:35.331005+0000","last_clean":"2024-01-11T17:56:35.331005+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:56:35.331005+0000","last_undegraded":"2024-01-11T17:56:35.331005+0000","last_fullsized":"2024-01-11T17:56:35.331005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341549+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:56:25.341549+0000","last_peered":"2024-01-11T17:56:25.341549+0000","last_clean":"2024-01-11T17:56:25.341549+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:56:25.341549+0000","last_undegraded":"2024-01-11T17:56:25.341549+0000","last_fullsized":"2024-01-11T17:56:25.341549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331924+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:56:25.331924+0000","last_peered":"2024-01-11T17:56:25.331924+0000","last_clean":"2024-01-11T17:56:25.331924+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:56:25.331924+0000","last_undegraded":"2024-01-11T17:56:25.331924+0000","last_fullsized":"2024-01-11T17:56:25.331924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388797+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:56:32.388797+0000","last_peered":"2024-01-11T17:56:32.388797+0000","last_clean":"2024-01-11T17:56:32.388797+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:56:32.388797+0000","last_undegraded":"2024-01-11T17:56:32.388797+0000","last_fullsized":"2024-01-11T17:56:32.388797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336728+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:56:25.336728+0000","last_peered":"2024-01-11T17:56:25.336728+0000","last_clean":"2024-01-11T17:56:25.336728+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:56:25.336728+0000","last_undegraded":"2024-01-11T17:56:25.336728+0000","last_fullsized":"2024-01-11T17:56:25.336728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336598+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:56:25.336598+0000","last_peered":"2024-01-11T17:56:25.336598+0000","last_clean":"2024-01-11T17:56:25.336598+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:56:25.336598+0000","last_undegraded":"2024-01-11T17:56:25.336598+0000","last_fullsized":"2024-01-11T17:56:25.336598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334864+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:56:25.334864+0000","last_peered":"2024-01-11T17:56:25.334864+0000","last_clean":"2024-01-11T17:56:25.334864+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:56:25.334864+0000","last_undegraded":"2024-01-11T17:56:25.334864+0000","last_fullsized":"2024-01-11T17:56:25.334864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335418+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:56:25.335418+0000","last_peered":"2024-01-11T17:56:25.335418+0000","last_clean":"2024-01-11T17:56:25.335418+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:56:25.335418+0000","last_undegraded":"2024-01-11T17:56:25.335418+0000","last_fullsized":"2024-01-11T17:56:25.335418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335675+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:56:25.335675+0000","last_peered":"2024-01-11T17:56:25.335675+0000","last_clean":"2024-01-11T17:56:25.335675+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:56:25.335675+0000","last_undegraded":"2024-01-11T17:56:25.335675+0000","last_fullsized":"2024-01-11T17:56:25.335675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337326+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:56:25.337326+0000","last_peered":"2024-01-11T17:56:25.337326+0000","last_clean":"2024-01-11T17:56:25.337326+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:56:25.337326+0000","last_undegraded":"2024-01-11T17:56:25.337326+0000","last_fullsized":"2024-01-11T17:56:25.337326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335306+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:56:25.335306+0000","last_peered":"2024-01-11T17:56:25.335306+0000","last_clean":"2024-01-11T17:56:25.335306+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:56:25.335306+0000","last_undegraded":"2024-01-11T17:56:25.335306+0000","last_fullsized":"2024-01-11T17:56:25.335306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331851+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:56:25.331851+0000","last_peered":"2024-01-11T17:56:25.331851+0000","last_clean":"2024-01-11T17:56:25.331851+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:56:25.331851+0000","last_undegraded":"2024-01-11T17:56:25.331851+0000","last_fullsized":"2024-01-11T17:56:25.331851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338402+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:56:26.338402+0000","last_peered":"2024-01-11T17:56:26.338402+0000","last_clean":"2024-01-11T17:56:26.338402+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:56:26.338402+0000","last_undegraded":"2024-01-11T17:56:26.338402+0000","last_fullsized":"2024-01-11T17:56:26.338402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335017+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:56:25.335017+0000","last_peered":"2024-01-11T17:56:25.335017+0000","last_clean":"2024-01-11T17:56:25.335017+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:56:25.335017+0000","last_undegraded":"2024-01-11T17:56:25.335017+0000","last_fullsized":"2024-01-11T17:56:25.335017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331773+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:56:25.331773+0000","last_peered":"2024-01-11T17:56:25.331773+0000","last_clean":"2024-01-11T17:56:25.331773+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:56:25.331773+0000","last_undegraded":"2024-01-11T17:56:25.331773+0000","last_fullsized":"2024-01-11T17:56:25.331773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338194+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:56:25.338194+0000","last_peered":"2024-01-11T17:56:25.338194+0000","last_clean":"2024-01-11T17:56:25.338194+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:56:25.338194+0000","last_undegraded":"2024-01-11T17:56:25.338194+0000","last_fullsized":"2024-01-11T17:56:25.338194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.326548+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:56:34.326548+0000","last_peered":"2024-01-11T17:56:34.326548+0000","last_clean":"2024-01-11T17:56:34.326548+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:56:34.326548+0000","last_undegraded":"2024-01-11T17:56:34.326548+0000","last_fullsized":"2024-01-11T17:56:34.326548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341442+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:56:25.341442+0000","last_peered":"2024-01-11T17:56:25.341442+0000","last_clean":"2024-01-11T17:56:25.341442+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:56:25.341442+0000","last_undegraded":"2024-01-11T17:56:25.341442+0000","last_fullsized":"2024-01-11T17:56:25.341442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338909+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:56:25.338909+0000","last_peered":"2024-01-11T17:56:25.338909+0000","last_clean":"2024-01-11T17:56:25.338909+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:56:25.338909+0000","last_undegraded":"2024-01-11T17:56:25.338909+0000","last_fullsized":"2024-01-11T17:56:25.338909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324345+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:56:34.324345+0000","last_peered":"2024-01-11T17:56:34.324345+0000","last_clean":"2024-01-11T17:56:34.324345+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:56:34.324345+0000","last_undegraded":"2024-01-11T17:56:34.324345+0000","last_fullsized":"2024-01-11T17:56:34.324345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341318+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:56:25.341318+0000","last_peered":"2024-01-11T17:56:25.341318+0000","last_clean":"2024-01-11T17:56:25.341318+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:56:25.341318+0000","last_undegraded":"2024-01-11T17:56:25.341318+0000","last_fullsized":"2024-01-11T17:56:25.341318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331435+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:56:25.331435+0000","last_peered":"2024-01-11T17:56:25.331435+0000","last_clean":"2024-01-11T17:56:25.331435+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:56:25.331435+0000","last_undegraded":"2024-01-11T17:56:25.331435+0000","last_fullsized":"2024-01-11T17:56:25.331435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.394917+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:56:33.394917+0000","last_peered":"2024-01-11T17:56:33.394917+0000","last_clean":"2024-01-11T17:56:33.394917+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:56:33.394917+0000","last_undegraded":"2024-01-11T17:56:33.394917+0000","last_fullsized":"2024-01-11T17:56:33.394917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335272+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:56:25.335272+0000","last_peered":"2024-01-11T17:56:25.335272+0000","last_clean":"2024-01-11T17:56:25.335272+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:56:25.335272+0000","last_undegraded":"2024-01-11T17:56:25.335272+0000","last_fullsized":"2024-01-11T17:56:25.335272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338904+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:56:26.338904+0000","last_peered":"2024-01-11T17:56:26.338904+0000","last_clean":"2024-01-11T17:56:26.338904+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:56:26.338904+0000","last_undegraded":"2024-01-11T17:56:26.338904+0000","last_fullsized":"2024-01-11T17:56:26.338904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332040+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:56:25.332040+0000","last_peered":"2024-01-11T17:56:25.332040+0000","last_clean":"2024-01-11T17:56:25.332040+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:56:25.332040+0000","last_undegraded":"2024-01-11T17:56:25.332040+0000","last_fullsized":"2024-01-11T17:56:25.332040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340003+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:56:25.340003+0000","last_peered":"2024-01-11T17:56:25.340003+0000","last_clean":"2024-01-11T17:56:25.340003+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:56:25.340003+0000","last_undegraded":"2024-01-11T17:56:25.340003+0000","last_fullsized":"2024-01-11T17:56:25.340003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.396237+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:56:33.396237+0000","last_peered":"2024-01-11T17:56:33.396237+0000","last_clean":"2024-01-11T17:56:33.396237+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:56:33.396237+0000","last_undegraded":"2024-01-11T17:56:33.396237+0000","last_fullsized":"2024-01-11T17:56:33.396237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339772+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:56:25.339772+0000","last_peered":"2024-01-11T17:56:25.339772+0000","last_clean":"2024-01-11T17:56:25.339772+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:56:25.339772+0000","last_undegraded":"2024-01-11T17:56:25.339772+0000","last_fullsized":"2024-01-11T17:56:25.339772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340165+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:56:25.340165+0000","last_peered":"2024-01-11T17:56:25.340165+0000","last_clean":"2024-01-11T17:56:25.340165+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:56:25.340165+0000","last_undegraded":"2024-01-11T17:56:25.340165+0000","last_fullsized":"2024-01-11T17:56:25.340165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.386925+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:56:32.386925+0000","last_peered":"2024-01-11T17:56:32.386925+0000","last_clean":"2024-01-11T17:56:32.386925+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:56:32.386925+0000","last_undegraded":"2024-01-11T17:56:32.386925+0000","last_fullsized":"2024-01-11T17:56:32.386925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336540+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:56:25.336540+0000","last_peered":"2024-01-11T17:56:25.336540+0000","last_clean":"2024-01-11T17:56:25.336540+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:56:25.336540+0000","last_undegraded":"2024-01-11T17:56:25.336540+0000","last_fullsized":"2024-01-11T17:56:25.336540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.346059+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:56:26.346059+0000","last_peered":"2024-01-11T17:56:26.346059+0000","last_clean":"2024-01-11T17:56:26.346059+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:56:26.346059+0000","last_undegraded":"2024-01-11T17:56:26.346059+0000","last_fullsized":"2024-01-11T17:56:26.346059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388647+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:56:32.388647+0000","last_peered":"2024-01-11T17:56:32.388647+0000","last_clean":"2024-01-11T17:56:32.388647+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:56:32.388647+0000","last_undegraded":"2024-01-11T17:56:32.388647+0000","last_fullsized":"2024-01-11T17:56:32.388647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334285+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:56:25.334285+0000","last_peered":"2024-01-11T17:56:25.334285+0000","last_clean":"2024-01-11T17:56:25.334285+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:56:25.334285+0000","last_undegraded":"2024-01-11T17:56:25.334285+0000","last_fullsized":"2024-01-11T17:56:25.334285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338275+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:56:25.338275+0000","last_peered":"2024-01-11T17:56:25.338275+0000","last_clean":"2024-01-11T17:56:25.338275+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:56:25.338275+0000","last_undegraded":"2024-01-11T17:56:25.338275+0000","last_fullsized":"2024-01-11T17:56:25.338275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389005+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:56:32.389005+0000","last_peered":"2024-01-11T17:56:32.389005+0000","last_clean":"2024-01-11T17:56:32.389005+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:56:32.389005+0000","last_undegraded":"2024-01-11T17:56:32.389005+0000","last_fullsized":"2024-01-11T17:56:32.389005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334931+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:56:25.334931+0000","last_peered":"2024-01-11T17:56:25.334931+0000","last_clean":"2024-01-11T17:56:25.334931+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:56:25.334931+0000","last_undegraded":"2024-01-11T17:56:25.334931+0000","last_fullsized":"2024-01-11T17:56:25.334931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331684+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:56:25.331684+0000","last_peered":"2024-01-11T17:56:25.331684+0000","last_clean":"2024-01-11T17:56:25.331684+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:56:25.331684+0000","last_undegraded":"2024-01-11T17:56:25.331684+0000","last_fullsized":"2024-01-11T17:56:25.331684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.347569+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:56:26.347569+0000","last_peered":"2024-01-11T17:56:26.347569+0000","last_clean":"2024-01-11T17:56:26.347569+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:56:26.347569+0000","last_undegraded":"2024-01-11T17:56:26.347569+0000","last_fullsized":"2024-01-11T17:56:26.347569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339412+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:56:25.339412+0000","last_peered":"2024-01-11T17:56:25.339412+0000","last_clean":"2024-01-11T17:56:25.339412+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:56:25.339412+0000","last_undegraded":"2024-01-11T17:56:25.339412+0000","last_fullsized":"2024-01-11T17:56:25.339412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331945+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:56:25.331945+0000","last_peered":"2024-01-11T17:56:25.331945+0000","last_clean":"2024-01-11T17:56:25.331945+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:56:25.331945+0000","last_undegraded":"2024-01-11T17:56:25.331945+0000","last_fullsized":"2024-01-11T17:56:25.331945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388801+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:56:32.388801+0000","last_peered":"2024-01-11T17:56:32.388801+0000","last_clean":"2024-01-11T17:56:32.388801+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:56:32.388801+0000","last_undegraded":"2024-01-11T17:56:32.388801+0000","last_fullsized":"2024-01-11T17:56:32.388801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341965+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:56:25.341965+0000","last_peered":"2024-01-11T17:56:25.341965+0000","last_clean":"2024-01-11T17:56:25.341965+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:56:25.341965+0000","last_undegraded":"2024-01-11T17:56:25.341965+0000","last_fullsized":"2024-01-11T17:56:25.341965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337490+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:56:25.337490+0000","last_peered":"2024-01-11T17:56:25.337490+0000","last_clean":"2024-01-11T17:56:25.337490+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:56:25.337490+0000","last_undegraded":"2024-01-11T17:56:25.337490+0000","last_fullsized":"2024-01-11T17:56:25.337490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.396644+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:56:33.396644+0000","last_peered":"2024-01-11T17:56:33.396644+0000","last_clean":"2024-01-11T17:56:33.396644+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:56:33.396644+0000","last_undegraded":"2024-01-11T17:56:33.396644+0000","last_fullsized":"2024-01-11T17:56:33.396644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341851+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:56:25.341851+0000","last_peered":"2024-01-11T17:56:25.341851+0000","last_clean":"2024-01-11T17:56:25.341851+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:56:25.341851+0000","last_undegraded":"2024-01-11T17:56:25.341851+0000","last_fullsized":"2024-01-11T17:56:25.341851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345480+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:56:26.345480+0000","last_peered":"2024-01-11T17:56:26.345480+0000","last_clean":"2024-01-11T17:56:26.345480+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:56:26.345480+0000","last_undegraded":"2024-01-11T17:56:26.345480+0000","last_fullsized":"2024-01-11T17:56:26.345480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345371+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:56:26.345371+0000","last_peered":"2024-01-11T17:56:26.345371+0000","last_clean":"2024-01-11T17:56:26.345371+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:56:26.345371+0000","last_undegraded":"2024-01-11T17:56:26.345371+0000","last_fullsized":"2024-01-11T17:56:26.345371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341451+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:56:25.341451+0000","last_peered":"2024-01-11T17:56:25.341451+0000","last_clean":"2024-01-11T17:56:25.341451+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:56:25.341451+0000","last_undegraded":"2024-01-11T17:56:25.341451+0000","last_fullsized":"2024-01-11T17:56:25.341451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337848+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:56:25.337848+0000","last_peered":"2024-01-11T17:56:25.337848+0000","last_clean":"2024-01-11T17:56:25.337848+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:56:25.337848+0000","last_undegraded":"2024-01-11T17:56:25.337848+0000","last_fullsized":"2024-01-11T17:56:25.337848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332029+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:56:25.332029+0000","last_peered":"2024-01-11T17:56:25.332029+0000","last_clean":"2024-01-11T17:56:25.332029+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:56:25.332029+0000","last_undegraded":"2024-01-11T17:56:25.332029+0000","last_fullsized":"2024-01-11T17:56:25.332029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341977+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:56:25.341977+0000","last_peered":"2024-01-11T17:56:25.341977+0000","last_clean":"2024-01-11T17:56:25.341977+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:56:25.341977+0000","last_undegraded":"2024-01-11T17:56:25.341977+0000","last_fullsized":"2024-01-11T17:56:25.341977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335030+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:56:25.335030+0000","last_peered":"2024-01-11T17:56:25.335030+0000","last_clean":"2024-01-11T17:56:25.335030+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:56:25.335030+0000","last_undegraded":"2024-01-11T17:56:25.335030+0000","last_fullsized":"2024-01-11T17:56:25.335030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355996+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:56:26.355996+0000","last_peered":"2024-01-11T17:56:26.355996+0000","last_clean":"2024-01-11T17:56:26.355996+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:56:26.355996+0000","last_undegraded":"2024-01-11T17:56:26.355996+0000","last_fullsized":"2024-01-11T17:56:26.355996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336409+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:56:25.336409+0000","last_peered":"2024-01-11T17:56:25.336409+0000","last_clean":"2024-01-11T17:56:25.336409+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:56:25.336409+0000","last_undegraded":"2024-01-11T17:56:25.336409+0000","last_fullsized":"2024-01-11T17:56:25.336409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336125+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:56:25.336125+0000","last_peered":"2024-01-11T17:56:25.336125+0000","last_clean":"2024-01-11T17:56:25.336125+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:56:25.336125+0000","last_undegraded":"2024-01-11T17:56:25.336125+0000","last_fullsized":"2024-01-11T17:56:25.336125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332015+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:56:25.332015+0000","last_peered":"2024-01-11T17:56:25.332015+0000","last_clean":"2024-01-11T17:56:25.332015+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:56:25.332015+0000","last_undegraded":"2024-01-11T17:56:25.332015+0000","last_fullsized":"2024-01-11T17:56:25.332015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336086+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:56:25.336086+0000","last_peered":"2024-01-11T17:56:25.336086+0000","last_clean":"2024-01-11T17:56:25.336086+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:56:25.336086+0000","last_undegraded":"2024-01-11T17:56:25.336086+0000","last_fullsized":"2024-01-11T17:56:25.336086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338479+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:56:25.338479+0000","last_peered":"2024-01-11T17:56:25.338479+0000","last_clean":"2024-01-11T17:56:25.338479+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:56:25.338479+0000","last_undegraded":"2024-01-11T17:56:25.338479+0000","last_fullsized":"2024-01-11T17:56:25.338479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341422+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:56:25.341422+0000","last_peered":"2024-01-11T17:56:25.341422+0000","last_clean":"2024-01-11T17:56:25.341422+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:56:25.341422+0000","last_undegraded":"2024-01-11T17:56:25.341422+0000","last_fullsized":"2024-01-11T17:56:25.341422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338107+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:56:25.338107+0000","last_peered":"2024-01-11T17:56:25.338107+0000","last_clean":"2024-01-11T17:56:25.338107+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:56:25.338107+0000","last_undegraded":"2024-01-11T17:56:25.338107+0000","last_fullsized":"2024-01-11T17:56:25.338107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339143+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:56:25.339143+0000","last_peered":"2024-01-11T17:56:25.339143+0000","last_clean":"2024-01-11T17:56:25.339143+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:56:25.339143+0000","last_undegraded":"2024-01-11T17:56:25.339143+0000","last_fullsized":"2024-01-11T17:56:25.339143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337858+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:56:25.337858+0000","last_peered":"2024-01-11T17:56:25.337858+0000","last_clean":"2024-01-11T17:56:25.337858+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:56:25.337858+0000","last_undegraded":"2024-01-11T17:56:25.337858+0000","last_fullsized":"2024-01-11T17:56:25.337858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332218+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:56:25.332218+0000","last_peered":"2024-01-11T17:56:25.332218+0000","last_clean":"2024-01-11T17:56:25.332218+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:56:25.332218+0000","last_undegraded":"2024-01-11T17:56:25.332218+0000","last_fullsized":"2024-01-11T17:56:25.332218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355928+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:56:26.355928+0000","last_peered":"2024-01-11T17:56:26.355928+0000","last_clean":"2024-01-11T17:56:26.355928+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:56:26.355928+0000","last_undegraded":"2024-01-11T17:56:26.355928+0000","last_fullsized":"2024-01-11T17:56:26.355928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7729,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":121,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":41929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":469002,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.495}]},{"osd":1,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.622},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.85399999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.92500000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776147,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":1064207,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.63},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78000000000000003}]}]},{"osd":9,"up_from":63,"seq":270582939670,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":473334,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.496}]},{"osd":1,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.629}]},{"osd":2,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.86099999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168603,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":469046,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.88},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1,"5min":1,"15min":1},"last":0.94699999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":1.097}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.997}]},{"osd":10,"last update":"Thu Jan 11 17:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.129},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.034}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397535,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1060580,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.63400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.627,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84499999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":1.137,"15min":1.137},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.65100000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.748},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.004,"5min":1.04,"15min":1.04},"last":0.60299999999999998}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0960000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.145,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.69699999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":6,"up_from":44,"seq":188978561059,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":468510,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.45900000000000002},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0920000000000001,"5min":1.179,"15min":1.179},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.54200000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0720000000000001,"5min":1.161,"15min":1.161},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92000000000000004,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0349999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1040000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.58599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95699999999999996,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89500000000000002,"15min":0.89400000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0669999999999999,"5min":1.131,"15min":1.131},"last":0.71499999999999997}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0980000000000001,"5min":1.155,"15min":1.155},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.871}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":478860,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.03,"5min":1.077,"15min":1.077},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89400000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.42899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 11 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98699999999999999,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.44400000000000001,"15min":0.40000000000000002},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.48199999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.69799999999999995,"15min":0.68000000000000005},"min":{"1min":0.61899999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.59399999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.67600000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.61499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.72499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.056,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 11 17:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.85599999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.83099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":469898,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.55500000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.62,"15min":0.58799999999999997},"min":{"1min":0.50900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.59899999999999998,"15min":0.56299999999999994},"min":{"1min":0.52100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 11 17:54:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.45500000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47099999999999997},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.70099999999999996,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.622}]},{"osd":5,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 11 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.746},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.68300000000000005,"15min":0.66300000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66400000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.76700000000000002}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.80400000000000005}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 11 17:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":0.90500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":473611,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60299999999999998,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.438,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.95599999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":468704,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.38900000000000001},"min":{"1min":0.34399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.375},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.39200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71499999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.57699999999999996,"15min":0.52100000000000002},"min":{"1min":0.53000000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.98599999999999999,"5min":1.052,"15min":1.052},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.59299999999999997,"15min":0.53800000000000003},"min":{"1min":0.57599999999999996,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.024,"5min":1.079,"15min":1.079},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.53400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98799999999999999,"5min":1.137,"15min":1.137},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.65300000000000002,"15min":0.59399999999999997},"min":{"1min":0.50700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.68700000000000006,"15min":0.628},"min":{"1min":0.57399999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1020000000000001,"5min":1.204,"15min":1.204},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.438}]},{"osd":7,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.65100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0580000000000001,"5min":1.379,"15min":1.379},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.499,"15min":0.499},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.73199999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953515,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":469454,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.374},"min":{"1min":0.36099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.33600000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.91700000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.60099999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1,"5min":1.034,"15min":1.034},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.61899999999999999,"15min":0.59099999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.57699999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89800000000000002,"5min":1.129,"15min":1.129},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.67400000000000004,"15min":0.65000000000000002},"min":{"1min":0.54400000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.69199999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.499,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73399999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93200000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.024,"5min":1.238,"15min":1.238},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.71799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789992,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1060264,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.35099999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.47799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2529999999999999,"5min":1.548,"15min":1.548},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.751},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.58899999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.61199999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.58499999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.56499999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93999999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.87}]},{"osd":9,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.79100000000000004}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:56:38.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 -- 172.21.15.7:0/3301504538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb381015a0 msgr2=0x7feb381019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 --2- 172.21.15.7:0/3301504538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb381015a0 0x7feb381019c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0099c0 tx=0x7feb2c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 -- 172.21.15.7:0/3301504538 shutdown_connections 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 --2- 172.21.15.7:0/3301504538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38103130 0x7feb381099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 --2- 172.21.15.7:0/3301504538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb381015a0 0x7feb381019c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 --2- 172.21.15.7:0/3301504538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38102790 0x7feb38102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 -- 172.21.15.7:0/3301504538 >> 172.21.15.7:0/3301504538 conn(0x7feb380fcb30 msgr2=0x7feb380fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 -- 172.21.15.7:0/3301504538 shutdown_connections 2024-01-11T17:56:38.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.566+0000 7feb3edff700 1 -- 172.21.15.7:0/3301504538 wait complete. 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.567+0000 7feb3edff700 1 Processor -- start 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.567+0000 7feb3edff700 1 -- start start 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3edff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb381015a0 0x7feb381996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 0x7feb38199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3edff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38103130 0x7feb381a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381a1260 con 0x7feb38102790 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3edff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381a13a0 con 0x7feb381015a0 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3edff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381a1530 con 0x7feb38103130 2024-01-11T17:56:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 0x7feb38199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 0x7feb38199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44400/0 (socket says 172.21.15.7:44400) 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 -- 172.21.15.7:0/1781927146 learned_addr learned my addr 172.21.15.7:0/1781927146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3d39c700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38103130 0x7feb381a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3cb9b700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb381015a0 0x7feb381996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 -- 172.21.15.7:0/1781927146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38103130 msgr2=0x7feb381a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38103130 0x7feb381a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 -- 172.21.15.7:0/1781927146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb381015a0 msgr2=0x7feb381996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb381015a0 0x7feb381996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb37fff700 1 -- 172.21.15.7:0/1781927146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c0095f0 con 0x7feb38102790 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.568+0000 7feb3d39c700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38103130 0x7feb381a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.569+0000 7feb37fff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 0x7feb38199c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e9a0 tx=0x7feb2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.569+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24009790 con 0x7feb38102790 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.569+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb24010430 con 0x7feb38102790 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.569+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb381a1810 con 0x7feb38102790 2024-01-11T17:56:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.569+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24018680 con 0x7feb38102790 2024-01-11T17:56:38.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.569+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb381a1d20 con 0x7feb38102790 2024-01-11T17:56:38.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.570+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb380683f0 con 0x7feb38102790 2024-01-11T17:56:38.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:38 smithi007 ceph-mon[33361]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:38.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:38 smithi007 ceph-mon[33361]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-01-11T17:56:38.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:38 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2966243198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:38.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.572+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7feb240105a0 con 0x7feb38102790 2024-01-11T17:56:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.573+0000 7feb35ffb700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feb280a1f70 0x7feb280a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.574+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb24014070 con 0x7feb38102790 2024-01-11T17:56:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.574+0000 7feb3cb9b700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feb280a1f70 0x7feb280a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.574+0000 7feb3cb9b700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feb280a1f70 0x7feb280a4430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb2c003730 tx=0x7feb2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.578+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7feb24052380 con 0x7feb38102790 2024-01-11T17:56:38.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:38 smithi016 ceph-mon[43195]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:38.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:38 smithi016 ceph-mon[43195]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-01-11T17:56:38.655 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:38 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2966243198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:38.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:38 smithi038 ceph-mon[43582]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:56:38.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:38 smithi038 ceph-mon[43582]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-01-11T17:56:38.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:38 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2966243198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:38.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.787+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb38074f60 con 0x7feb280a1f70 2024-01-11T17:56:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.800+0000 7feb35ffb700 1 -- 172.21.15.7:0/1781927146 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422564 (secure 0 0 0) 0x7feb38074f60 con 0x7feb280a1f70 2024-01-11T17:56:38.807 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feb280a1f70 msgr2=0x7feb280a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feb280a1f70 0x7feb280a4430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb2c003730 tx=0x7feb2c000bc0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 msgr2=0x7feb38199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 0x7feb38199c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e9a0 tx=0x7feb2400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 shutdown_connections 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feb280a1f70 0x7feb280a4430 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb38103130 0x7feb381a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb381015a0 0x7feb381996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 --2- 172.21.15.7:0/1781927146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb38102790 0x7feb38199c20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 >> 172.21.15.7:0/1781927146 conn(0x7feb380fcb30 msgr2=0x7feb38073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 shutdown_connections 2024-01-11T17:56:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:38.806+0000 7feb3edff700 1 -- 172.21.15.7:0/1781927146 wait complete. 2024-01-11T17:56:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:56:38.825 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-01-11T17:56:38.572895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598209,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":153,"num_read_kb":135,"num_write":310,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385408,"kb_used_data":11296,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556416,"statfs":{"total":1151940427776,"available":1151545769984,"internally_reserved":0,"allocated":11567104,"data_stored":7429678,"data_compressed":320463,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3343,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":27,"num_read_kb":26,"num_write":82,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338271+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:56:25.338271+0000","last_peered":"2024-01-11T17:56:25.338271+0000","last_clean":"2024-01-11T17:56:25.338271+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:56:25.338271+0000","last_undegraded":"2024-01-11T17:56:25.338271+0000","last_fullsized":"2024-01-11T17:56:25.338271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336434+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:56:25.336434+0000","last_peered":"2024-01-11T17:56:25.336434+0000","last_clean":"2024-01-11T17:56:25.336434+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:56:25.336434+0000","last_undegraded":"2024-01-11T17:56:25.336434+0000","last_fullsized":"2024-01-11T17:56:25.336434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336251+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:56:25.336251+0000","last_peered":"2024-01-11T17:56:25.336251+0000","last_clean":"2024-01-11T17:56:25.336251+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:56:25.336251+0000","last_undegraded":"2024-01-11T17:56:25.336251+0000","last_fullsized":"2024-01-11T17:56:25.336251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332674+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:56:25.332674+0000","last_peered":"2024-01-11T17:56:25.332674+0000","last_clean":"2024-01-11T17:56:25.332674+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:56:25.332674+0000","last_undegraded":"2024-01-11T17:56:25.332674+0000","last_fullsized":"2024-01-11T17:56:25.332674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355950+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:56:26.355950+0000","last_peered":"2024-01-11T17:56:26.355950+0000","last_clean":"2024-01-11T17:56:26.355950+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:56:26.355950+0000","last_undegraded":"2024-01-11T17:56:26.355950+0000","last_fullsized":"2024-01-11T17:56:26.355950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355667+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:56:26.355667+0000","last_peered":"2024-01-11T17:56:26.355667+0000","last_clean":"2024-01-11T17:56:26.355667+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:56:26.355667+0000","last_undegraded":"2024-01-11T17:56:26.355667+0000","last_fullsized":"2024-01-11T17:56:26.355667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341507+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:56:25.341507+0000","last_peered":"2024-01-11T17:56:25.341507+0000","last_clean":"2024-01-11T17:56:25.341507+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:56:25.341507+0000","last_undegraded":"2024-01-11T17:56:25.341507+0000","last_fullsized":"2024-01-11T17:56:25.341507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338751+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:56:25.338751+0000","last_peered":"2024-01-11T17:56:25.338751+0000","last_clean":"2024-01-11T17:56:25.338751+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:56:25.338751+0000","last_undegraded":"2024-01-11T17:56:25.338751+0000","last_fullsized":"2024-01-11T17:56:25.338751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338413+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:56:25.338413+0000","last_peered":"2024-01-11T17:56:25.338413+0000","last_clean":"2024-01-11T17:56:25.338413+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:56:25.338413+0000","last_undegraded":"2024-01-11T17:56:25.338413+0000","last_fullsized":"2024-01-11T17:56:25.338413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335441+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:56:25.335441+0000","last_peered":"2024-01-11T17:56:25.335441+0000","last_clean":"2024-01-11T17:56:25.335441+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:56:25.335441+0000","last_undegraded":"2024-01-11T17:56:25.335441+0000","last_fullsized":"2024-01-11T17:56:25.335441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341624+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:56:25.341624+0000","last_peered":"2024-01-11T17:56:25.341624+0000","last_clean":"2024-01-11T17:56:25.341624+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:56:25.341624+0000","last_undegraded":"2024-01-11T17:56:25.341624+0000","last_fullsized":"2024-01-11T17:56:25.341624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332662+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:56:25.332662+0000","last_peered":"2024-01-11T17:56:25.332662+0000","last_clean":"2024-01-11T17:56:25.332662+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:56:25.332662+0000","last_undegraded":"2024-01-11T17:56:25.332662+0000","last_fullsized":"2024-01-11T17:56:25.332662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332139+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:56:25.332139+0000","last_peered":"2024-01-11T17:56:25.332139+0000","last_clean":"2024-01-11T17:56:25.332139+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:56:25.332139+0000","last_undegraded":"2024-01-11T17:56:25.332139+0000","last_fullsized":"2024-01-11T17:56:25.332139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335487+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:56:25.335487+0000","last_peered":"2024-01-11T17:56:25.335487+0000","last_clean":"2024-01-11T17:56:25.335487+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:56:25.335487+0000","last_undegraded":"2024-01-11T17:56:25.335487+0000","last_fullsized":"2024-01-11T17:56:25.335487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355728+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:56:26.355728+0000","last_peered":"2024-01-11T17:56:26.355728+0000","last_clean":"2024-01-11T17:56:26.355728+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:56:26.355728+0000","last_undegraded":"2024-01-11T17:56:26.355728+0000","last_fullsized":"2024-01-11T17:56:26.355728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337407+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:56:25.337407+0000","last_peered":"2024-01-11T17:56:25.337407+0000","last_clean":"2024-01-11T17:56:25.337407+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:56:25.337407+0000","last_undegraded":"2024-01-11T17:56:25.337407+0000","last_fullsized":"2024-01-11T17:56:25.337407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337914+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:56:25.337914+0000","last_peered":"2024-01-11T17:56:25.337914+0000","last_clean":"2024-01-11T17:56:25.337914+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:56:25.337914+0000","last_undegraded":"2024-01-11T17:56:25.337914+0000","last_fullsized":"2024-01-11T17:56:25.337914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336121+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:56:25.336121+0000","last_peered":"2024-01-11T17:56:25.336121+0000","last_clean":"2024-01-11T17:56:25.336121+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:56:25.336121+0000","last_undegraded":"2024-01-11T17:56:25.336121+0000","last_fullsized":"2024-01-11T17:56:25.336121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338989+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:56:25.338989+0000","last_peered":"2024-01-11T17:56:25.338989+0000","last_clean":"2024-01-11T17:56:25.338989+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:56:25.338989+0000","last_undegraded":"2024-01-11T17:56:25.338989+0000","last_fullsized":"2024-01-11T17:56:25.338989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340885+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:56:25.340885+0000","last_peered":"2024-01-11T17:56:25.340885+0000","last_clean":"2024-01-11T17:56:25.340885+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:56:25.340885+0000","last_undegraded":"2024-01-11T17:56:25.340885+0000","last_fullsized":"2024-01-11T17:56:25.340885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397873+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:56:33.397873+0000","last_peered":"2024-01-11T17:56:33.397873+0000","last_clean":"2024-01-11T17:56:33.397873+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:56:33.397873+0000","last_undegraded":"2024-01-11T17:56:33.397873+0000","last_fullsized":"2024-01-11T17:56:33.397873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339633+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:56:25.339633+0000","last_peered":"2024-01-11T17:56:25.339633+0000","last_clean":"2024-01-11T17:56:25.339633+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:56:25.339633+0000","last_undegraded":"2024-01-11T17:56:25.339633+0000","last_fullsized":"2024-01-11T17:56:25.339633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331322+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:56:25.331322+0000","last_peered":"2024-01-11T17:56:25.331322+0000","last_clean":"2024-01-11T17:56:25.331322+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:56:25.331322+0000","last_undegraded":"2024-01-11T17:56:25.331322+0000","last_fullsized":"2024-01-11T17:56:25.331322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.333494+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:56:26.333494+0000","last_peered":"2024-01-11T17:56:26.333494+0000","last_clean":"2024-01-11T17:56:26.333494+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:56:26.333494+0000","last_undegraded":"2024-01-11T17:56:26.333494+0000","last_fullsized":"2024-01-11T17:56:26.333494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336511+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:56:25.336511+0000","last_peered":"2024-01-11T17:56:25.336511+0000","last_clean":"2024-01-11T17:56:25.336511+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:56:25.336511+0000","last_undegraded":"2024-01-11T17:56:25.336511+0000","last_fullsized":"2024-01-11T17:56:25.336511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331455+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:56:25.331455+0000","last_peered":"2024-01-11T17:56:25.331455+0000","last_clean":"2024-01-11T17:56:25.331455+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:56:25.331455+0000","last_undegraded":"2024-01-11T17:56:25.331455+0000","last_fullsized":"2024-01-11T17:56:25.331455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395899+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:56:33.395899+0000","last_peered":"2024-01-11T17:56:33.395899+0000","last_clean":"2024-01-11T17:56:33.395899+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:56:33.395899+0000","last_undegraded":"2024-01-11T17:56:33.395899+0000","last_fullsized":"2024-01-11T17:56:33.395899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337771+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:56:25.337771+0000","last_peered":"2024-01-11T17:56:25.337771+0000","last_clean":"2024-01-11T17:56:25.337771+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:56:25.337771+0000","last_undegraded":"2024-01-11T17:56:25.337771+0000","last_fullsized":"2024-01-11T17:56:25.337771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331648+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:56:25.331648+0000","last_peered":"2024-01-11T17:56:25.331648+0000","last_clean":"2024-01-11T17:56:25.331648+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:56:25.331648+0000","last_undegraded":"2024-01-11T17:56:25.331648+0000","last_fullsized":"2024-01-11T17:56:25.331648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.351148+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:56:26.351148+0000","last_peered":"2024-01-11T17:56:26.351148+0000","last_clean":"2024-01-11T17:56:26.351148+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:56:26.351148+0000","last_undegraded":"2024-01-11T17:56:26.351148+0000","last_fullsized":"2024-01-11T17:56:26.351148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335605+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:56:25.335605+0000","last_peered":"2024-01-11T17:56:25.335605+0000","last_clean":"2024-01-11T17:56:25.335605+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:56:25.335605+0000","last_undegraded":"2024-01-11T17:56:25.335605+0000","last_fullsized":"2024-01-11T17:56:25.335605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338781+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:56:25.338781+0000","last_peered":"2024-01-11T17:56:25.338781+0000","last_clean":"2024-01-11T17:56:25.338781+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:56:25.338781+0000","last_undegraded":"2024-01-11T17:56:25.338781+0000","last_fullsized":"2024-01-11T17:56:25.338781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.326764+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:56:34.326764+0000","last_peered":"2024-01-11T17:56:34.326764+0000","last_clean":"2024-01-11T17:56:34.326764+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:56:34.326764+0000","last_undegraded":"2024-01-11T17:56:34.326764+0000","last_fullsized":"2024-01-11T17:56:34.326764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335788+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:56:25.335788+0000","last_peered":"2024-01-11T17:56:25.335788+0000","last_clean":"2024-01-11T17:56:25.335788+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:56:25.335788+0000","last_undegraded":"2024-01-11T17:56:25.335788+0000","last_fullsized":"2024-01-11T17:56:25.335788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337755+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:56:25.337755+0000","last_peered":"2024-01-11T17:56:25.337755+0000","last_clean":"2024-01-11T17:56:25.337755+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:56:25.337755+0000","last_undegraded":"2024-01-11T17:56:25.337755+0000","last_fullsized":"2024-01-11T17:56:25.337755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395256+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:56:33.395256+0000","last_peered":"2024-01-11T17:56:33.395256+0000","last_clean":"2024-01-11T17:56:33.395256+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:56:33.395256+0000","last_undegraded":"2024-01-11T17:56:33.395256+0000","last_fullsized":"2024-01-11T17:56:33.395256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337679+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:56:25.337679+0000","last_peered":"2024-01-11T17:56:25.337679+0000","last_clean":"2024-01-11T17:56:25.337679+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:56:25.337679+0000","last_undegraded":"2024-01-11T17:56:25.337679+0000","last_fullsized":"2024-01-11T17:56:25.337679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337811+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:56:25.337811+0000","last_peered":"2024-01-11T17:56:25.337811+0000","last_clean":"2024-01-11T17:56:25.337811+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:56:25.337811+0000","last_undegraded":"2024-01-11T17:56:25.337811+0000","last_fullsized":"2024-01-11T17:56:25.337811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397732+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:56:33.397732+0000","last_peered":"2024-01-11T17:56:33.397732+0000","last_clean":"2024-01-11T17:56:33.397732+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:56:33.397732+0000","last_undegraded":"2024-01-11T17:56:33.397732+0000","last_fullsized":"2024-01-11T17:56:33.397732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338662+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:56:25.338662+0000","last_peered":"2024-01-11T17:56:25.338662+0000","last_clean":"2024-01-11T17:56:25.338662+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:56:25.338662+0000","last_undegraded":"2024-01-11T17:56:25.338662+0000","last_fullsized":"2024-01-11T17:56:25.338662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336435+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:56:25.336435+0000","last_peered":"2024-01-11T17:56:25.336435+0000","last_clean":"2024-01-11T17:56:25.336435+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:56:25.336435+0000","last_undegraded":"2024-01-11T17:56:25.336435+0000","last_fullsized":"2024-01-11T17:56:25.336435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.344358+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:56:26.344358+0000","last_peered":"2024-01-11T17:56:26.344358+0000","last_clean":"2024-01-11T17:56:26.344358+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:56:26.344358+0000","last_undegraded":"2024-01-11T17:56:26.344358+0000","last_fullsized":"2024-01-11T17:56:26.344358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335428+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:56:25.335428+0000","last_peered":"2024-01-11T17:56:25.335428+0000","last_clean":"2024-01-11T17:56:25.335428+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:56:25.335428+0000","last_undegraded":"2024-01-11T17:56:25.335428+0000","last_fullsized":"2024-01-11T17:56:25.335428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338974+0000","last_change":"2024-01-11T17:56:12.824145+0000","last_active":"2024-01-11T17:56:25.338974+0000","last_peered":"2024-01-11T17:56:25.338974+0000","last_clean":"2024-01-11T17:56:25.338974+0000","last_became_active":"2024-01-11T17:56:12.823443+0000","last_became_peered":"2024-01-11T17:56:12.823443+0000","last_unstale":"2024-01-11T17:56:25.338974+0000","last_undegraded":"2024-01-11T17:56:25.338974+0000","last_fullsized":"2024-01-11T17:56:25.338974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:34:19.317304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.390278+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:56:32.390278+0000","last_peered":"2024-01-11T17:56:32.390278+0000","last_clean":"2024-01-11T17:56:32.390278+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:56:32.390278+0000","last_undegraded":"2024-01-11T17:56:32.390278+0000","last_fullsized":"2024-01-11T17:56:32.390278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338217+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:56:25.338217+0000","last_peered":"2024-01-11T17:56:25.338217+0000","last_clean":"2024-01-11T17:56:25.338217+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:56:25.338217+0000","last_undegraded":"2024-01-11T17:56:25.338217+0000","last_fullsized":"2024-01-11T17:56:25.338217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334836+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:56:25.334836+0000","last_peered":"2024-01-11T17:56:25.334836+0000","last_clean":"2024-01-11T17:56:25.334836+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:56:25.334836+0000","last_undegraded":"2024-01-11T17:56:25.334836+0000","last_fullsized":"2024-01-11T17:56:25.334836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388337+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:56:32.388337+0000","last_peered":"2024-01-11T17:56:32.388337+0000","last_clean":"2024-01-11T17:56:32.388337+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:56:32.388337+0000","last_undegraded":"2024-01-11T17:56:32.388337+0000","last_fullsized":"2024-01-11T17:56:32.388337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336605+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:56:25.336605+0000","last_peered":"2024-01-11T17:56:25.336605+0000","last_clean":"2024-01-11T17:56:25.336605+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:56:25.336605+0000","last_undegraded":"2024-01-11T17:56:25.336605+0000","last_fullsized":"2024-01-11T17:56:25.336605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331894+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:56:25.331894+0000","last_peered":"2024-01-11T17:56:25.331894+0000","last_clean":"2024-01-11T17:56:25.331894+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:56:25.331894+0000","last_undegraded":"2024-01-11T17:56:25.331894+0000","last_fullsized":"2024-01-11T17:56:25.331894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391761+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:56:32.391761+0000","last_peered":"2024-01-11T17:56:32.391761+0000","last_clean":"2024-01-11T17:56:32.391761+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:56:32.391761+0000","last_undegraded":"2024-01-11T17:56:32.391761+0000","last_fullsized":"2024-01-11T17:56:32.391761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337845+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:56:25.337845+0000","last_peered":"2024-01-11T17:56:25.337845+0000","last_clean":"2024-01-11T17:56:25.337845+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:56:25.337845+0000","last_undegraded":"2024-01-11T17:56:25.337845+0000","last_fullsized":"2024-01-11T17:56:25.337845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336863+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:56:25.336863+0000","last_peered":"2024-01-11T17:56:25.336863+0000","last_clean":"2024-01-11T17:56:25.336863+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:56:25.336863+0000","last_undegraded":"2024-01-11T17:56:25.336863+0000","last_fullsized":"2024-01-11T17:56:25.336863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.327572+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:56:34.327572+0000","last_peered":"2024-01-11T17:56:34.327572+0000","last_clean":"2024-01-11T17:56:34.327572+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:56:34.327572+0000","last_undegraded":"2024-01-11T17:56:34.327572+0000","last_fullsized":"2024-01-11T17:56:34.327572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336092+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:56:25.336092+0000","last_peered":"2024-01-11T17:56:25.336092+0000","last_clean":"2024-01-11T17:56:25.336092+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:56:25.336092+0000","last_undegraded":"2024-01-11T17:56:25.336092+0000","last_fullsized":"2024-01-11T17:56:25.336092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395453+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:56:33.395453+0000","last_peered":"2024-01-11T17:56:33.395453+0000","last_clean":"2024-01-11T17:56:33.395453+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:56:33.395453+0000","last_undegraded":"2024-01-11T17:56:33.395453+0000","last_fullsized":"2024-01-11T17:56:33.395453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331833+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:56:25.331833+0000","last_peered":"2024-01-11T17:56:25.331833+0000","last_clean":"2024-01-11T17:56:25.331833+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:56:25.331833+0000","last_undegraded":"2024-01-11T17:56:25.331833+0000","last_fullsized":"2024-01-11T17:56:25.331833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337590+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:56:25.337590+0000","last_peered":"2024-01-11T17:56:25.337590+0000","last_clean":"2024-01-11T17:56:25.337590+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:56:25.337590+0000","last_undegraded":"2024-01-11T17:56:25.337590+0000","last_fullsized":"2024-01-11T17:56:25.337590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391050+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:56:32.391050+0000","last_peered":"2024-01-11T17:56:32.391050+0000","last_clean":"2024-01-11T17:56:32.391050+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:56:32.391050+0000","last_undegraded":"2024-01-11T17:56:32.391050+0000","last_fullsized":"2024-01-11T17:56:32.391050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341538+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:56:25.341538+0000","last_peered":"2024-01-11T17:56:25.341538+0000","last_clean":"2024-01-11T17:56:25.341538+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:56:25.341538+0000","last_undegraded":"2024-01-11T17:56:25.341538+0000","last_fullsized":"2024-01-11T17:56:25.341538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.343158+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:56:26.343158+0000","last_peered":"2024-01-11T17:56:26.343158+0000","last_clean":"2024-01-11T17:56:26.343158+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:56:26.343158+0000","last_undegraded":"2024-01-11T17:56:26.343158+0000","last_fullsized":"2024-01-11T17:56:26.343158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:35.331005+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:56:35.331005+0000","last_peered":"2024-01-11T17:56:35.331005+0000","last_clean":"2024-01-11T17:56:35.331005+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:56:35.331005+0000","last_undegraded":"2024-01-11T17:56:35.331005+0000","last_fullsized":"2024-01-11T17:56:35.331005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341549+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:56:25.341549+0000","last_peered":"2024-01-11T17:56:25.341549+0000","last_clean":"2024-01-11T17:56:25.341549+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:56:25.341549+0000","last_undegraded":"2024-01-11T17:56:25.341549+0000","last_fullsized":"2024-01-11T17:56:25.341549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331924+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:56:25.331924+0000","last_peered":"2024-01-11T17:56:25.331924+0000","last_clean":"2024-01-11T17:56:25.331924+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:56:25.331924+0000","last_undegraded":"2024-01-11T17:56:25.331924+0000","last_fullsized":"2024-01-11T17:56:25.331924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.398418+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:56:33.398418+0000","last_peered":"2024-01-11T17:56:33.398418+0000","last_clean":"2024-01-11T17:56:33.398418+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:56:33.398418+0000","last_undegraded":"2024-01-11T17:56:33.398418+0000","last_fullsized":"2024-01-11T17:56:33.398418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336728+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:56:25.336728+0000","last_peered":"2024-01-11T17:56:25.336728+0000","last_clean":"2024-01-11T17:56:25.336728+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:56:25.336728+0000","last_undegraded":"2024-01-11T17:56:25.336728+0000","last_fullsized":"2024-01-11T17:56:25.336728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336598+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:56:25.336598+0000","last_peered":"2024-01-11T17:56:25.336598+0000","last_clean":"2024-01-11T17:56:25.336598+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:56:25.336598+0000","last_undegraded":"2024-01-11T17:56:25.336598+0000","last_fullsized":"2024-01-11T17:56:25.336598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324929+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:56:34.324929+0000","last_peered":"2024-01-11T17:56:34.324929+0000","last_clean":"2024-01-11T17:56:34.324929+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:56:34.324929+0000","last_undegraded":"2024-01-11T17:56:34.324929+0000","last_fullsized":"2024-01-11T17:56:34.324929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335418+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:56:25.335418+0000","last_peered":"2024-01-11T17:56:25.335418+0000","last_clean":"2024-01-11T17:56:25.335418+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:56:25.335418+0000","last_undegraded":"2024-01-11T17:56:25.335418+0000","last_fullsized":"2024-01-11T17:56:25.335418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335675+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:56:25.335675+0000","last_peered":"2024-01-11T17:56:25.335675+0000","last_clean":"2024-01-11T17:56:25.335675+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:56:25.335675+0000","last_undegraded":"2024-01-11T17:56:25.335675+0000","last_fullsized":"2024-01-11T17:56:25.335675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337326+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:56:25.337326+0000","last_peered":"2024-01-11T17:56:25.337326+0000","last_clean":"2024-01-11T17:56:25.337326+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:56:25.337326+0000","last_undegraded":"2024-01-11T17:56:25.337326+0000","last_fullsized":"2024-01-11T17:56:25.337326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335306+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:56:25.335306+0000","last_peered":"2024-01-11T17:56:25.335306+0000","last_clean":"2024-01-11T17:56:25.335306+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:56:25.335306+0000","last_undegraded":"2024-01-11T17:56:25.335306+0000","last_fullsized":"2024-01-11T17:56:25.335306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331851+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:56:25.331851+0000","last_peered":"2024-01-11T17:56:25.331851+0000","last_clean":"2024-01-11T17:56:25.331851+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:56:25.331851+0000","last_undegraded":"2024-01-11T17:56:25.331851+0000","last_fullsized":"2024-01-11T17:56:25.331851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338402+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:56:26.338402+0000","last_peered":"2024-01-11T17:56:26.338402+0000","last_clean":"2024-01-11T17:56:26.338402+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:56:26.338402+0000","last_undegraded":"2024-01-11T17:56:26.338402+0000","last_fullsized":"2024-01-11T17:56:26.338402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335017+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:56:25.335017+0000","last_peered":"2024-01-11T17:56:25.335017+0000","last_clean":"2024-01-11T17:56:25.335017+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:56:25.335017+0000","last_undegraded":"2024-01-11T17:56:25.335017+0000","last_fullsized":"2024-01-11T17:56:25.335017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331773+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:56:25.331773+0000","last_peered":"2024-01-11T17:56:25.331773+0000","last_clean":"2024-01-11T17:56:25.331773+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:56:25.331773+0000","last_undegraded":"2024-01-11T17:56:25.331773+0000","last_fullsized":"2024-01-11T17:56:25.331773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338194+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:56:25.338194+0000","last_peered":"2024-01-11T17:56:25.338194+0000","last_clean":"2024-01-11T17:56:25.338194+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:56:25.338194+0000","last_undegraded":"2024-01-11T17:56:25.338194+0000","last_fullsized":"2024-01-11T17:56:25.338194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.326548+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:56:34.326548+0000","last_peered":"2024-01-11T17:56:34.326548+0000","last_clean":"2024-01-11T17:56:34.326548+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:56:34.326548+0000","last_undegraded":"2024-01-11T17:56:34.326548+0000","last_fullsized":"2024-01-11T17:56:34.326548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341442+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:56:25.341442+0000","last_peered":"2024-01-11T17:56:25.341442+0000","last_clean":"2024-01-11T17:56:25.341442+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:56:25.341442+0000","last_undegraded":"2024-01-11T17:56:25.341442+0000","last_fullsized":"2024-01-11T17:56:25.341442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338909+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:56:25.338909+0000","last_peered":"2024-01-11T17:56:25.338909+0000","last_clean":"2024-01-11T17:56:25.338909+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:56:25.338909+0000","last_undegraded":"2024-01-11T17:56:25.338909+0000","last_fullsized":"2024-01-11T17:56:25.338909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324345+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:56:34.324345+0000","last_peered":"2024-01-11T17:56:34.324345+0000","last_clean":"2024-01-11T17:56:34.324345+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:56:34.324345+0000","last_undegraded":"2024-01-11T17:56:34.324345+0000","last_fullsized":"2024-01-11T17:56:34.324345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341318+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:56:25.341318+0000","last_peered":"2024-01-11T17:56:25.341318+0000","last_clean":"2024-01-11T17:56:25.341318+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:56:25.341318+0000","last_undegraded":"2024-01-11T17:56:25.341318+0000","last_fullsized":"2024-01-11T17:56:25.341318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331435+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:56:25.331435+0000","last_peered":"2024-01-11T17:56:25.331435+0000","last_clean":"2024-01-11T17:56:25.331435+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:56:25.331435+0000","last_undegraded":"2024-01-11T17:56:25.331435+0000","last_fullsized":"2024-01-11T17:56:25.331435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.394917+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:56:33.394917+0000","last_peered":"2024-01-11T17:56:33.394917+0000","last_clean":"2024-01-11T17:56:33.394917+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:56:33.394917+0000","last_undegraded":"2024-01-11T17:56:33.394917+0000","last_fullsized":"2024-01-11T17:56:33.394917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335272+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:56:25.335272+0000","last_peered":"2024-01-11T17:56:25.335272+0000","last_clean":"2024-01-11T17:56:25.335272+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:56:25.335272+0000","last_undegraded":"2024-01-11T17:56:25.335272+0000","last_fullsized":"2024-01-11T17:56:25.335272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388565+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:56:32.388565+0000","last_peered":"2024-01-11T17:56:32.388565+0000","last_clean":"2024-01-11T17:56:32.388565+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:56:32.388565+0000","last_undegraded":"2024-01-11T17:56:32.388565+0000","last_fullsized":"2024-01-11T17:56:32.388565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332040+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:56:25.332040+0000","last_peered":"2024-01-11T17:56:25.332040+0000","last_clean":"2024-01-11T17:56:25.332040+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:56:25.332040+0000","last_undegraded":"2024-01-11T17:56:25.332040+0000","last_fullsized":"2024-01-11T17:56:25.332040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340003+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:56:25.340003+0000","last_peered":"2024-01-11T17:56:25.340003+0000","last_clean":"2024-01-11T17:56:25.340003+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:56:25.340003+0000","last_undegraded":"2024-01-11T17:56:25.340003+0000","last_fullsized":"2024-01-11T17:56:25.340003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.396237+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:56:33.396237+0000","last_peered":"2024-01-11T17:56:33.396237+0000","last_clean":"2024-01-11T17:56:33.396237+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:56:33.396237+0000","last_undegraded":"2024-01-11T17:56:33.396237+0000","last_fullsized":"2024-01-11T17:56:33.396237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339772+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:56:25.339772+0000","last_peered":"2024-01-11T17:56:25.339772+0000","last_clean":"2024-01-11T17:56:25.339772+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:56:25.339772+0000","last_undegraded":"2024-01-11T17:56:25.339772+0000","last_fullsized":"2024-01-11T17:56:25.339772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340165+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:56:25.340165+0000","last_peered":"2024-01-11T17:56:25.340165+0000","last_clean":"2024-01-11T17:56:25.340165+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:56:25.340165+0000","last_undegraded":"2024-01-11T17:56:25.340165+0000","last_fullsized":"2024-01-11T17:56:25.340165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.386925+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:56:32.386925+0000","last_peered":"2024-01-11T17:56:32.386925+0000","last_clean":"2024-01-11T17:56:32.386925+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:56:32.386925+0000","last_undegraded":"2024-01-11T17:56:32.386925+0000","last_fullsized":"2024-01-11T17:56:32.386925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336540+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:56:25.336540+0000","last_peered":"2024-01-11T17:56:25.336540+0000","last_clean":"2024-01-11T17:56:25.336540+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:56:25.336540+0000","last_undegraded":"2024-01-11T17:56:25.336540+0000","last_fullsized":"2024-01-11T17:56:25.336540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.346059+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:56:26.346059+0000","last_peered":"2024-01-11T17:56:26.346059+0000","last_clean":"2024-01-11T17:56:26.346059+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:56:26.346059+0000","last_undegraded":"2024-01-11T17:56:26.346059+0000","last_fullsized":"2024-01-11T17:56:26.346059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388647+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:56:32.388647+0000","last_peered":"2024-01-11T17:56:32.388647+0000","last_clean":"2024-01-11T17:56:32.388647+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:56:32.388647+0000","last_undegraded":"2024-01-11T17:56:32.388647+0000","last_fullsized":"2024-01-11T17:56:32.388647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334285+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:56:25.334285+0000","last_peered":"2024-01-11T17:56:25.334285+0000","last_clean":"2024-01-11T17:56:25.334285+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:56:25.334285+0000","last_undegraded":"2024-01-11T17:56:25.334285+0000","last_fullsized":"2024-01-11T17:56:25.334285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338275+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:56:25.338275+0000","last_peered":"2024-01-11T17:56:25.338275+0000","last_clean":"2024-01-11T17:56:25.338275+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:56:25.338275+0000","last_undegraded":"2024-01-11T17:56:25.338275+0000","last_fullsized":"2024-01-11T17:56:25.338275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389005+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:56:32.389005+0000","last_peered":"2024-01-11T17:56:32.389005+0000","last_clean":"2024-01-11T17:56:32.389005+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:56:32.389005+0000","last_undegraded":"2024-01-11T17:56:32.389005+0000","last_fullsized":"2024-01-11T17:56:32.389005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334931+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:56:25.334931+0000","last_peered":"2024-01-11T17:56:25.334931+0000","last_clean":"2024-01-11T17:56:25.334931+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:56:25.334931+0000","last_undegraded":"2024-01-11T17:56:25.334931+0000","last_fullsized":"2024-01-11T17:56:25.334931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331684+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:56:25.331684+0000","last_peered":"2024-01-11T17:56:25.331684+0000","last_clean":"2024-01-11T17:56:25.331684+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:56:25.331684+0000","last_undegraded":"2024-01-11T17:56:25.331684+0000","last_fullsized":"2024-01-11T17:56:25.331684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.347569+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:56:26.347569+0000","last_peered":"2024-01-11T17:56:26.347569+0000","last_clean":"2024-01-11T17:56:26.347569+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:56:26.347569+0000","last_undegraded":"2024-01-11T17:56:26.347569+0000","last_fullsized":"2024-01-11T17:56:26.347569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339412+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:56:25.339412+0000","last_peered":"2024-01-11T17:56:25.339412+0000","last_clean":"2024-01-11T17:56:25.339412+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:56:25.339412+0000","last_undegraded":"2024-01-11T17:56:25.339412+0000","last_fullsized":"2024-01-11T17:56:25.339412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331945+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:56:25.331945+0000","last_peered":"2024-01-11T17:56:25.331945+0000","last_clean":"2024-01-11T17:56:25.331945+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:56:25.331945+0000","last_undegraded":"2024-01-11T17:56:25.331945+0000","last_fullsized":"2024-01-11T17:56:25.331945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389594+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:56:32.389594+0000","last_peered":"2024-01-11T17:56:32.389594+0000","last_clean":"2024-01-11T17:56:32.389594+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:56:32.389594+0000","last_undegraded":"2024-01-11T17:56:32.389594+0000","last_fullsized":"2024-01-11T17:56:32.389594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341965+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:56:25.341965+0000","last_peered":"2024-01-11T17:56:25.341965+0000","last_clean":"2024-01-11T17:56:25.341965+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:56:25.341965+0000","last_undegraded":"2024-01-11T17:56:25.341965+0000","last_fullsized":"2024-01-11T17:56:25.341965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337490+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:56:25.337490+0000","last_peered":"2024-01-11T17:56:25.337490+0000","last_clean":"2024-01-11T17:56:25.337490+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:56:25.337490+0000","last_undegraded":"2024-01-11T17:56:25.337490+0000","last_fullsized":"2024-01-11T17:56:25.337490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.396644+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:56:33.396644+0000","last_peered":"2024-01-11T17:56:33.396644+0000","last_clean":"2024-01-11T17:56:33.396644+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:56:33.396644+0000","last_undegraded":"2024-01-11T17:56:33.396644+0000","last_fullsized":"2024-01-11T17:56:33.396644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341851+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:56:25.341851+0000","last_peered":"2024-01-11T17:56:25.341851+0000","last_clean":"2024-01-11T17:56:25.341851+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:56:25.341851+0000","last_undegraded":"2024-01-11T17:56:25.341851+0000","last_fullsized":"2024-01-11T17:56:25.341851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345480+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:56:26.345480+0000","last_peered":"2024-01-11T17:56:26.345480+0000","last_clean":"2024-01-11T17:56:26.345480+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:56:26.345480+0000","last_undegraded":"2024-01-11T17:56:26.345480+0000","last_fullsized":"2024-01-11T17:56:26.345480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345371+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:56:26.345371+0000","last_peered":"2024-01-11T17:56:26.345371+0000","last_clean":"2024-01-11T17:56:26.345371+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:56:26.345371+0000","last_undegraded":"2024-01-11T17:56:26.345371+0000","last_fullsized":"2024-01-11T17:56:26.345371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341451+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:56:25.341451+0000","last_peered":"2024-01-11T17:56:25.341451+0000","last_clean":"2024-01-11T17:56:25.341451+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:56:25.341451+0000","last_undegraded":"2024-01-11T17:56:25.341451+0000","last_fullsized":"2024-01-11T17:56:25.341451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337848+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:56:25.337848+0000","last_peered":"2024-01-11T17:56:25.337848+0000","last_clean":"2024-01-11T17:56:25.337848+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:56:25.337848+0000","last_undegraded":"2024-01-11T17:56:25.337848+0000","last_fullsized":"2024-01-11T17:56:25.337848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332029+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:56:25.332029+0000","last_peered":"2024-01-11T17:56:25.332029+0000","last_clean":"2024-01-11T17:56:25.332029+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:56:25.332029+0000","last_undegraded":"2024-01-11T17:56:25.332029+0000","last_fullsized":"2024-01-11T17:56:25.332029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341977+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:56:25.341977+0000","last_peered":"2024-01-11T17:56:25.341977+0000","last_clean":"2024-01-11T17:56:25.341977+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:56:25.341977+0000","last_undegraded":"2024-01-11T17:56:25.341977+0000","last_fullsized":"2024-01-11T17:56:25.341977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335030+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:56:25.335030+0000","last_peered":"2024-01-11T17:56:25.335030+0000","last_clean":"2024-01-11T17:56:25.335030+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:56:25.335030+0000","last_undegraded":"2024-01-11T17:56:25.335030+0000","last_fullsized":"2024-01-11T17:56:25.335030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355996+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:56:26.355996+0000","last_peered":"2024-01-11T17:56:26.355996+0000","last_clean":"2024-01-11T17:56:26.355996+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:56:26.355996+0000","last_undegraded":"2024-01-11T17:56:26.355996+0000","last_fullsized":"2024-01-11T17:56:26.355996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336409+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:56:25.336409+0000","last_peered":"2024-01-11T17:56:25.336409+0000","last_clean":"2024-01-11T17:56:25.336409+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:56:25.336409+0000","last_undegraded":"2024-01-11T17:56:25.336409+0000","last_fullsized":"2024-01-11T17:56:25.336409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336125+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:56:25.336125+0000","last_peered":"2024-01-11T17:56:25.336125+0000","last_clean":"2024-01-11T17:56:25.336125+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:56:25.336125+0000","last_undegraded":"2024-01-11T17:56:25.336125+0000","last_fullsized":"2024-01-11T17:56:25.336125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332015+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:56:25.332015+0000","last_peered":"2024-01-11T17:56:25.332015+0000","last_clean":"2024-01-11T17:56:25.332015+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:56:25.332015+0000","last_undegraded":"2024-01-11T17:56:25.332015+0000","last_fullsized":"2024-01-11T17:56:25.332015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336086+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:56:25.336086+0000","last_peered":"2024-01-11T17:56:25.336086+0000","last_clean":"2024-01-11T17:56:25.336086+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:56:25.336086+0000","last_undegraded":"2024-01-11T17:56:25.336086+0000","last_fullsized":"2024-01-11T17:56:25.336086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338479+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:56:25.338479+0000","last_peered":"2024-01-11T17:56:25.338479+0000","last_clean":"2024-01-11T17:56:25.338479+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:56:25.338479+0000","last_undegraded":"2024-01-11T17:56:25.338479+0000","last_fullsized":"2024-01-11T17:56:25.338479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341422+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:56:25.341422+0000","last_peered":"2024-01-11T17:56:25.341422+0000","last_clean":"2024-01-11T17:56:25.341422+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:56:25.341422+0000","last_undegraded":"2024-01-11T17:56:25.341422+0000","last_fullsized":"2024-01-11T17:56:25.341422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338107+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:56:25.338107+0000","last_peered":"2024-01-11T17:56:25.338107+0000","last_clean":"2024-01-11T17:56:25.338107+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:56:25.338107+0000","last_undegraded":"2024-01-11T17:56:25.338107+0000","last_fullsized":"2024-01-11T17:56:25.338107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339143+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:56:25.339143+0000","last_peered":"2024-01-11T17:56:25.339143+0000","last_clean":"2024-01-11T17:56:25.339143+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:56:25.339143+0000","last_undegraded":"2024-01-11T17:56:25.339143+0000","last_fullsized":"2024-01-11T17:56:25.339143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337858+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:56:25.337858+0000","last_peered":"2024-01-11T17:56:25.337858+0000","last_clean":"2024-01-11T17:56:25.337858+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:56:25.337858+0000","last_undegraded":"2024-01-11T17:56:25.337858+0000","last_fullsized":"2024-01-11T17:56:25.337858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332218+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:56:25.332218+0000","last_peered":"2024-01-11T17:56:25.332218+0000","last_clean":"2024-01-11T17:56:25.332218+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:56:25.332218+0000","last_undegraded":"2024-01-11T17:56:25.332218+0000","last_fullsized":"2024-01-11T17:56:25.332218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355928+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:56:26.355928+0000","last_peered":"2024-01-11T17:56:26.355928+0000","last_clean":"2024-01-11T17:56:26.355928+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:56:26.355928+0000","last_undegraded":"2024-01-11T17:56:26.355928+0000","last_fullsized":"2024-01-11T17:56:26.355928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7841,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":27,"num_read_kb":26,"num_write":129,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":42041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":469002,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.495}]},{"osd":1,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.622},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.85399999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.92500000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776147,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":1064207,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.63},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Jan 11 17:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78000000000000003}]}]},{"osd":9,"up_from":63,"seq":270582939670,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":473334,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.496}]},{"osd":1,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.629}]},{"osd":2,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.45600000000000002}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.86099999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168604,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":473254,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.62}]},{"osd":1,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94199999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.872,"5min":1.097,"15min":1.097},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78300000000000003,"15min":0.78700000000000003},"min":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94799999999999995,"5min":1,"15min":1},"last":0.998}]},{"osd":7,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93300000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.018},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.03}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.125},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0509999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.1419999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397535,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1060580,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.63400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.627,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84499999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":1.137,"15min":1.137},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.65100000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.748},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.004,"5min":1.04,"15min":1.04},"last":0.60299999999999998}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0960000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.145,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.69699999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":6,"up_from":44,"seq":188978561059,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":468510,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.45900000000000002},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0920000000000001,"5min":1.179,"15min":1.179},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.54200000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0720000000000001,"5min":1.161,"15min":1.161},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92000000000000004,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0349999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1040000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.58599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95699999999999996,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89500000000000002,"15min":0.89400000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0669999999999999,"5min":1.131,"15min":1.131},"last":0.71499999999999997}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0980000000000001,"5min":1.155,"15min":1.155},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:55:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.871}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":478860,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.03,"5min":1.077,"15min":1.077},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89400000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51600000000000001,"15min":0.48999999999999999},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.42899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 11 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98699999999999999,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.44400000000000001,"15min":0.40000000000000002},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.48199999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.69799999999999995,"15min":0.68000000000000005},"min":{"1min":0.61899999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.59399999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.67600000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.61499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.72499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.056,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 11 17:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.85599999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.83099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":469898,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.55500000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.62,"15min":0.58799999999999997},"min":{"1min":0.50900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.59899999999999998,"15min":0.56299999999999994},"min":{"1min":0.52100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 11 17:54:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.45500000000000002,"15min":0.441},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47099999999999997},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.70099999999999996,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.622}]},{"osd":5,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 11 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.433,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.746},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.68300000000000005,"15min":0.66300000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66400000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.76700000000000002}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.80400000000000005}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 11 17:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":0.90500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":473611,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60299999999999998,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:56:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.438,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.95599999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":468704,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.38900000000000001},"min":{"1min":0.34399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.375},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.39200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71499999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.57699999999999996,"15min":0.52100000000000002},"min":{"1min":0.53000000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.98599999999999999,"5min":1.052,"15min":1.052},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.59299999999999997,"15min":0.53800000000000003},"min":{"1min":0.57599999999999996,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.024,"5min":1.079,"15min":1.079},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.53400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98799999999999999,"5min":1.137,"15min":1.137},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.65300000000000002,"15min":0.59399999999999997},"min":{"1min":0.50700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.68700000000000006,"15min":0.628},"min":{"1min":0.57399999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 11 17:55:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1020000000000001,"5min":1.204,"15min":1.204},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.438}]},{"osd":7,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.65100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0580000000000001,"5min":1.379,"15min":1.379},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.499,"15min":0.499},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.73199999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953515,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":469454,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.374},"min":{"1min":0.36099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.33600000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.91700000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.60099999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1,"5min":1.034,"15min":1.034},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.61899999999999999,"15min":0.59099999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.57699999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89800000000000002,"5min":1.129,"15min":1.129},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.67400000000000004,"15min":0.65000000000000002},"min":{"1min":0.54400000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.69199999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.499,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73399999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93200000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.024,"5min":1.238,"15min":1.238},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 11 17:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.71799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789992,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1060264,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.35099999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.47799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2529999999999999,"5min":1.548,"15min":1.548},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.751},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.58899999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.61199999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.58499999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.56499999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93999999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.87}]},{"osd":9,"last update":"Thu Jan 11 17:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.79100000000000004}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:56:38.833 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T17:56:38.833 INFO:tasks.cephadm:Setting up client nodes... 2024-01-11T17:56:38.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-01-11T17:56:38.996 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 17:56:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: 2024-01-11T17:56:38.654+0000 7f7f477aa700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-01-11T17:56:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:39.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:39 smithi016 ceph-mon[43195]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:56:39.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:39 smithi016 ceph-mon[43195]: pgmap v263: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:39.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:39 smithi038 ceph-mon[43582]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:56:39.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:39 smithi038 ceph-mon[43582]: pgmap v263: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:39.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:39 smithi007 ceph-mon[33361]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:56:39.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:39 smithi007 ceph-mon[33361]: pgmap v263: 129 pgs: 129 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 -- 172.21.15.7:0/2321612600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc104a20 msgr2=0x7f46fc106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 --2- 172.21.15.7:0/2321612600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc104a20 0x7f46fc106e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009990 tx=0x7f46ec009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 -- 172.21.15.7:0/2321612600 shutdown_connections 2024-01-11T17:56:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 --2- 172.21.15.7:0/2321612600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46fc107350 0x7f46fc1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 --2- 172.21.15.7:0/2321612600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc1020f0 0x7f46fc1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 --2- 172.21.15.7:0/2321612600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc104a20 0x7f46fc106e10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 -- 172.21.15.7:0/2321612600 >> 172.21.15.7:0/2321612600 conn(0x7f46fc0fbb00 msgr2=0x7f46fc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.544+0000 7f470475b700 1 -- 172.21.15.7:0/2321612600 shutdown_connections 2024-01-11T17:56:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.545+0000 7f470475b700 1 -- 172.21.15.7:0/2321612600 wait complete. 2024-01-11T17:56:40.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.545+0000 7f470475b700 1 Processor -- start 2024-01-11T17:56:40.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 -- start start 2024-01-11T17:56:40.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1020f0 0x7f46fc197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 0x7f46fc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46fc107350 0x7f46fc19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc19f020 con 0x7f46fc1020f0 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc19f160 con 0x7f46fc104a20 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f470475b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc19f2f0 con 0x7f46fc107350 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f4701cf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 0x7f46fc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f47024f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1020f0 0x7f46fc197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f4701cf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 0x7f46fc1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37922/0 (socket says 172.21.15.7:37922) 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.546+0000 7f4701cf6700 1 -- 172.21.15.7:0/217791072 learned_addr learned my addr 172.21.15.7:0/217791072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4702cf8700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46fc107350 0x7f46fc19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4701cf6700 1 -- 172.21.15.7:0/217791072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46fc107350 msgr2=0x7f46fc19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4701cf6700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46fc107350 0x7f46fc19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4701cf6700 1 -- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1020f0 msgr2=0x7f46fc197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:40.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4701cf6700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1020f0 0x7f46fc197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4701cf6700 1 -- 172.21.15.7:0/217791072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0095f0 con 0x7f46fc104a20 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f47024f7700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1020f0 0x7f46fc197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f4701cf6700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 0x7f46fc1979d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b680 tx=0x7f46ec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01d070 con 0x7f46fc104a20 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.547+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ec022470 con 0x7f46fc104a20 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.548+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc19f570 con 0x7f46fc104a20 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.548+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec017650 con 0x7f46fc104a20 2024-01-11T17:56:40.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.548+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc19f9e0 con 0x7f46fc104a20 2024-01-11T17:56:40.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.549+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc0fd680 con 0x7f46fc104a20 2024-01-11T17:56:40.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.551+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f46ec0225e0 con 0x7f46fc104a20 2024-01-11T17:56:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.552+0000 7f46f37fe700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46e80a20a0 0x7f46e80a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.552+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f46ec0cf190 con 0x7f46fc104a20 2024-01-11T17:56:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.552+0000 7f47024f7700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46e80a20a0 0x7f46e80a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:40.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.553+0000 7f47024f7700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46e80a20a0 0x7f46e80a4560 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46f8011490 tx=0x7f46f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:40.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.555+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f46ec097e70 con 0x7f46fc104a20 2024-01-11T17:56:40.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:40 smithi007 ceph-mon[33361]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:56:40.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:40 smithi016 ceph-mon[43195]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:56:40.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:40 smithi038 ceph-mon[43582]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:56:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.853+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f46fc10cfc0 con 0x7f46fc104a20 2024-01-11T17:56:40.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.859+0000 7f46f37fe700 1 -- 172.21.15.7:0/217791072 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f46ec09b490 con 0x7f46fc104a20 2024-01-11T17:56:40.860 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-01-11T17:56:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDYK6BlX3LzMhAACEgm8sTRZHf9muD7sLpErw== 2024-01-11T17:56:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46e80a20a0 msgr2=0x7f46e80a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46e80a20a0 0x7f46e80a4560 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46f8011490 tx=0x7f46f8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 msgr2=0x7f46fc1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 0x7f46fc1979d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b680 tx=0x7f46ec009580 comp rx=0 tx=0).stop 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 shutdown_connections 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46e80a20a0 0x7f46e80a4560 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46fc107350 0x7f46fc19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc1020f0 0x7f46fc197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 --2- 172.21.15.7:0/217791072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46fc104a20 0x7f46fc1979d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 >> 172.21.15.7:0/217791072 conn(0x7f46fc0fbb00 msgr2=0x7f46fc102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 shutdown_connections 2024-01-11T17:56:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:40.862+0000 7f470475b700 1 -- 172.21.15.7:0/217791072 wait complete. 2024-01-11T17:56:41.215 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:56:41.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-01-11T17:56:41.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-11T17:56:41.318 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-11T17:56:41.318 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-01-11T17:56:41.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph mgr dump --format=json 2024-01-11T17:56:41.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:41 smithi007 ceph-mon[33361]: pgmap v264: 129 pgs: 129 active+clean; 626 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:41.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:41 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/217791072' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-11T17:56:41.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:41 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-11T17:56:41.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:41 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-11T17:56:41.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:41 smithi016 ceph-mon[43195]: pgmap v264: 129 pgs: 129 active+clean; 626 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:41.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:41 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/217791072' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-11T17:56:41.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:41 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-11T17:56:41.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:41 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-11T17:56:41.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:41 smithi038 ceph-mon[43582]: pgmap v264: 129 pgs: 129 active+clean; 626 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-01-11T17:56:41.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:41 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/217791072' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-11T17:56:41.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:41 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-11T17:56:41.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:41 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-11T17:56:41.784 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.007+0000 7ff931e69700 1 -- 172.21.15.7:0/403300876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c074250 msgr2=0x7ff92c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.007+0000 7ff931e69700 1 --2- 172.21.15.7:0/403300876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c074250 0x7ff92c10d870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff918009990 tx=0x7ff918009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 -- 172.21.15.7:0/403300876 shutdown_connections 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 --2- 172.21.15.7:0/403300876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff92c10ddb0 0x7ff92c1101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 --2- 172.21.15.7:0/403300876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c073930 0x7ff92c073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 --2- 172.21.15.7:0/403300876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c074250 0x7ff92c10d870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 -- 172.21.15.7:0/403300876 >> 172.21.15.7:0/403300876 conn(0x7ff92c0fcfe0 msgr2=0x7ff92c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 -- 172.21.15.7:0/403300876 shutdown_connections 2024-01-11T17:56:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.008+0000 7ff931e69700 1 -- 172.21.15.7:0/403300876 wait complete. 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.009+0000 7ff931e69700 1 Processor -- start 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.009+0000 7ff931e69700 1 -- start start 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.009+0000 7ff931e69700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 0x7ff92c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.009+0000 7ff931e69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c074250 0x7ff92c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff931e69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff92c10ddb0 0x7ff92c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff931e69700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c199fb0 con 0x7ff92c074250 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff931e69700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c19a0f0 con 0x7ff92c10ddb0 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff931e69700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c19a2b0 con 0x7ff92c073930 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 0x7ff92c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c074250 0x7ff92c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 0x7ff92c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38928/0 (socket says 172.21.15.7:38928) 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 -- 172.21.15.7:0/1183416485 learned_addr learned my addr 172.21.15.7:0/1183416485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff931668700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff92c10ddb0 0x7ff92c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 -- 172.21.15.7:0/1183416485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff92c10ddb0 msgr2=0x7ff92c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff92c10ddb0 0x7ff92c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 -- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c074250 msgr2=0x7ff92c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c074250 0x7ff92c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff930e67700 1 -- 172.21.15.7:0/1183416485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920009520 con 0x7ff92c073930 2024-01-11T17:56:43.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.010+0000 7ff92bfff700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c074250 0x7ff92c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.011+0000 7ff930e67700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 0x7ff92c19f790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff92000eb30 tx=0x7ff92000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.011+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920009880 con 0x7ff92c073930 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.011+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180095f0 con 0x7ff92c073930 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.011+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9200044e0 con 0x7ff92c073930 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.011+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920003d80 con 0x7ff92c073930 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.011+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c1a6660 con 0x7ff92c073930 2024-01-11T17:56:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.012+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c108fc0 con 0x7ff92c073930 2024-01-11T17:56:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.014+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7ff920003f20 con 0x7ff92c073930 2024-01-11T17:56:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.015+0000 7ff929ffb700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff91c0a1f70 0x7ff91c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.016+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff920014070 con 0x7ff92c073930 2024-01-11T17:56:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.016+0000 7ff92bfff700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff91c0a1f70 0x7ff91c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:43.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.016+0000 7ff92bfff700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff91c0a1f70 0x7ff91c0a4430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b2e0 tx=0x7ff918005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.020+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff920096fa0 con 0x7ff92c073930 2024-01-11T17:56:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.289+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff92c0682d0 con 0x7ff92c073930 2024-01-11T17:56:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.295+0000 7ff929ffb700 1 -- 172.21.15.7:0/1183416485 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261339 (secure 0 0 0) 0x7ff9200041d0 con 0x7ff92c073930 2024-01-11T17:56:43.299 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff91c0a1f70 msgr2=0x7ff91c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff91c0a1f70 0x7ff91c0a4430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b2e0 tx=0x7ff918005960 comp rx=0 tx=0).stop 2024-01-11T17:56:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 msgr2=0x7ff92c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 0x7ff92c19f790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff92000eb30 tx=0x7ff92000ee40 comp rx=0 tx=0).stop 2024-01-11T17:56:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 shutdown_connections 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff91c0a1f70 0x7ff91c0a4430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff92c073930 0x7ff92c19f790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff92c10ddb0 0x7ff92c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 --2- 172.21.15.7:0/1183416485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c074250 0x7ff92c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 >> 172.21.15.7:0/1183416485 conn(0x7ff92c0fcfe0 msgr2=0x7ff92c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 shutdown_connections 2024-01-11T17:56:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:43.299+0000 7ff931e69700 1 -- 172.21.15.7:0/1183416485 wait complete. 2024-01-11T17:56:43.713 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":314922909},{"type":"v1","addr":"172.21.15.7:6801","nonce":314922909}]},"active_addr":"172.21.15.7:6801/314922909","active_change":"2024-01-11T17:50:24.497439+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2542526837}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2039092284}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2434319049}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":206690448}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3849322626}]}]} 2024-01-11T17:56:43.717 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-01-11T17:56:43.718 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-01-11T17:56:43.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd dump --format=json 2024-01-11T17:56:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:43 smithi016 ceph-mon[43195]: pgmap v265: 129 pgs: 129 active+clean; 626 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-01-11T17:56:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:43 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1183416485' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-11T17:56:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:43 smithi038 ceph-mon[43582]: pgmap v265: 129 pgs: 129 active+clean; 626 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-01-11T17:56:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:43 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1183416485' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-11T17:56:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:43 smithi007 ceph-mon[33361]: pgmap v265: 129 pgs: 129 active+clean; 626 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-01-11T17:56:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:43 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1183416485' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-11T17:56:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 -- 172.21.15.7:0/3512387265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0102020 msgr2=0x7f4af010e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 --2- 172.21.15.7:0/3512387265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0102020 0x7f4af010e520 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800b0a0 tx=0x7f4ae800b3b0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 -- 172.21.15.7:0/3512387265 shutdown_connections 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 --2- 172.21.15.7:0/3512387265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0107650 0x7f4af0107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 --2- 172.21.15.7:0/3512387265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0102020 0x7f4af010e520 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 --2- 172.21.15.7:0/3512387265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 -- 172.21.15.7:0/3512387265 >> 172.21.15.7:0/3512387265 conn(0x7f4af0078770 msgr2=0x7f4af0078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.531+0000 7f4af7553700 1 -- 172.21.15.7:0/3512387265 shutdown_connections 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.532+0000 7f4af7553700 1 -- 172.21.15.7:0/3512387265 wait complete. 2024-01-11T17:56:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.533+0000 7f4af7553700 1 Processor -- start 2024-01-11T17:56:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.533+0000 7f4af7553700 1 -- start start 2024-01-11T17:56:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af7553700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af7553700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 0x7f4af0073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af7553700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0107650 0x7f4af0074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af7553700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a6020 con 0x7f4af0107650 2024-01-11T17:56:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af7553700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a6190 con 0x7f4af0101660 2024-01-11T17:56:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af7553700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a6350 con 0x7f4af0102020 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af5af0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0107650 0x7f4af0074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af4aee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 0x7f4af0073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af52ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af4aee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 0x7f4af0073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38950/0 (socket says 172.21.15.7:38950) 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af4aee700 1 -- 172.21.15.7:0/1572812750 learned_addr learned my addr 172.21.15.7:0/1572812750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.534+0000 7f4af52ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57788/0 (socket says 172.21.15.7:57788) 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af52ef700 1 -- 172.21.15.7:0/1572812750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 msgr2=0x7f4af0073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af52ef700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 0x7f4af0073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af52ef700 1 -- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0107650 msgr2=0x7f4af0074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af52ef700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0107650 0x7f4af0074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af52ef700 1 -- 172.21.15.7:0/1572812750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec009520 con 0x7f4af0101660 2024-01-11T17:56:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af4aee700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 0x7f4af0073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af5af0700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0107650 0x7f4af0074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af52ef700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0073910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00e9f0 tx=0x7f4aec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00cb70 con 0x7f4af0101660 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4aec00ee90 con 0x7f4af0101660 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae8009d00 con 0x7f4af0101660 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec003cf0 con 0x7f4af0101660 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.535+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01a69a0 con 0x7f4af0101660 2024-01-11T17:56:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.536+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af0109c50 con 0x7f4af0101660 2024-01-11T17:56:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.538+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f4aec00ccd0 con 0x7f4af0101660 2024-01-11T17:56:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.540+0000 7f4ae67fc700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4adc0a1f70 0x7f4adc0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.540+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4aec014070 con 0x7f4af0101660 2024-01-11T17:56:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.540+0000 7f4af4aee700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4adc0a1f70 0x7f4adc0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.541+0000 7f4af4aee700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4adc0a1f70 0x7f4adc0a4430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4af0078c70 tx=0x7f4ae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.544+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f4aec096cd0 con 0x7f4af0101660 2024-01-11T17:56:45.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:45 smithi007 ceph-mon[33361]: pgmap v266: 129 pgs: 129 active+clean; 626 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-01-11T17:56:45.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:45 smithi016 ceph-mon[43195]: pgmap v266: 129 pgs: 129 active+clean; 626 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-01-11T17:56:45.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.759+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4af00682d0 con 0x7f4af0101660 2024-01-11T17:56:45.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.761+0000 7f4ae67fc700 1 -- 172.21.15.7:0/1572812750 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7f4aec09a2f0 con 0x7f4af0101660 2024-01-11T17:56:45.762 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:45.762 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:45.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4adc0a1f70 msgr2=0x7f4adc0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:45.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4adc0a1f70 0x7f4adc0a4430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4af0078c70 tx=0x7f4ae0009260 comp rx=0 tx=0).stop 2024-01-11T17:56:45.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 msgr2=0x7f4af0073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:45.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0073910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00e9f0 tx=0x7f4aec00ed00 comp rx=0 tx=0).stop 2024-01-11T17:56:45.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 shutdown_connections 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4adc0a1f70 0x7f4adc0a4430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af0107650 0x7f4af0074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4af0101660 0x7f4af0073910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 --2- 172.21.15.7:0/1572812750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af0102020 0x7f4af0073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 >> 172.21.15.7:0/1572812750 conn(0x7f4af0078770 msgr2=0x7f4af00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.763+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 shutdown_connections 2024-01-11T17:56:45.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:45.764+0000 7f4af7553700 1 -- 172.21.15.7:0/1572812750 wait complete. 2024-01-11T17:56:45.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:45 smithi038 ceph-mon[43582]: pgmap v266: 129 pgs: 129 active+clean; 626 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-01-11T17:56:46.146 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-01-11T17:56:46.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd dump --format=json 2024-01-11T17:56:46.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:46 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1572812750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:46.573 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:46.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:46 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1572812750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:46.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:46 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1572812750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:47.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:47 smithi016 ceph-mon[43195]: pgmap v267: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-01-11T17:56:47.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:47 smithi038 ceph-mon[43582]: pgmap v267: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 -- 172.21.15.7:0/2080314060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 msgr2=0x7f2e18101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 --2- 172.21.15.7:0/2080314060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 0x7f2e18101ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009990 tx=0x7f2e08009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 -- 172.21.15.7:0/2080314060 shutdown_connections 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 --2- 172.21.15.7:0/2080314060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18107650 0x7f2e18107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 --2- 172.21.15.7:0/2080314060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18102020 0x7f2e1810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 --2- 172.21.15.7:0/2080314060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 0x7f2e18101ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:47.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 -- 172.21.15.7:0/2080314060 >> 172.21.15.7:0/2080314060 conn(0x7f2e18078770 msgr2=0x7f2e18078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 -- 172.21.15.7:0/2080314060 shutdown_connections 2024-01-11T17:56:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.809+0000 7f2e20080700 1 -- 172.21.15.7:0/2080314060 wait complete. 2024-01-11T17:56:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.810+0000 7f2e20080700 1 Processor -- start 2024-01-11T17:56:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 -- start start 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 0x7f2e18199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 0x7f2e1819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107650 0x7f2e1819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1819ec00 con 0x7f2e18107650 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1819ed70 con 0x7f2e18101660 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.811+0000 7f2e20080700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1819ef30 con 0x7f2e18102020 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 0x7f2e1819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 0x7f2e1819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38978/0 (socket says 172.21.15.7:38978) 2024-01-11T17:56:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 -- 172.21.15.7:0/36731559 learned_addr learned my addr 172.21.15.7:0/36731559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1de1c700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 0x7f2e18199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1e61d700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107650 0x7f2e1819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 -- 172.21.15.7:0/36731559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 msgr2=0x7f2e18199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 0x7f2e18199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 -- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107650 msgr2=0x7f2e1819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107650 0x7f2e1819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1d61b700 1 -- 172.21.15.7:0/36731559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14009520 con 0x7f2e18102020 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.812+0000 7f2e1e61d700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107650 0x7f2e1819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.813+0000 7f2e1d61b700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 0x7f2e1819a260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e08000c00 tx=0x7f2e08005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.813+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0801d070 con 0x7f2e18102020 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.813+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e080095f0 con 0x7f2e18102020 2024-01-11T17:56:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.813+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e08022470 con 0x7f2e18102020 2024-01-11T17:56:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.813+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e08017650 con 0x7f2e18102020 2024-01-11T17:56:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.813+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181a68a0 con 0x7f2e18102020 2024-01-11T17:56:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.814+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e180682d0 con 0x7f2e18102020 2024-01-11T17:56:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.817+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f2e080225e0 con 0x7f2e18102020 2024-01-11T17:56:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.818+0000 7f2e0effd700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1ff0 0x7f2e040a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.818+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2e080cf210 con 0x7f2e18102020 2024-01-11T17:56:47.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.818+0000 7f2e1de1c700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1ff0 0x7f2e040a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:47.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.819+0000 7f2e1de1c700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1ff0 0x7f2e040a44b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009d70 tx=0x7f2e14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:47.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:47 smithi007 ceph-mon[33361]: pgmap v267: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-01-11T17:56:47.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:47.821+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2e08097e70 con 0x7f2e18102020 2024-01-11T17:56:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.034+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e18111690 con 0x7f2e18102020 2024-01-11T17:56:48.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.035+0000 7f2e0effd700 1 -- 172.21.15.7:0/36731559 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7f2e0809b490 con 0x7f2e18102020 2024-01-11T17:56:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:56:48.037 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:56:48.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1ff0 msgr2=0x7f2e040a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:48.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1ff0 0x7f2e040a44b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009d70 tx=0x7f2e14009260 comp rx=0 tx=0).stop 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 msgr2=0x7f2e1819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 0x7f2e1819a260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e08000c00 tx=0x7f2e08005740 comp rx=0 tx=0).stop 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 shutdown_connections 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1ff0 0x7f2e040a44b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e18107650 0x7f2e1819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e18101660 0x7f2e18199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 --2- 172.21.15.7:0/36731559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e18102020 0x7f2e1819a260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 >> 172.21.15.7:0/36731559 conn(0x7f2e18078770 msgr2=0x7f2e180ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 shutdown_connections 2024-01-11T17:56:48.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:48.038+0000 7f2e20080700 1 -- 172.21.15.7:0/36731559 wait complete. 2024-01-11T17:56:48.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-01-11T17:56:48.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-01-11T17:56:48.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-01-11T17:56:48.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-01-11T17:56:48.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-01-11T17:56:48.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-01-11T17:56:48.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-01-11T17:56:48.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-01-11T17:56:48.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-01-11T17:56:48.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-01-11T17:56:48.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-01-11T17:56:48.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-01-11T17:56:48.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:48 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/36731559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:48.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:48 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/36731559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:48 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/36731559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:56:49.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:49 smithi016 ceph-mon[43195]: pgmap v268: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-01-11T17:56:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:49 smithi038 ceph-mon[43582]: pgmap v268: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-01-11T17:56:49.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:49 smithi007 ceph-mon[33361]: pgmap v268: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-01-11T17:56:50.690 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:50.742 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:51.081 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:51 smithi007 ceph-mon[33361]: pgmap v269: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-01-11T17:56:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:51 smithi016 ceph-mon[43195]: pgmap v269: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-01-11T17:56:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:51 smithi038 ceph-mon[43582]: pgmap v269: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-01-11T17:56:52.161 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:53.323 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:53.920 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:53 smithi007 ceph-mon[33361]: pgmap v270: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-01-11T17:56:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:53 smithi016 ceph-mon[43195]: pgmap v270: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-01-11T17:56:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:53 smithi038 ceph-mon[43582]: pgmap v270: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-01-11T17:56:54.129 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:56:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/2919212355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa01075c0 msgr2=0x7ffaa01079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/2919212355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa01075c0 0x7ffaa01079a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa900099c0 tx=0x7ffa90009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/2919212355 shutdown_connections 2024-01-11T17:56:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/2919212355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaa0101ed0 0x7ffaa010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/2919212355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa01075c0 0x7ffaa01079a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/2919212355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaa0101510 0x7ffaa0101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/2919212355 >> 172.21.15.7:0/2919212355 conn(0x7ffaa00fcfe0 msgr2=0x7ffaa00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/2919212355 shutdown_connections 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.697+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/2919212355 wait complete. 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.698+0000 7ffaa4e3a700 1 Processor -- start 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 -- start start 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaa0101510 0x7ffaa019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa0101ed0 0x7ffaa019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaa01075c0 0x7ffaa0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0197ed0 con 0x7ffaa0101ed0 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0198040 con 0x7ffaa01075c0 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffaa4e3a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0198200 con 0x7ffaa0101510 2024-01-11T17:56:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffa9effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa0101ed0 0x7ffaa019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffa9effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa0101ed0 0x7ffaa019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34596/0 (socket says 172.21.15.7:34596) 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffa9effd700 1 -- 172.21.15.7:0/3171881039 learned_addr learned my addr 172.21.15.7:0/3171881039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.699+0000 7ffa9f7fe700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaa0101510 0x7ffaa019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9ffff700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaa01075c0 0x7ffaa0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9effd700 1 -- 172.21.15.7:0/3171881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaa0101510 msgr2=0x7ffaa019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9effd700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaa0101510 0x7ffaa019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9effd700 1 -- 172.21.15.7:0/3171881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaa01075c0 msgr2=0x7ffaa0197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9effd700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaa01075c0 0x7ffaa0197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9effd700 1 -- 172.21.15.7:0/3171881039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900095f0 con 0x7ffaa0101ed0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9effd700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa0101ed0 0x7ffaa019db40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d950 tx=0x7ffa9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400f890 con 0x7ffaa0101ed0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa94010460 con 0x7ffaa0101ed0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400e610 con 0x7ffaa0101ed0 2024-01-11T17:56:54.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.700+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa01984e0 con 0x7ffaa0101ed0 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.701+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa01a45d0 con 0x7ffaa0101ed0 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.701+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffaa0102c30 con 0x7ffaa0101ed0 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.704+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7ffa940108d0 con 0x7ffaa0101ed0 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.705+0000 7ffa9cff9700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa880a1ff0 0x7ffa880a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.705+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa940ce100 con 0x7ffaa0101ed0 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.705+0000 7ffa9f7fe700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa880a1ff0 0x7ffa880a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.705+0000 7ffa9cff9700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] conn(0x7ffa880ad010 0x7ffa880af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.705+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 --> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa880afb80 con 0x7ffa880ad010 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.705+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa940ce470 con 0x7ffaa0101ed0 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.706+0000 7ffa9ffff700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] conn(0x7ffa880ad010 0x7ffa880af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.706+0000 7ffa9f7fe700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa880a1ff0 0x7ffa880a44b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009990 tx=0x7ffa90019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.706+0000 7ffa9ffff700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] conn(0x7ffa880ad010 0x7ffa880af4b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.707+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== osd.9 v2:172.21.15.7:6826/1883138205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffa880afb80 con 0x7ffa880ad010 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.725+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 --> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffaa0111790 con 0x7ffa880ad010 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.725+0000 7ffa9cff9700 1 -- 172.21.15.7:0/3171881039 <== osd.9 v2:172.21.15.7:6826/1883138205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffaa0111790 con 0x7ffa880ad010 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.725+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] conn(0x7ffa880ad010 msgr2=0x7ffa880af4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.725+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] conn(0x7ffa880ad010 0x7ffa880af4b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa880a1ff0 msgr2=0x7ffa880a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ffa880a1ff0 0x7ffa880a44b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009990 tx=0x7ffa90019040 comp rx=0 tx=0).stop 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa0101ed0 msgr2=0x7ffaa019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 --2- 172.21.15.7:0/3171881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa0101ed0 0x7ffaa019db40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d950 tx=0x7ffa9400dc60 comp rx=0 tx=0).stop 2024-01-11T17:56:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffa9ffff700 1 -- 172.21.15.7:0/3171881039 reap_dead start 2024-01-11T17:56:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 shutdown_connections 2024-01-11T17:56:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 >> 172.21.15.7:0/3171881039 conn(0x7ffaa00fcfe0 msgr2=0x7ffaa00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 shutdown_connections 2024-01-11T17:56:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:53.726+0000 7ffaa4e3a700 1 -- 172.21.15.7:0/3171881039 wait complete. 2024-01-11T17:56:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 -- 172.21.15.7:0/1474900896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc107650 msgr2=0x7fe3fc107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 --2- 172.21.15.7:0/1474900896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc107650 0x7fe3fc107a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009990 tx=0x7fe3f8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 -- 172.21.15.7:0/1474900896 shutdown_connections 2024-01-11T17:56:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 --2- 172.21.15.7:0/1474900896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc107650 0x7fe3fc107a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 --2- 172.21.15.7:0/1474900896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102020 0x7fe3fc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 --2- 172.21.15.7:0/1474900896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3fc101660 0x7fe3fc101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 -- 172.21.15.7:0/1474900896 >> 172.21.15.7:0/1474900896 conn(0x7fe3fc078770 msgr2=0x7fe3fc078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 -- 172.21.15.7:0/1474900896 shutdown_connections 2024-01-11T17:56:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.069+0000 7fe4039aa700 1 -- 172.21.15.7:0/1474900896 wait complete. 2024-01-11T17:56:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.070+0000 7fe4039aa700 1 Processor -- start 2024-01-11T17:56:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.070+0000 7fe4039aa700 1 -- start start 2024-01-11T17:56:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe4039aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc101660 0x7fe3fc19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe4039aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102020 0x7fe3fc19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe4039aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3fc107650 0x7fe3fc199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe4039aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc19a2e0 con 0x7fe3fc102020 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe4039aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc19a450 con 0x7fe3fc107650 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe4039aa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc19a610 con 0x7fe3fc101660 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe401746700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc101660 0x7fe3fc19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe401746700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc101660 0x7fe3fc19fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:36984/0 (socket says 172.21.15.7:36984) 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe401746700 1 -- 172.21.15.7:0/772904673 learned_addr learned my addr 172.21.15.7:0/772904673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe401f47700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3fc107650 0x7fe3fc199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.071+0000 7fe400f45700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102020 0x7fe3fc19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe401746700 1 -- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3fc107650 msgr2=0x7fe3fc199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe401746700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3fc107650 0x7fe3fc199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe401746700 1 -- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102020 msgr2=0x7fe3fc19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe401746700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102020 0x7fe3fc19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe401746700 1 -- 172.21.15.7:0/772904673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80095f0 con 0x7fe3fc101660 2024-01-11T17:56:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe400f45700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc102020 0x7fe3fc19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe401746700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc101660 0x7fe3fc19fa60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005910 tx=0x7fe3f8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f801d070 con 0x7fe3fc101660 2024-01-11T17:56:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3f8022470 con 0x7fe3fc101660 2024-01-11T17:56:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc19a890 con 0x7fe3fc101660 2024-01-11T17:56:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.072+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8017650 con 0x7fe3fc101660 2024-01-11T17:56:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.073+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc1a69e0 con 0x7fe3fc101660 2024-01-11T17:56:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.073+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3fc111910 con 0x7fe3fc101660 2024-01-11T17:56:55.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.077+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fe3f80225e0 con 0x7fe3fc101660 2024-01-11T17:56:55.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.079+0000 7fe3f27fc700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe3e80a1f70 0x7fe3e80a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.079+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe3f80cf8b0 con 0x7fe3fc101660 2024-01-11T17:56:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.079+0000 7fe400f45700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe3e80a1f70 0x7fe3e80a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.079+0000 7fe3f27fc700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] conn(0x7fe3e80acf90 0x7fe3e80af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.079+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 --> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3e80afb00 con 0x7fe3e80acf90 2024-01-11T17:56:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.079+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f80cfc20 con 0x7fe3fc101660 2024-01-11T17:56:55.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.080+0000 7fe401f47700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] conn(0x7fe3e80acf90 0x7fe3e80af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:55.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.080+0000 7fe400f45700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe3e80a1f70 0x7fe3e80a4430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc19b280 tx=0x7fe3ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:55.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.080+0000 7fe401f47700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] conn(0x7fe3e80acf90 0x7fe3e80af430 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:55.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.081+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== osd.7 v2:172.21.15.16:6816/760747298 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe3e80afb00 con 0x7fe3e80acf90 2024-01-11T17:56:55.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.100+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 --> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe3fc0ffeb0 con 0x7fe3e80acf90 2024-01-11T17:56:55.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe3f27fc700 1 -- 172.21.15.7:0/772904673 <== osd.7 v2:172.21.15.16:6816/760747298 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe3fc0ffeb0 con 0x7fe3e80acf90 2024-01-11T17:56:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] conn(0x7fe3e80acf90 msgr2=0x7fe3e80af430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] conn(0x7fe3e80acf90 0x7fe3e80af430 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe3e80a1f70 msgr2=0x7fe3e80a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe3e80a1f70 0x7fe3e80a4430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc19b280 tx=0x7fe3ec009260 comp rx=0 tx=0).stop 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc101660 msgr2=0x7fe3fc19fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 --2- 172.21.15.7:0/772904673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3fc101660 0x7fe3fc19fa60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005910 tx=0x7fe3f8004730 comp rx=0 tx=0).stop 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe401f47700 1 -- 172.21.15.7:0/772904673 reap_dead start 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 shutdown_connections 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.101+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 >> 172.21.15.7:0/772904673 conn(0x7fe3fc078770 msgr2=0x7fe3fc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.102+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 shutdown_connections 2024-01-11T17:56:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.102+0000 7fe4039aa700 1 -- 172.21.15.7:0/772904673 wait complete. 2024-01-11T17:56:55.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:55 smithi007 ceph-mon[33361]: pgmap v271: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-01-11T17:56:55.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:55 smithi016 ceph-mon[43195]: pgmap v271: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-01-11T17:56:55.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:55 smithi038 ceph-mon[43582]: pgmap v271: 129 pgs: 129 active+clean; 638 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-01-11T17:56:56.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 -- 172.21.15.7:0/3145865000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 msgr2=0x7f3648073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:56.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 --2- 172.21.15.7:0/3145865000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648073e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3638009990 tx=0x7f3638009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 -- 172.21.15.7:0/3145865000 shutdown_connections 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 --2- 172.21.15.7:0/3145865000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f364810dd30 0x7f36481101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 --2- 172.21.15.7:0/3145865000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648073e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 --2- 172.21.15.7:0/3145865000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3648074360 0x7f364810d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 -- 172.21.15.7:0/3145865000 >> 172.21.15.7:0/3145865000 conn(0x7f36480fcfe0 msgr2=0x7f36480ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 -- 172.21.15.7:0/3145865000 shutdown_connections 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.047+0000 7f364ca36700 1 -- 172.21.15.7:0/3145865000 wait complete. 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.049+0000 7f364ca36700 1 Processor -- start 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.049+0000 7f364ca36700 1 -- start start 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364ca36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364ca36700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3648074360 0x7f364819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364ca36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364810dd30 0x7f364819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364ca36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364819eb40 con 0x7f3648073a40 2024-01-11T17:56:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364ca36700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364819ecb0 con 0x7f364810dd30 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364ca36700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364819ee70 con 0x7f3648074360 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34608/0 (socket says 172.21.15.7:34608) 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 -- 172.21.15.7:0/2853154718 learned_addr learned my addr 172.21.15.7:0/2853154718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364659c700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3648074360 0x7f364819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364759e700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364810dd30 0x7f364819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 -- 172.21.15.7:0/2853154718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3648074360 msgr2=0x7f364819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3648074360 0x7f364819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 -- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364810dd30 msgr2=0x7f364819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364810dd30 0x7f364819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f3646d9d700 1 -- 172.21.15.7:0/2853154718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009520 con 0x7f3648073a40 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.050+0000 7f364759e700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364810dd30 0x7f364819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.051+0000 7f3646d9d700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648199c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3638005910 tx=0x7f3638004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.051+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363801d070 con 0x7f3648073a40 2024-01-11T17:56:56.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.051+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3638022470 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.051+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36380095f0 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.051+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638017650 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.051+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36481a6820 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.052+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3648113970 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.054+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f36380225e0 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f362ffff700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36300a1f70 0x7f36300a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f36380cf8b0 con 0x7f3648073a40 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f364659c700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36300a1f70 0x7f36300a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f362ffff700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] conn(0x7f36300acf90 0x7f36300af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 --> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36300afb00 con 0x7f36300acf90 2024-01-11T17:56:56.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36380cfc20 con 0x7f3648073a40 2024-01-11T17:56:56.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.056+0000 7f364759e700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] conn(0x7f36300acf90 0x7f36300af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:56.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.057+0000 7f364659c700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36300a1f70 0x7f36300a4430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3648100a60 tx=0x7f363c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:56.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.057+0000 7f364759e700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] conn(0x7f36300acf90 0x7f36300af430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:56.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.058+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== osd.1 v2:172.21.15.16:6800/2714939694 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f36300afb00 con 0x7f36300acf90 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.082+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 --> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3648061ca0 con 0x7f36300acf90 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f362ffff700 1 -- 172.21.15.7:0/2853154718 <== osd.1 v2:172.21.15.16:6800/2714939694 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3648061ca0 con 0x7f36300acf90 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] conn(0x7f36300acf90 msgr2=0x7f36300af430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364ca36700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] conn(0x7f36300acf90 0x7f36300af430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36300a1f70 msgr2=0x7f36300a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364ca36700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f36300a1f70 0x7f36300a4430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3648100a60 tx=0x7f363c009260 comp rx=0 tx=0).stop 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 msgr2=0x7f3648199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364ca36700 1 --2- 172.21.15.7:0/2853154718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3648073a40 0x7f3648199c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3638005910 tx=0x7f3638004730 comp rx=0 tx=0).stop 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.083+0000 7f364759e700 1 -- 172.21.15.7:0/2853154718 reap_dead start 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.084+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 shutdown_connections 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.084+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 >> 172.21.15.7:0/2853154718 conn(0x7f36480fcfe0 msgr2=0x7f36480fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.084+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 shutdown_connections 2024-01-11T17:56:56.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:55.084+0000 7f364ca36700 1 -- 172.21.15.7:0/2853154718 wait complete. 2024-01-11T17:56:56.620 INFO:teuthology.orchestra.run.smithi007.stdout:55834574908 2024-01-11T17:56:56.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-01-11T17:56:57.244 INFO:teuthology.orchestra.run.smithi007.stdout:210453397540 2024-01-11T17:56:57.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 -- 172.21.15.7:0/2409291895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660106520 msgr2=0x7f4660106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 --2- 172.21.15.7:0/2409291895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660106520 0x7f4660106900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4650009990 tx=0x7f4650009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 -- 172.21.15.7:0/2409291895 shutdown_connections 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 --2- 172.21.15.7:0/2409291895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4660100ed0 0x7f466010d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 --2- 172.21.15.7:0/2409291895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660106520 0x7f4660106900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 --2- 172.21.15.7:0/2409291895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660100510 0x7f4660100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.455+0000 7f466611b700 1 -- 172.21.15.7:0/2409291895 >> 172.21.15.7:0/2409291895 conn(0x7f4660077440 msgr2=0x7f4660077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.456+0000 7f466611b700 1 -- 172.21.15.7:0/2409291895 shutdown_connections 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.456+0000 7f466611b700 1 -- 172.21.15.7:0/2409291895 wait complete. 2024-01-11T17:56:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.457+0000 7f466611b700 1 Processor -- start 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 -- start start 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660100510 0x7f4660102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660100ed0 0x7f4660102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4660106520 0x7f4660102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601a5d80 con 0x7f4660100ed0 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601a5ef0 con 0x7f4660100510 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.458+0000 7f466611b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601a60b0 con 0x7f4660106520 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660100ed0 0x7f4660102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660100510 0x7f4660102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660100ed0 0x7f4660102690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34646/0 (socket says 172.21.15.7:34646) 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465effd700 1 -- 172.21.15.7:0/2730951999 learned_addr learned my addr 172.21.15.7:0/2730951999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465ffff700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4660106520 0x7f4660102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465f7fe700 1 -- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4660106520 msgr2=0x7f4660102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465f7fe700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4660106520 0x7f4660102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465f7fe700 1 -- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660100ed0 msgr2=0x7f4660102690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465f7fe700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4660100ed0 0x7f4660102690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465f7fe700 1 -- 172.21.15.7:0/2730951999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500095f0 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.459+0000 7f465ffff700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4660106520 0x7f4660102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.460+0000 7f465f7fe700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660100510 0x7f4660102150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4650006010 tx=0x7f4650005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.460+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465001d070 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.460+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4650022470 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.460+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650017650 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.460+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46601a6330 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.460+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46601a66f0 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.461+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4660110710 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.464+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f4650022890 con 0x7f4660100510 2024-01-11T17:56:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.466+0000 7f465cff9700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46480a1ff0 0x7f46480a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.466+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f46500cfb40 con 0x7f4660100510 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.466+0000 7f465effd700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46480a1ff0 0x7f46480a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.467+0000 7f465cff9700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] conn(0x7f46480acf90 0x7f46480af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.467+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 --> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46480afb00 con 0x7f46480acf90 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.467+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46500cfeb0 con 0x7f4660100510 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.467+0000 7f465ffff700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] conn(0x7f46480acf90 0x7f46480af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.468+0000 7f465effd700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46480a1ff0 0x7f46480a44b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46600fec10 tx=0x7f4654009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.468+0000 7f465ffff700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] conn(0x7f46480acf90 0x7f46480af430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.469+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== osd.11 v2:172.21.15.38:6824/3838329833 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f46480afb00 con 0x7f46480acf90 2024-01-11T17:56:57.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.487+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 --> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46601044a0 con 0x7f46480acf90 2024-01-11T17:56:57.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f465cff9700 1 -- 172.21.15.7:0/2730951999 <== osd.11 v2:172.21.15.38:6824/3838329833 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f46601044a0 con 0x7f46480acf90 2024-01-11T17:56:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] conn(0x7f46480acf90 msgr2=0x7f46480af430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f466611b700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] conn(0x7f46480acf90 0x7f46480af430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46480a1ff0 msgr2=0x7f46480a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f466611b700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46480a1ff0 0x7f46480a44b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46600fec10 tx=0x7f4654009260 comp rx=0 tx=0).stop 2024-01-11T17:56:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660100510 msgr2=0x7f4660102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f466611b700 1 --2- 172.21.15.7:0/2730951999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660100510 0x7f4660102150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4650006010 tx=0x7f4650005740 comp rx=0 tx=0).stop 2024-01-11T17:56:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.488+0000 7f465ffff700 1 -- 172.21.15.7:0/2730951999 reap_dead start 2024-01-11T17:56:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.489+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 shutdown_connections 2024-01-11T17:56:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.489+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 >> 172.21.15.7:0/2730951999 conn(0x7f4660077440 msgr2=0x7f46600ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.489+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 shutdown_connections 2024-01-11T17:56:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.489+0000 7f466611b700 1 -- 172.21.15.7:0/2730951999 wait complete. 2024-01-11T17:56:57.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:57 smithi007 ceph-mon[33361]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-01-11T17:56:57.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:57 smithi016 ceph-mon[43195]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-01-11T17:56:58.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:57 smithi038 ceph-mon[43582]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-01-11T17:56:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.827+0000 7fb1ce66d700 1 -- 172.21.15.7:0/2215203557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c80692c0 msgr2=0x7fb1c8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.827+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/2215203557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c80692c0 0x7fb1c8069740 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0099c0 tx=0x7fb1bc009cd0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 -- 172.21.15.7:0/2215203557 shutdown_connections 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/2215203557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1c8107640 0x7fb1c8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/2215203557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c8069c80 0x7fb1c810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/2215203557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c80692c0 0x7fb1c8069740 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 -- 172.21.15.7:0/2215203557 >> 172.21.15.7:0/2215203557 conn(0x7fb1c8077440 msgr2=0x7fb1c8077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 -- 172.21.15.7:0/2215203557 shutdown_connections 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.828+0000 7fb1ce66d700 1 -- 172.21.15.7:0/2215203557 wait complete. 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 Processor -- start 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 -- start start 2024-01-11T17:56:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1c80692c0 0x7fb1c819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c8069c80 0x7fb1c819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c8107640 0x7fb1c8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819a080 con 0x7fb1c8069c80 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819a1c0 con 0x7fb1c8107640 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.829+0000 7fb1ce66d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819a380 con 0x7fb1c80692c0 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c8069c80 0x7fb1c819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c8069c80 0x7fb1c819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34660/0 (socket says 172.21.15.7:34660) 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 -- 172.21.15.7:0/1939946564 learned_addr learned my addr 172.21.15.7:0/1939946564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c7fff700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1c80692c0 0x7fb1c819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1ccc0a700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c8107640 0x7fb1c8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 -- 172.21.15.7:0/1939946564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1c80692c0 msgr2=0x7fb1c819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1c80692c0 0x7fb1c819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 -- 172.21.15.7:0/1939946564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c8107640 msgr2=0x7fb1c8199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c8107640 0x7fb1c8199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 -- 172.21.15.7:0/1939946564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0095f0 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c7fff700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1c80692c0 0x7fb1c819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1ccc0a700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1c8107640 0x7fb1c8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.830+0000 7fb1c77fe700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c8069c80 0x7fb1c819fda0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc005b40 tx=0x7fb1bc00ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.831+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc01d070 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.831+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1bc004bd0 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.831+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c819a600 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.831+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc00fc10 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.831+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c81a6900 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.832+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1c8111890 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.834+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb1bc004710 con 0x7fb1c8069c80 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.835+0000 7fb1c57fa700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1b00a1f70 0x7fb1b00a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.835+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb1bc0cf370 con 0x7fb1c8069c80 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.836+0000 7fb1c7fff700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1b00a1f70 0x7fb1b00a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.836+0000 7fb1c57fa700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] conn(0x7fb1b00acf10 0x7fb1b00af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.836+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 --> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1b00afa80 con 0x7fb1b00acf10 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.836+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1bc0cf780 con 0x7fb1c8069c80 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.836+0000 7fb1ccc0a700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] conn(0x7fb1b00acf10 0x7fb1b00af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.836+0000 7fb1c7fff700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1b00a1f70 0x7fb1b00a4430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1c819ab60 tx=0x7fb1b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.837+0000 7fb1ccc0a700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] conn(0x7fb1b00acf10 0x7fb1b00af3b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.837+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== osd.6 v2:172.21.15.7:6818/1282115079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb1b00afa80 con 0x7fb1b00acf10 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.856+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 --> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb1c810d1d0 con 0x7fb1b00acf10 2024-01-11T17:56:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.856+0000 7fb1c57fa700 1 -- 172.21.15.7:0/1939946564 <== osd.6 v2:172.21.15.7:6818/1282115079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb1c810d1d0 con 0x7fb1b00acf10 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.856+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] conn(0x7fb1b00acf10 msgr2=0x7fb1b00af3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] conn(0x7fb1b00acf10 0x7fb1b00af3b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1b00a1f70 msgr2=0x7fb1b00a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1b00a1f70 0x7fb1b00a4430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1c819ab60 tx=0x7fb1b8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c8069c80 msgr2=0x7fb1c819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 --2- 172.21.15.7:0/1939946564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1c8069c80 0x7fb1c819fda0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc005b40 tx=0x7fb1bc00ba30 comp rx=0 tx=0).stop 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ccc0a700 1 -- 172.21.15.7:0/1939946564 reap_dead start 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 shutdown_connections 2024-01-11T17:56:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 >> 172.21.15.7:0/1939946564 conn(0x7fb1c8077440 msgr2=0x7fb1c810c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 shutdown_connections 2024-01-11T17:56:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:57.857+0000 7fb1ce66d700 1 -- 172.21.15.7:0/1939946564 wait complete. 2024-01-11T17:56:58.871 INFO:teuthology.orchestra.run.smithi007.stdout:270582939674 2024-01-11T17:56:58.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.571+0000 7fc6004c3700 1 -- 172.21.15.7:0/649125486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 msgr2=0x7fc5f8073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.571+0000 7fc6004c3700 1 --2- 172.21.15.7:0/649125486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f8073d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009990 tx=0x7fc5f4009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.571+0000 7fc6004c3700 1 -- 172.21.15.7:0/649125486 shutdown_connections 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.572+0000 7fc6004c3700 1 --2- 172.21.15.7:0/649125486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f810def0 0x7fc5f81102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.572+0000 7fc6004c3700 1 --2- 172.21.15.7:0/649125486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f8073d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.572+0000 7fc6004c3700 1 --2- 172.21.15.7:0/649125486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8074250 0x7fc5f810d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.572+0000 7fc6004c3700 1 -- 172.21.15.7:0/649125486 >> 172.21.15.7:0/649125486 conn(0x7fc5f80fd0e0 msgr2=0x7fc5f80ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.572+0000 7fc6004c3700 1 -- 172.21.15.7:0/649125486 shutdown_connections 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.572+0000 7fc6004c3700 1 -- 172.21.15.7:0/649125486 wait complete. 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 Processor -- start 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 -- start start 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f8074250 0x7fc5f819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f810def0 0x7fc5f8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819a0f0 con 0x7fc5f8073930 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819a230 con 0x7fc5f810def0 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.573+0000 7fc6004c3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f819a3f0 con 0x7fc5f8074250 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34678/0 (socket says 172.21.15.7:34678) 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 -- 172.21.15.7:0/1417248376 learned_addr learned my addr 172.21.15.7:0/1417248376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fda5e700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f8074250 0x7fc5f819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fea60700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f810def0 0x7fc5f8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 -- 172.21.15.7:0/1417248376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f8074250 msgr2=0x7fc5f819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f8074250 0x7fc5f819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 -- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f810def0 msgr2=0x7fc5f8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f810def0 0x7fc5f8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 -- 172.21.15.7:0/1417248376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8009520 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fea60700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f810def0 0x7fc5f8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.574+0000 7fc5fe25f700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f819f8d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000c00 tx=0x7fc5f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.575+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f401d070 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.575+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f40095f0 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.575+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5f4022470 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.575+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4017650 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.575+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f81a67a0 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.576+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5f8113ac0 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.578+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc5f40225e0 con 0x7fc5f8073930 2024-01-11T17:56:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.579+0000 7fc5ef7fe700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5e40a1ff0 0x7fc5e40a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5f40cfaf0 con 0x7fc5f8073930 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5fda5e700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5e40a1ff0 0x7fc5e40a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5ef7fe700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] conn(0x7fc5e40ad010 0x7fc5e40af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 --> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5e40afb80 con 0x7fc5e40ad010 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f40cfe60 con 0x7fc5f8073930 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5fea60700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] conn(0x7fc5e40ad010 0x7fc5e40af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.580+0000 7fc5fda5e700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5e40a1ff0 0x7fc5e40a44b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5f810f1a0 tx=0x7fc5e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.581+0000 7fc5fea60700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] conn(0x7fc5e40ad010 0x7fc5e40af4b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.582+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== osd.10 v2:172.21.15.16:6824/2304237438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc5e40afb80 con 0x7fc5e40ad010 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 --> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc5f8061ca0 con 0x7fc5e40ad010 2024-01-11T17:56:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc5ef7fe700 1 -- 172.21.15.7:0/1417248376 <== osd.10 v2:172.21.15.16:6824/2304237438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc5f8061ca0 con 0x7fc5e40ad010 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] conn(0x7fc5e40ad010 msgr2=0x7fc5e40af4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] conn(0x7fc5e40ad010 0x7fc5e40af4b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5e40a1ff0 msgr2=0x7fc5e40a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5e40a1ff0 0x7fc5e40a44b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5f810f1a0 tx=0x7fc5e8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 msgr2=0x7fc5f819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 --2- 172.21.15.7:0/1417248376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8073930 0x7fc5f819f8d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000c00 tx=0x7fc5f4005740 comp rx=0 tx=0).stop 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc5fea60700 1 -- 172.21.15.7:0/1417248376 reap_dead start 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 shutdown_connections 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 >> 172.21.15.7:0/1417248376 conn(0x7fc5f80fd0e0 msgr2=0x7fc5f80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 shutdown_connections 2024-01-11T17:56:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.597+0000 7fc6004c3700 1 -- 172.21.15.7:0/1417248376 wait complete. 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.834+0000 7f50d87cb700 1 -- 172.21.15.7:0/4012745660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 msgr2=0x7f50d0103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.834+0000 7f50d87cb700 1 --2- 172.21.15.7:0/4012745660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d0103c60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009990 tx=0x7f50c0009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 -- 172.21.15.7:0/4012745660 shutdown_connections 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 --2- 172.21.15.7:0/4012745660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d01041a0 0x7f50d01106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 --2- 172.21.15.7:0/4012745660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d01097e0 0x7f50d0109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 --2- 172.21.15.7:0/4012745660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d0103c60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 -- 172.21.15.7:0/4012745660 >> 172.21.15.7:0/4012745660 conn(0x7f50d00ff2f0 msgr2=0x7f50d0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 -- 172.21.15.7:0/4012745660 shutdown_connections 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.835+0000 7f50d87cb700 1 -- 172.21.15.7:0/4012745660 wait complete. 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.836+0000 7f50d87cb700 1 Processor -- start 2024-01-11T17:56:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 -- start start 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d01041a0 0x7f50d019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d01097e0 0x7f50d0199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019a0c0 con 0x7f50d01041a0 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019a200 con 0x7f50d01097e0 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d87cb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019a3c0 con 0x7f50d01037e0 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d6567700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d5d66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d01041a0 0x7f50d019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d6567700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37062/0 (socket says 172.21.15.7:37062) 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.837+0000 7f50d6567700 1 -- 172.21.15.7:0/1555557943 learned_addr learned my addr 172.21.15.7:0/1555557943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6d68700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d01097e0 0x7f50d0199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6567700 1 -- 172.21.15.7:0/1555557943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d01097e0 msgr2=0x7f50d0199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6567700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d01097e0 0x7f50d0199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6567700 1 -- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d01041a0 msgr2=0x7f50d019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6567700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d01041a0 0x7f50d019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6567700 1 -- 172.21.15.7:0/1555557943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00095f0 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.838+0000 7f50d6567700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d019f8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e9a0 tx=0x7f50cc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.839+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc009790 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.839+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50cc010430 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.839+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc018680 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.839+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d019a6a0 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.839+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d01a6940 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.840+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50d0104f20 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.842+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f50cc0105a0 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.844+0000 7f50c77fe700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50bc0a1ff0 0x7f50bc0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.844+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50cc014070 con 0x7f50d01037e0 2024-01-11T17:56:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.844+0000 7f50d5d66700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50bc0a1ff0 0x7f50bc0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.845+0000 7f50c77fe700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] conn(0x7f50bc0ad010 0x7f50bc0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.845+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 --> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50bc0afb80 con 0x7f50bc0ad010 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.845+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50cc0ce5b0 con 0x7f50d01037e0 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.845+0000 7f50d5d66700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50bc0a1ff0 0x7f50bc0a44b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f50d0108e00 tx=0x7f50c000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.845+0000 7f50d6d68700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] conn(0x7f50bc0ad010 0x7f50bc0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.846+0000 7f50d6d68700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] conn(0x7f50bc0ad010 0x7f50bc0af4b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.847+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== osd.2 v2:172.21.15.38:6800/2733698254 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f50bc0afb80 con 0x7f50bc0ad010 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.865+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 --> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50d0113b30 con 0x7f50bc0ad010 2024-01-11T17:56:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.866+0000 7f50c77fe700 1 -- 172.21.15.7:0/1555557943 <== osd.2 v2:172.21.15.38:6800/2733698254 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f50d0113b30 con 0x7f50bc0ad010 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] conn(0x7f50bc0ad010 msgr2=0x7f50bc0af4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] conn(0x7f50bc0ad010 0x7f50bc0af4b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50bc0a1ff0 msgr2=0x7f50bc0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f50bc0a1ff0 0x7f50bc0a44b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f50d0108e00 tx=0x7f50c000b280 comp rx=0 tx=0).stop 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 msgr2=0x7f50d019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 --2- 172.21.15.7:0/1555557943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50d01037e0 0x7f50d019f8a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e9a0 tx=0x7f50cc00ecb0 comp rx=0 tx=0).stop 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d6d68700 1 -- 172.21.15.7:0/1555557943 reap_dead start 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 shutdown_connections 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 >> 172.21.15.7:0/1555557943 conn(0x7f50d00ff2f0 msgr2=0x7f50d00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 shutdown_connections 2024-01-11T17:56:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:58.867+0000 7f50d87cb700 1 -- 172.21.15.7:0/1555557943 wait complete. 2024-01-11T17:56:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 -- 172.21.15.7:0/3849981711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc101660 msgr2=0x7f96bc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 --2- 172.21.15.7:0/3849981711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc101660 0x7f96bc101ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009990 tx=0x7f96ac009ca0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 -- 172.21.15.7:0/3849981711 shutdown_connections 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 --2- 172.21.15.7:0/3849981711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc102020 0x7f96bc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 --2- 172.21.15.7:0/3849981711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc107650 0x7f96bc107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 --2- 172.21.15.7:0/3849981711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc101660 0x7f96bc101ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 -- 172.21.15.7:0/3849981711 >> 172.21.15.7:0/3849981711 conn(0x7f96bc078770 msgr2=0x7f96bc078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 -- 172.21.15.7:0/3849981711 shutdown_connections 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.033+0000 7f96c40c6700 1 -- 172.21.15.7:0/3849981711 wait complete. 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 Processor -- start 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 -- start start 2024-01-11T17:56:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc101660 0x7f96bc19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc102020 0x7f96bc19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc107650 0x7f96bc197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc197f80 con 0x7f96bc107650 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1980c0 con 0x7f96bc101660 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.035+0000 7f96c40c6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc198280 con 0x7f96bc102020 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c2663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc107650 0x7f96bc197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1e62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc101660 0x7f96bc19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1661700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc102020 0x7f96bc19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c2663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc107650 0x7f96bc197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34704/0 (socket says 172.21.15.7:34704) 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c2663700 1 -- 172.21.15.7:0/1302804867 learned_addr learned my addr 172.21.15.7:0/1302804867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1661700 1 -- 172.21.15.7:0/1302804867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc101660 msgr2=0x7f96bc19d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1661700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc101660 0x7f96bc19d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1661700 1 -- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc107650 msgr2=0x7f96bc197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1661700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc107650 0x7f96bc197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1661700 1 -- 172.21.15.7:0/1302804867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8009520 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c1e62700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc101660 0x7f96bc19d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.036+0000 7f96c2663700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc107650 0x7f96bc197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.037+0000 7f96c1661700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc102020 0x7f96bc19dca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96ac000c00 tx=0x7f96ac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.037+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac01d070 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.037+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96ac021410 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.037+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac0095f0 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.037+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac021ea0 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.037+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc1a4630 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.038+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96bc111910 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.040+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f96ac021580 con 0x7f96bc102020 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96b2ffd700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f96a80a1ff0 0x7f96a80a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f96ac0cf590 con 0x7f96bc102020 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96c1e62700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f96a80a1ff0 0x7f96a80a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96b2ffd700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] conn(0x7f96a80ad010 0x7f96a80af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 --> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96a80afb80 con 0x7f96a80ad010 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96ac0cf900 con 0x7f96bc102020 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.042+0000 7f96c2663700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] conn(0x7f96a80ad010 0x7f96a80af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.043+0000 7f96c1e62700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f96a80a1ff0 0x7f96a80a44b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96bc198a60 tx=0x7f96b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.043+0000 7f96c2663700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] conn(0x7f96a80ad010 0x7f96a80af4b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.044+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== osd.0 v2:172.21.15.7:6802/953692014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f96a80afb80 con 0x7f96a80ad010 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.062+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 --> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96bc061d20 con 0x7f96a80ad010 2024-01-11T17:56:59.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.062+0000 7f96b2ffd700 1 -- 172.21.15.7:0/1302804867 <== osd.0 v2:172.21.15.7:6802/953692014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96bc061d20 con 0x7f96a80ad010 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.063+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] conn(0x7f96a80ad010 msgr2=0x7f96a80af4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.063+0000 7f96c40c6700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] conn(0x7f96a80ad010 0x7f96a80af4b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.063+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f96a80a1ff0 msgr2=0x7f96a80a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.063+0000 7f96c40c6700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f96a80a1ff0 0x7f96a80a44b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96bc198a60 tx=0x7f96b8009260 comp rx=0 tx=0).stop 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.063+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc102020 msgr2=0x7f96bc19dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.063+0000 7f96c40c6700 1 --2- 172.21.15.7:0/1302804867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc102020 0x7f96bc19dca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96ac000c00 tx=0x7f96ac004730 comp rx=0 tx=0).stop 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.064+0000 7f96c2663700 1 -- 172.21.15.7:0/1302804867 reap_dead start 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.064+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 shutdown_connections 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.064+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 >> 172.21.15.7:0/1302804867 conn(0x7f96bc078770 msgr2=0x7f96bc0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.064+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 shutdown_connections 2024-01-11T17:56:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.064+0000 7f96c40c6700 1 -- 172.21.15.7:0/1302804867 wait complete. 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.641+0000 7f5de53f8700 1 -- 172.21.15.7:0/4159514059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 msgr2=0x7f5de010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.641+0000 7f5de53f8700 1 --2- 172.21.15.7:0/4159514059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 0x7f5de010e390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b0d0 tx=0x7f5dd800b3e0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 -- 172.21.15.7:0/4159514059 shutdown_connections 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 --2- 172.21.15.7:0/4159514059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5de01075c0 0x7f5de01079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 --2- 172.21.15.7:0/4159514059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 0x7f5de010e390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 --2- 172.21.15.7:0/4159514059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 0x7f5de0101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 -- 172.21.15.7:0/4159514059 >> 172.21.15.7:0/4159514059 conn(0x7f5de00fcfe0 msgr2=0x7f5de00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 -- 172.21.15.7:0/4159514059 shutdown_connections 2024-01-11T17:56:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.642+0000 7f5de53f8700 1 -- 172.21.15.7:0/4159514059 wait complete. 2024-01-11T17:56:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 Processor -- start 2024-01-11T17:56:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 -- start start 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 0x7f5de0073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 0x7f5de0073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5de01075c0 0x7f5de0074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01a5dd0 con 0x7f5de0101ed0 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01a5f40 con 0x7f5de0101510 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.643+0000 7f5de53f8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01a6100 con 0x7f5de01075c0 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 0x7f5de0073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 0x7f5de0073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 0x7f5de0073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41708/0 (socket says 172.21.15.7:41708) 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddffff700 1 -- 172.21.15.7:0/3560960792 learned_addr learned my addr 172.21.15.7:0/3560960792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5de4bf7700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5de01075c0 0x7f5de0074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 -- 172.21.15.7:0/3560960792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5de01075c0 msgr2=0x7f5de0074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5de01075c0 0x7f5de0074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 -- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 msgr2=0x7f5de0073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 0x7f5de0073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 -- 172.21.15.7:0/3560960792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8009d00 con 0x7f5de0101ed0 2024-01-11T17:56:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddffff700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de0101510 0x7f5de0073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5de4bf7700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5de01075c0 0x7f5de0074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.644+0000 7f5ddf7fe700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 0x7f5de0073e50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e9f0 tx=0x7f5dd400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.645+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400c9c0 con 0x7f5de0101ed0 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.645+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd4010430 con 0x7f5de0101ed0 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.645+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4018680 con 0x7f5de0101ed0 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.645+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de01a63e0 con 0x7f5de0101ed0 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.645+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de01a6930 con 0x7f5de0101ed0 2024-01-11T17:56:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.646+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5de0102c30 con 0x7f5de0101ed0 2024-01-11T17:56:59.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.648+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f5dd4010860 con 0x7f5de0101ed0 2024-01-11T17:56:59.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.649+0000 7f5ddd7fa700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dc80a1ff0 0x7f5dc80a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.649+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5dd4014070 con 0x7f5de0101ed0 2024-01-11T17:56:59.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.649+0000 7f5ddffff700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dc80a1ff0 0x7f5dc80a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.650+0000 7f5ddd7fa700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] conn(0x7f5dc80ad010 0x7f5dc80af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:56:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.650+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 --> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dc80afb80 con 0x7f5dc80ad010 2024-01-11T17:56:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.650+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dd40ce5b0 con 0x7f5de0101ed0 2024-01-11T17:56:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.650+0000 7f5ddffff700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dc80a1ff0 0x7f5dc80a44b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0011550 tx=0x7f5dd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.650+0000 7f5de4bf7700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] conn(0x7f5dc80ad010 0x7f5dc80af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:56:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.651+0000 7f5de4bf7700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] conn(0x7f5dc80ad010 0x7f5dc80af4b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:56:59.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.652+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== osd.4 v2:172.21.15.16:6808/1933820349 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5dc80afb80 con 0x7f5dc80ad010 2024-01-11T17:56:59.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.669+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 --> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5de0111790 con 0x7f5dc80ad010 2024-01-11T17:56:59.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5ddd7fa700 1 -- 172.21.15.7:0/3560960792 <== osd.4 v2:172.21.15.16:6808/1933820349 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5de0111790 con 0x7f5dc80ad010 2024-01-11T17:56:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] conn(0x7f5dc80ad010 msgr2=0x7f5dc80af4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5de53f8700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] conn(0x7f5dc80ad010 0x7f5dc80af4b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:56:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dc80a1ff0 msgr2=0x7f5dc80a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5de53f8700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dc80a1ff0 0x7f5dc80a44b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0011550 tx=0x7f5dd0009260 comp rx=0 tx=0).stop 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 msgr2=0x7f5de0073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.670+0000 7f5de53f8700 1 --2- 172.21.15.7:0/3560960792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de0101ed0 0x7f5de0073e50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e9f0 tx=0x7f5dd400ed00 comp rx=0 tx=0).stop 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.671+0000 7f5de4bf7700 1 -- 172.21.15.7:0/3560960792 reap_dead start 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.671+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 shutdown_connections 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.671+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 >> 172.21.15.7:0/3560960792 conn(0x7f5de00fcfe0 msgr2=0x7f5de00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.671+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 shutdown_connections 2024-01-11T17:56:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:56:59.671+0000 7f5de53f8700 1 -- 172.21.15.7:0/3560960792 wait complete. 2024-01-11T17:56:59.791 INFO:teuthology.orchestra.run.smithi007.stdout:322122547221 2024-01-11T17:56:59.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-01-11T17:56:59.877 INFO:teuthology.orchestra.run.smithi007.stdout:188978561064 2024-01-11T17:56:59.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-01-11T17:56:59.940 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:56:59 smithi007 ceph-mon[33361]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-11T17:56:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:56:59 smithi016 ceph-mon[43195]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-11T17:57:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:56:59 smithi038 ceph-mon[43582]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-11T17:57:00.165 INFO:teuthology.orchestra.run.smithi007.stdout:292057776152 2024-01-11T17:57:00.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-01-11T17:57:00.388 INFO:teuthology.orchestra.run.smithi007.stdout:34359738432 2024-01-11T17:57:00.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-01-11T17:57:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 -- 172.21.15.7:0/2596974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 msgr2=0x7ff660073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 --2- 172.21.15.7:0/2596974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff660073c20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff650009990 tx=0x7ff650009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 -- 172.21.15.7:0/2596974548 shutdown_connections 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 --2- 172.21.15.7:0/2596974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff660073c20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 --2- 172.21.15.7:0/2596974548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6601127e0 0x7ff660114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 --2- 172.21.15.7:0/2596974548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff660074160 0x7ff6601122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 -- 172.21.15.7:0/2596974548 >> 172.21.15.7:0/2596974548 conn(0x7ff660078770 msgr2=0x7ff660078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 -- 172.21.15.7:0/2596974548 shutdown_connections 2024-01-11T17:57:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.758+0000 7ff665844700 1 -- 172.21.15.7:0/2596974548 wait complete. 2024-01-11T17:57:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.759+0000 7ff665844700 1 Processor -- start 2024-01-11T17:57:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.759+0000 7ff665844700 1 -- start start 2024-01-11T17:57:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.759+0000 7ff665844700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff66019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.759+0000 7ff665844700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff660074160 0x7ff66019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff665844700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6601127e0 0x7ff6601a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff665844700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6601a3040 con 0x7ff6601127e0 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff665844700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6601a31b0 con 0x7ff660074160 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff665844700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6601a3370 con 0x7ff660073840 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff66019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff66019e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37134/0 (socket says 172.21.15.7:37134) 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 -- 172.21.15.7:0/1078342544 learned_addr learned my addr 172.21.15.7:0/1078342544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65f7fe700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6601127e0 0x7ff6601a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65e7fc700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff660074160 0x7ff66019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 -- 172.21.15.7:0/1078342544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff660074160 msgr2=0x7ff66019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff660074160 0x7ff66019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 -- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6601127e0 msgr2=0x7ff6601a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6601127e0 0x7ff6601a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.760+0000 7ff65effd700 1 -- 172.21.15.7:0/1078342544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6500095f0 con 0x7ff660073840 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.761+0000 7ff65effd700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff66019e160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff650000c00 tx=0x7ff650005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.761+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65001d070 con 0x7ff660073840 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.761+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff650022470 con 0x7ff660073840 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.761+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6601a35f0 con 0x7ff660073840 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.761+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650017650 con 0x7ff660073840 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.761+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6601aaeb0 con 0x7ff660073840 2024-01-11T17:57:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.762+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff660117ea0 con 0x7ff660073840 2024-01-11T17:57:00.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.764+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7ff6500225e0 con 0x7ff660073840 2024-01-11T17:57:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.766+0000 7ff664842700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6480a1f70 0x7ff6480a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.766+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6500cfb00 con 0x7ff660073840 2024-01-11T17:57:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.766+0000 7ff664842700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] conn(0x7ff6480acf10 0x7ff6480af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.766+0000 7ff65e7fc700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6480a1f70 0x7ff6480a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.766+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 --> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6480afa80 con 0x7ff6480acf10 2024-01-11T17:57:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.766+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6500cfe70 con 0x7ff660073840 2024-01-11T17:57:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.767+0000 7ff65f7fe700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] conn(0x7ff6480acf10 0x7ff6480af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.767+0000 7ff65f7fe700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] conn(0x7ff6480acf10 0x7ff6480af3b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.767+0000 7ff65e7fc700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6480a1f70 0x7ff6480a4430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff660111d50 tx=0x7ff654009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.768+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== osd.3 v2:172.21.15.7:6810/1132146736 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff6480afa80 con 0x7ff6480acf10 2024-01-11T17:57:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.792+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 --> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff66010ebe0 con 0x7ff6480acf10 2024-01-11T17:57:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.792+0000 7ff664842700 1 -- 172.21.15.7:0/1078342544 <== osd.3 v2:172.21.15.7:6810/1132146736 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff66010ebe0 con 0x7ff6480acf10 2024-01-11T17:57:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] conn(0x7ff6480acf10 msgr2=0x7ff6480af3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] conn(0x7ff6480acf10 0x7ff6480af3b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6480a1f70 msgr2=0x7ff6480a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6480a1f70 0x7ff6480a4430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff660111d50 tx=0x7ff654009260 comp rx=0 tx=0).stop 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 msgr2=0x7ff66019e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 --2- 172.21.15.7:0/1078342544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff660073840 0x7ff66019e160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff650000c00 tx=0x7ff650005740 comp rx=0 tx=0).stop 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff65f7fe700 1 -- 172.21.15.7:0/1078342544 reap_dead start 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 shutdown_connections 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 >> 172.21.15.7:0/1078342544 conn(0x7ff660078770 msgr2=0x7ff66010e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 shutdown_connections 2024-01-11T17:57:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.793+0000 7ff665844700 1 -- 172.21.15.7:0/1078342544 wait complete. 2024-01-11T17:57:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 -- 172.21.15.7:0/2379420106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e04069be0 msgr2=0x7f5e0410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 --2- 172.21.15.7:0/2379420106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e04069be0 0x7f5e0410db80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009990 tx=0x7f5df8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 -- 172.21.15.7:0/2379420106 shutdown_connections 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 --2- 172.21.15.7:0/2379420106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e0410e0c0 0x7f5e041104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 --2- 172.21.15.7:0/2379420106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e040692c0 0x7f5e040696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 --2- 172.21.15.7:0/2379420106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e04069be0 0x7f5e0410db80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 -- 172.21.15.7:0/2379420106 >> 172.21.15.7:0/2379420106 conn(0x7f5e04077440 msgr2=0x7f5e04077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.840+0000 7f5e0a661700 1 -- 172.21.15.7:0/2379420106 shutdown_connections 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.841+0000 7f5e0a661700 1 -- 172.21.15.7:0/2379420106 wait complete. 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 Processor -- start 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 -- start start 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e040692c0 0x7f5e0419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04069be0 0x7f5e0419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0410e0c0 0x7f5e041a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a30d0 con 0x7f5e0410e0c0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a3240 con 0x7f5e04069be0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e0a661700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a3400 con 0x7f5e040692c0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.842+0000 7f5e03fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e040692c0 0x7f5e0419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0410e0c0 0x7f5e041a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e03fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e040692c0 0x7f5e0419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37152/0 (socket says 172.21.15.7:37152) 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e03fff700 1 -- 172.21.15.7:0/639602624 learned_addr learned my addr 172.21.15.7:0/639602624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e037fe700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04069be0 0x7f5e0419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 -- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e040692c0 msgr2=0x7f5e0419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e040692c0 0x7f5e0419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 -- 172.21.15.7:0/639602624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04069be0 msgr2=0x7f5e0419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e04069be0 0x7f5e0419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 -- 172.21.15.7:0/639602624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80095f0 con 0x7f5e0410e0c0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.843+0000 7f5e08bfe700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0410e0c0 0x7f5e041a2ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00bdf0 tx=0x7f5dfc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.844+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00cb30 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.844+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc00cc90 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.844+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0125e0 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.844+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e041a36e0 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.844+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e041aaf40 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.845+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e04105310 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.847+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f5dfc014440 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.848+0000 7f5e017fa700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dec0a2030 0x7f5dec0a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5dfc006e40 con 0x7f5e0410e0c0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e03fff700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dec0a2030 0x7f5dec0a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e017fa700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] conn(0x7f5dec0ad050 0x7f5dec0af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 --> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dec0afbc0 con 0x7f5dec0ad050 2024-01-11T17:57:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc0cdeb0 con 0x7f5e0410e0c0 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e037fe700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] conn(0x7f5dec0ad050 0x7f5dec0af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.849+0000 7f5e03fff700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dec0a2030 0x7f5dec0a44f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ad0 tx=0x7f5df4009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.850+0000 7f5e037fe700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] conn(0x7f5dec0ad050 0x7f5dec0af4f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.851+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== osd.8 v2:172.21.15.38:6816/659578485 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5dec0afbc0 con 0x7f5dec0ad050 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.868+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 --> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e04061d20 con 0x7f5dec0ad050 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.869+0000 7f5e017fa700 1 -- 172.21.15.7:0/639602624 <== osd.8 v2:172.21.15.38:6816/659578485 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5e04061d20 con 0x7f5dec0ad050 2024-01-11T17:57:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.869+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] conn(0x7f5dec0ad050 msgr2=0x7f5dec0af4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.869+0000 7f5e0a661700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] conn(0x7f5dec0ad050 0x7f5dec0af4f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dec0a2030 msgr2=0x7f5dec0a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5dec0a2030 0x7f5dec0a44f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ad0 tx=0x7f5df4009290 comp rx=0 tx=0).stop 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0410e0c0 msgr2=0x7f5e041a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 --2- 172.21.15.7:0/639602624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e0410e0c0 0x7f5e041a2ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00bdf0 tx=0x7f5dfc009ed0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e08bfe700 1 -- 172.21.15.7:0/639602624 reap_dead start 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 shutdown_connections 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 >> 172.21.15.7:0/639602624 conn(0x7f5e04077440 msgr2=0x7f5e0410f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 shutdown_connections 2024-01-11T17:57:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.870+0000 7f5e0a661700 1 -- 172.21.15.7:0/639602624 wait complete. 2024-01-11T17:57:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 -- 172.21.15.7:0/1709853388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 msgr2=0x7f8168110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 --2- 172.21.15.7:0/1709853388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 0x7f8168110400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8160009f90 tx=0x7f816000b330 comp rx=0 tx=0).stop 2024-01-11T17:57:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 -- 172.21.15.7:0/1709853388 shutdown_connections 2024-01-11T17:57:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 --2- 172.21.15.7:0/1709853388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 0x7f8168110400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 --2- 172.21.15.7:0/1709853388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8168073840 0x7f8168073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 --2- 172.21.15.7:0/1709853388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168074160 0x7f816810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 -- 172.21.15.7:0/1709853388 >> 172.21.15.7:0/1709853388 conn(0x7f8168078770 msgr2=0x7f8168078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 -- 172.21.15.7:0/1709853388 shutdown_connections 2024-01-11T17:57:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.938+0000 7f816e628700 1 -- 172.21.15.7:0/1709853388 wait complete. 2024-01-11T17:57:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 Processor -- start 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 -- start start 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168073840 0x7f8168199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8168074160 0x7f816819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 0x7f816819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819ecc0 con 0x7f8168074160 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819ee30 con 0x7f8168073840 2024-01-11T17:57:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f816e628700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819eff0 con 0x7f816810e010 2024-01-11T17:57:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.940+0000 7f8167fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168073840 0x7f8168199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8168074160 0x7f816819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f8167fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168073840 0x7f8168199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41774/0 (socket says 172.21.15.7:41774) 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f8167fff700 1 -- 172.21.15.7:0/3462553743 learned_addr learned my addr 172.21.15.7:0/3462553743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f816cbc5700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 0x7f816819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 -- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 msgr2=0x7f816819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816810e010 0x7f816819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 -- 172.21.15.7:0/3462553743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168073840 msgr2=0x7f8168199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168073840 0x7f8168199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 -- 172.21.15.7:0/3462553743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160009c40 con 0x7f8168074160 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f8167fff700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168073840 0x7f8168199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.941+0000 7f81677fe700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8168074160 0x7f816819a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f815c00e9c0 tx=0x7f815c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.942+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c00c9c0 con 0x7f8168074160 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.942+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815c010430 con 0x7f8168074160 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.942+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c018680 con 0x7f8168074160 2024-01-11T17:57:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.942+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816819f2d0 con 0x7f8168074160 2024-01-11T17:57:00.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.942+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81681a6b30 con 0x7f8168074160 2024-01-11T17:57:00.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.943+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8168113b00 con 0x7f8168074160 2024-01-11T17:57:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.945+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f815c010860 con 0x7f8168074160 2024-01-11T17:57:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.947+0000 7f81657fa700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81500a1ff0 0x7f81500a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.947+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f815c014070 con 0x7f8168074160 2024-01-11T17:57:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.947+0000 7f8167fff700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81500a1ff0 0x7f81500a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.947+0000 7f81657fa700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] conn(0x7f81500ad010 0x7f81500af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.947+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 --> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81500afb80 con 0x7f81500ad010 2024-01-11T17:57:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.947+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f815c0d0270 con 0x7f8168074160 2024-01-11T17:57:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.948+0000 7f816cbc5700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] conn(0x7f81500ad010 0x7f81500af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.948+0000 7f8167fff700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81500a1ff0 0x7f81500a44b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8158011550 tx=0x7f8158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.948+0000 7f816cbc5700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] conn(0x7f81500ad010 0x7f81500af4b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.949+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== osd.5 v2:172.21.15.38:6808/3973290350 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f81500afb80 con 0x7f81500ad010 2024-01-11T17:57:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.980+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 --> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81680ff0a0 con 0x7f81500ad010 2024-01-11T17:57:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.981+0000 7f81657fa700 1 -- 172.21.15.7:0/3462553743 <== osd.5 v2:172.21.15.38:6808/3973290350 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f81680ff0a0 con 0x7f81500ad010 2024-01-11T17:57:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] conn(0x7f81500ad010 msgr2=0x7f81500af4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] conn(0x7f81500ad010 0x7f81500af4b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81500a1ff0 msgr2=0x7f81500a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81500a1ff0 0x7f81500a44b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8158011550 tx=0x7f8158009260 comp rx=0 tx=0).stop 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8168074160 msgr2=0x7f816819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 --2- 172.21.15.7:0/3462553743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8168074160 0x7f816819a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f815c00e9c0 tx=0x7f815c00ecd0 comp rx=0 tx=0).stop 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816cbc5700 1 -- 172.21.15.7:0/3462553743 reap_dead start 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 shutdown_connections 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 >> 172.21.15.7:0/3462553743 conn(0x7f8168078770 msgr2=0x7f81680fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 shutdown_connections 2024-01-11T17:57:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:00.982+0000 7f816e628700 1 -- 172.21.15.7:0/3462553743 wait complete. 2024-01-11T17:57:01.154 INFO:teuthology.orchestra.run.smithi007.stdout:137438953520 2024-01-11T17:57:01.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-01-11T17:57:01.724 INFO:teuthology.orchestra.run.smithi007.stdout:107374182452 2024-01-11T17:57:01.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-01-11T17:57:01.805 INFO:teuthology.orchestra.run.smithi007.stdout:77309411384 2024-01-11T17:57:01.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-01-11T17:57:01.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:01 smithi007 ceph-mon[33361]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-01-11T17:57:01.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:01 smithi016 ceph-mon[43195]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-01-11T17:57:02.000 INFO:teuthology.orchestra.run.smithi007.stdout:158913789998 2024-01-11T17:57:02.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-01-11T17:57:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:01 smithi038 ceph-mon[43582]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-01-11T17:57:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.343 INFO:teuthology.orchestra.run.smithi007.stdout:240518168609 2024-01-11T17:57:02.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-01-11T17:57:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.525 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.838 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:03.923 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:03 smithi007 ceph-mon[33361]: pgmap v275: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-11T17:57:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:03 smithi016 ceph-mon[43195]: pgmap v275: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-11T17:57:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:03 smithi038 ceph-mon[43582]: pgmap v275: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-11T17:57:05.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:05 smithi007 ceph-mon[33361]: pgmap v276: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-11T17:57:05.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:05 smithi016 ceph-mon[43195]: pgmap v276: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-11T17:57:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:05 smithi038 ceph-mon[43582]: pgmap v276: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-11T17:57:05.821 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 -- 172.21.15.7:0/3164899540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568107650 msgr2=0x7f3568107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 --2- 172.21.15.7:0/3164899540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568107650 0x7f3568107a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3558009990 tx=0x7f3558009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 -- 172.21.15.7:0/3164899540 shutdown_connections 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 --2- 172.21.15.7:0/3164899540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568107650 0x7f3568107a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 --2- 172.21.15.7:0/3164899540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568102020 0x7f356810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 --2- 172.21.15.7:0/3164899540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568101660 0x7f3568101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 -- 172.21.15.7:0/3164899540 >> 172.21.15.7:0/3164899540 conn(0x7f3568078770 msgr2=0x7f3568078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 -- 172.21.15.7:0/3164899540 shutdown_connections 2024-01-11T17:57:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.485+0000 7f356c978700 1 -- 172.21.15.7:0/3164899540 wait complete. 2024-01-11T17:57:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.486+0000 7f356c978700 1 Processor -- start 2024-01-11T17:57:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 -- start start 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 0x7f356819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568102020 0x7f356819ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 0x7f3568199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356819a2d0 con 0x7f3568101660 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356819a440 con 0x7f3568107650 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356c978700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356819a600 con 0x7f3568102020 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 0x7f356819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f3566d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 0x7f3568199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f3565d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568102020 0x7f356819ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 0x7f356819fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44540/0 (socket says 172.21.15.7:44540) 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f3566d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 0x7f3568199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43046/0 (socket says 172.21.15.7:43046) 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.487+0000 7f356659c700 1 -- 172.21.15.7:0/190201254 learned_addr learned my addr 172.21.15.7:0/190201254 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f3566d9d700 1 -- 172.21.15.7:0/190201254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568102020 msgr2=0x7f356819ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f3566d9d700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568102020 0x7f356819ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f3566d9d700 1 -- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 msgr2=0x7f356819fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f3566d9d700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 0x7f356819fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f3566d9d700 1 -- 172.21.15.7:0/190201254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580095f0 con 0x7f3568107650 2024-01-11T17:57:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f356659c700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 0x7f356819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f3566d9d700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 0x7f3568199b20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f356000efc0 tx=0x7f356000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560018070 con 0x7f3568107650 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f356000f040 con 0x7f3568107650 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356819a880 con 0x7f3568107650 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000cc80 con 0x7f3568107650 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.488+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35681a69d0 con 0x7f3568107650 2024-01-11T17:57:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.489+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3568102d90 con 0x7f3568107650 2024-01-11T17:57:06.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.491+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f3560003a00 con 0x7f3568107650 2024-01-11T17:57:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.493+0000 7f35577fe700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35500a1f70 0x7f35500a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.493+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f35600ce2f0 con 0x7f3568107650 2024-01-11T17:57:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.493+0000 7f356659c700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35500a1f70 0x7f35500a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.494+0000 7f356659c700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35500a1f70 0x7f35500a4430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3558006010 tx=0x7f3558005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.495+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3560096f50 con 0x7f3568107650 2024-01-11T17:57:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.733+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f3568111680 con 0x7f3568107650 2024-01-11T17:57:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.734+0000 7f35577fe700 1 -- 172.21.15.7:0/190201254 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f356009a570 con 0x7f3568107650 2024-01-11T17:57:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:270582939677 2024-01-11T17:57:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.736+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35500a1f70 msgr2=0x7f35500a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.736+0000 7f356c978700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35500a1f70 0x7f35500a4430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3558006010 tx=0x7f3558005bc0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.736+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 msgr2=0x7f3568199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.736+0000 7f356c978700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 0x7f3568199b20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f356000efc0 tx=0x7f356000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 shutdown_connections 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35500a1f70 0x7f35500a4430 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3568107650 0x7f3568199b20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3568101660 0x7f356819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 --2- 172.21.15.7:0/190201254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3568102020 0x7f356819ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 >> 172.21.15.7:0/190201254 conn(0x7f3568078770 msgr2=0x7f35680ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 shutdown_connections 2024-01-11T17:57:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.737+0000 7f356c978700 1 -- 172.21.15.7:0/190201254 wait complete. 2024-01-11T17:57:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 -- 172.21.15.7:0/798170063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 msgr2=0x7fac5c1105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 --2- 172.21.15.7:0/798170063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 0x7fac5c1105b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac54009f90 tx=0x7fac5400b330 comp rx=0 tx=0).stop 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 -- 172.21.15.7:0/798170063 shutdown_connections 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 --2- 172.21.15.7:0/798170063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 0x7fac5c1105b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 --2- 172.21.15.7:0/798170063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1096e0 0x7fac5c109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 --2- 172.21.15.7:0/798170063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1036e0 0x7fac5c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 -- 172.21.15.7:0/798170063 >> 172.21.15.7:0/798170063 conn(0x7fac5c0ff1f0 msgr2=0x7fac5c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.948+0000 7fac5b59e700 1 -- 172.21.15.7:0/798170063 shutdown_connections 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.949+0000 7fac5b59e700 1 -- 172.21.15.7:0/798170063 wait complete. 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 Processor -- start 2024-01-11T17:57:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 -- start start 2024-01-11T17:57:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 0x7fac5c19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 0x7fac5c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1096e0 0x7fac5c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c19a140 con 0x7fac5c1036e0 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c19a280 con 0x7fac5c1096e0 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.950+0000 7fac5b59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c19a440 con 0x7fac5c1040a0 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 0x7fac5c19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1096e0 0x7fac5c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac59d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 0x7fac5c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 0x7fac5c19f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44552/0 (socket says 172.21.15.7:44552) 2024-01-11T17:57:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5a59c700 1 -- 172.21.15.7:0/983759053 learned_addr learned my addr 172.21.15.7:0/983759053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 -- 172.21.15.7:0/983759053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 msgr2=0x7fac5c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 0x7fac5c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 -- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 msgr2=0x7fac5c19f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 0x7fac5c19f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 -- 172.21.15.7:0/983759053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0095f0 con 0x7fac5c1096e0 2024-01-11T17:57:06.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5a59c700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 0x7fac5c19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.951+0000 7fac5ad9d700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1096e0 0x7fac5c199ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac5400b6c0 tx=0x7fac540094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.952+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5400e040 con 0x7fac5c1096e0 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.952+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac5401b450 con 0x7fac5c1096e0 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.952+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac54009c40 con 0x7fac5c1096e0 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.952+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540052a0 con 0x7fac5c1096e0 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.952+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c1a67a0 con 0x7fac5c1096e0 2024-01-11T17:57:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.953+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5c10bd00 con 0x7fac5c1096e0 2024-01-11T17:57:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.955+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fac5401b5c0 con 0x7fac5c1096e0 2024-01-11T17:57:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.956+0000 7fac4b7fe700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fac440a1f70 0x7fac440a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.957+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fac540cefe0 con 0x7fac5c1096e0 2024-01-11T17:57:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.957+0000 7fac5a59c700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fac440a1f70 0x7fac440a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.957+0000 7fac5a59c700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fac440a1f70 0x7fac440a4430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac5c101260 tx=0x7fac4c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:06.961+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fac54097c80 con 0x7fac5c1096e0 2024-01-11T17:57:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.204+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fac5c19afc0 con 0x7fac5c1096e0 2024-01-11T17:57:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.205+0000 7fac4b7fe700 1 -- 172.21.15.7:0/983759053 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fac5409b2a0 con 0x7fac5c1096e0 2024-01-11T17:57:07.207 INFO:teuthology.orchestra.run.smithi007.stdout:322122547223 2024-01-11T17:57:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fac440a1f70 msgr2=0x7fac440a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fac440a1f70 0x7fac440a4430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac5c101260 tx=0x7fac4c009310 comp rx=0 tx=0).stop 2024-01-11T17:57:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1096e0 msgr2=0x7fac5c199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1096e0 0x7fac5c199ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac5400b6c0 tx=0x7fac540094b0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 shutdown_connections 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fac440a1f70 0x7fac440a4430 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac5c1096e0 0x7fac5c199ae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac5c1036e0 0x7fac5c19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 --2- 172.21.15.7:0/983759053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac5c1040a0 0x7fac5c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.207+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 >> 172.21.15.7:0/983759053 conn(0x7fac5c0ff1f0 msgr2=0x7fac5c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.208+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 shutdown_connections 2024-01-11T17:57:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.208+0000 7fac5b59e700 1 -- 172.21.15.7:0/983759053 wait complete. 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 -- 172.21.15.7:0/2612455884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 msgr2=0x7f16b8101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 --2- 172.21.15.7:0/2612455884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8101b10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009990 tx=0x7f16ac009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 -- 172.21.15.7:0/2612455884 shutdown_connections 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 --2- 172.21.15.7:0/2612455884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16b8107690 0x7f16b8107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 --2- 172.21.15.7:0/2612455884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8102050 0x7f16b810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 --2- 172.21.15.7:0/2612455884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8101b10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 -- 172.21.15.7:0/2612455884 >> 172.21.15.7:0/2612455884 conn(0x7f16b8078770 msgr2=0x7f16b8078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 -- 172.21.15.7:0/2612455884 shutdown_connections 2024-01-11T17:57:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.358+0000 7f16be10a700 1 -- 172.21.15.7:0/2612455884 wait complete. 2024-01-11T17:57:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.359+0000 7f16be10a700 1 Processor -- start 2024-01-11T17:57:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.360+0000 7f16be10a700 1 -- start start 2024-01-11T17:57:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.360+0000 7f16be10a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.360+0000 7f16be10a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8102050 0x7f16b8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16be10a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16b8107690 0x7f16b8074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16be10a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a5e80 con 0x7f16b8101690 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16be10a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a5ff0 con 0x7f16b8102050 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16be10a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a61b0 con 0x7f16b8107690 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44566/0 (socket says 172.21.15.7:44566) 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 -- 172.21.15.7:0/1944090690 learned_addr learned my addr 172.21.15.7:0/1944090690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b7fff700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16b8107690 0x7f16b8074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b6ffd700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8102050 0x7f16b8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 -- 172.21.15.7:0/1944090690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16b8107690 msgr2=0x7f16b8074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16b8107690 0x7f16b8074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 -- 172.21.15.7:0/1944090690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8102050 msgr2=0x7f16b8073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8102050 0x7f16b8073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.361+0000 7f16b77fe700 1 -- 172.21.15.7:0/1944090690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009520 con 0x7f16b8101690 2024-01-11T17:57:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.362+0000 7f16b77fe700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8073910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f16a800ea50 tx=0x7f16a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.362+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800cbd0 con 0x7f16b8101690 2024-01-11T17:57:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.362+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a800cd30 con 0x7f16b8101690 2024-01-11T17:57:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.362+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80186c0 con 0x7f16b8101690 2024-01-11T17:57:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.362+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac0095f0 con 0x7f16b8101690 2024-01-11T17:57:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.362+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b81a6800 con 0x7f16b8101690 2024-01-11T17:57:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.364+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b8109c20 con 0x7f16b8101690 2024-01-11T17:57:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.365+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f16a8018820 con 0x7f16b8101690 2024-01-11T17:57:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.368+0000 7f16b4ff9700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16a00a1f70 0x7f16a00a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.368+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f16a8014070 con 0x7f16b8101690 2024-01-11T17:57:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.368+0000 7f16b6ffd700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16a00a1f70 0x7f16a00a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.369+0000 7f16b6ffd700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16a00a1f70 0x7f16a00a4430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b2e0 tx=0x7f16ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.372+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f16a8096f20 con 0x7f16b8101690 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 -- 172.21.15.7:0/356029700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc910107640 msgr2=0x7fc910107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 --2- 172.21.15.7:0/356029700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc910107640 0x7fc910107a20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0099c0 tx=0x7fc90c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 -- 172.21.15.7:0/356029700 shutdown_connections 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 --2- 172.21.15.7:0/356029700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910069c80 0x7fc91010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 --2- 172.21.15.7:0/356029700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc910107640 0x7fc910107a20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 --2- 172.21.15.7:0/356029700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9100692c0 0x7fc910069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 -- 172.21.15.7:0/356029700 >> 172.21.15.7:0/356029700 conn(0x7fc910077440 msgr2=0x7fc910077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 -- 172.21.15.7:0/356029700 shutdown_connections 2024-01-11T17:57:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.457+0000 7fc917d7f700 1 -- 172.21.15.7:0/356029700 wait complete. 2024-01-11T17:57:07.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.458+0000 7fc917d7f700 1 Processor -- start 2024-01-11T17:57:07.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.458+0000 7fc917d7f700 1 -- start start 2024-01-11T17:57:07.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc917d7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9100692c0 0x7fc9101005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc917d7f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc910069c80 0x7fc910100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc917d7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910107640 0x7fc910101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc917d7f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9101a5d80 con 0x7fc9100692c0 2024-01-11T17:57:07.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc917d7f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9101a5ef0 con 0x7fc910107640 2024-01-11T17:57:07.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc917d7f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9101a60b0 con 0x7fc910069c80 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc915b1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9100692c0 0x7fc9101005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc915b1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9100692c0 0x7fc9101005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44586/0 (socket says 172.21.15.7:44586) 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc915b1b700 1 -- 172.21.15.7:0/814010387 learned_addr learned my addr 172.21.15.7:0/814010387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.459+0000 7fc91531a700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc910069c80 0x7fc910100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91631c700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910107640 0x7fc910101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91531a700 1 -- 172.21.15.7:0/814010387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910107640 msgr2=0x7fc910101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91531a700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910107640 0x7fc910101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91531a700 1 -- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9100692c0 msgr2=0x7fc9101005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91531a700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9100692c0 0x7fc9101005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91531a700 1 -- 172.21.15.7:0/814010387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0095f0 con 0x7fc910069c80 2024-01-11T17:57:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91631c700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910107640 0x7fc910101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.460+0000 7fc91531a700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc910069c80 0x7fc910100b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e9f0 tx=0x7fc90000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.461+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000c9c0 con 0x7fc910069c80 2024-01-11T17:57:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.461+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc900010430 con 0x7fc910069c80 2024-01-11T17:57:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.461+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9101a6390 con 0x7fc910069c80 2024-01-11T17:57:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.461+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900018680 con 0x7fc910069c80 2024-01-11T17:57:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.461+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9101a6890 con 0x7fc910069c80 2024-01-11T17:57:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.462+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc910109b80 con 0x7fc910069c80 2024-01-11T17:57:07.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.464+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc900010860 con 0x7fc910069c80 2024-01-11T17:57:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.465+0000 7fc906ffd700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8fc0a1f70 0x7fc8fc0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.466+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc900014070 con 0x7fc910069c80 2024-01-11T17:57:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.466+0000 7fc915b1b700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8fc0a1f70 0x7fc8fc0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.466+0000 7fc915b1b700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8fc0a1f70 0x7fc8fc0a4430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009990 tx=0x7fc90c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.468+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc900096d60 con 0x7fc910069c80 2024-01-11T17:57:07.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.607+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f16b8074e40 con 0x7f16b8101690 2024-01-11T17:57:07.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.608+0000 7f16b4ff9700 1 -- 172.21.15.7:0/1944090690 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16a809a540 con 0x7f16b8101690 2024-01-11T17:57:07.609 INFO:teuthology.orchestra.run.smithi007.stdout:55834574911 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16a00a1f70 msgr2=0x7f16a00a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16a00a1f70 0x7f16a00a4430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b2e0 tx=0x7f16ac005fd0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 msgr2=0x7f16b8073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8073910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f16a800ea50 tx=0x7f16a800ed60 comp rx=0 tx=0).stop 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 shutdown_connections 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f16a00a1f70 0x7f16a00a4430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16b8107690 0x7f16b8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16b8101690 0x7f16b8073910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 --2- 172.21.15.7:0/1944090690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b8102050 0x7f16b8073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 >> 172.21.15.7:0/1944090690 conn(0x7f16b8078770 msgr2=0x7f16b8100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 shutdown_connections 2024-01-11T17:57:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.610+0000 7f16be10a700 1 -- 172.21.15.7:0/1944090690 wait complete. 2024-01-11T17:57:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.691+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc9101114e0 con 0x7fc910069c80 2024-01-11T17:57:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.692+0000 7fc906ffd700 1 -- 172.21.15.7:0/814010387 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc90009a380 con 0x7fc910069c80 2024-01-11T17:57:07.694 INFO:teuthology.orchestra.run.smithi007.stdout:292057776154 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.694+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8fc0a1f70 msgr2=0x7fc8fc0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.694+0000 7fc917d7f700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8fc0a1f70 0x7fc8fc0a4430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009990 tx=0x7fc90c005ab0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.694+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc910069c80 msgr2=0x7fc910100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.694+0000 7fc917d7f700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc910069c80 0x7fc910100b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e9f0 tx=0x7fc90000ed00 comp rx=0 tx=0).stop 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 shutdown_connections 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8fc0a1f70 0x7fc8fc0a4430 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910107640 0x7fc910101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9100692c0 0x7fc9101005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 --2- 172.21.15.7:0/814010387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc910069c80 0x7fc910100b20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 >> 172.21.15.7:0/814010387 conn(0x7fc910077440 msgr2=0x7fc9100ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 shutdown_connections 2024-01-11T17:57:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:07.695+0000 7fc917d7f700 1 -- 172.21.15.7:0/814010387 wait complete. 2024-01-11T17:57:07.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:07 smithi007 ceph-mon[33361]: pgmap v277: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-01-11T17:57:07.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/190201254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-11T17:57:07.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/983759053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-11T17:57:07.928 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1944090690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-11T17:57:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:07 smithi016 ceph-mon[43195]: pgmap v277: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-01-11T17:57:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/190201254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-11T17:57:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/983759053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-11T17:57:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1944090690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-11T17:57:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:07 smithi038 ceph-mon[43582]: pgmap v277: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-01-11T17:57:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/190201254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-11T17:57:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/983759053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-11T17:57:08.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1944090690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-11T17:57:08.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 -- 172.21.15.7:0/3268119194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c107630 msgr2=0x7efc2c107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 --2- 172.21.15.7:0/3268119194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c107630 0x7efc2c107a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc28009990 tx=0x7efc28009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 -- 172.21.15.7:0/3268119194 shutdown_connections 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 --2- 172.21.15.7:0/3268119194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c107630 0x7efc2c107a10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 --2- 172.21.15.7:0/3268119194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c101ff0 0x7efc2c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 --2- 172.21.15.7:0/3268119194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c101630 0x7efc2c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 -- 172.21.15.7:0/3268119194 >> 172.21.15.7:0/3268119194 conn(0x7efc2c0fd0e0 msgr2=0x7efc2c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.147+0000 7efc33c5e700 1 -- 172.21.15.7:0/3268119194 shutdown_connections 2024-01-11T17:57:08.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.148+0000 7efc33c5e700 1 -- 172.21.15.7:0/3268119194 wait complete. 2024-01-11T17:57:08.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.149+0000 7efc33c5e700 1 Processor -- start 2024-01-11T17:57:08.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 -- start start 2024-01-11T17:57:08.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c101630 0x7efc2c10d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c101ff0 0x7efc2c10dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 0x7efc2c1a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1a2f80 con 0x7efc2c101630 2024-01-11T17:57:08.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1a30f0 con 0x7efc2c107630 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc33c5e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1a32b0 con 0x7efc2c101ff0 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.150+0000 7efc319fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c101630 0x7efc2c10d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 0x7efc2c1a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 0x7efc2c1a2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43140/0 (socket says 172.21.15.7:43140) 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 -- 172.21.15.7:0/520729913 learned_addr learned my addr 172.21.15.7:0/520729913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 -- 172.21.15.7:0/520729913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c101ff0 msgr2=0x7efc2c10dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c101ff0 0x7efc2c10dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 -- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c101630 msgr2=0x7efc2c10d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c101630 0x7efc2c10d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 -- 172.21.15.7:0/520729913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c009520 con 0x7efc2c107630 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.151+0000 7efc321fb700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 0x7efc2c1a2980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc2400ef30 tx=0x7efc2400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.152+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24018070 con 0x7efc2c107630 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.152+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc2400f040 con 0x7efc2c107630 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.152+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24004020 con 0x7efc2c107630 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.152+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc280095f0 con 0x7efc2c107630 2024-01-11T17:57:08.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.152+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c1aac30 con 0x7efc2c107630 2024-01-11T17:57:08.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.153+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c102d70 con 0x7efc2c107630 2024-01-11T17:57:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.156+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7efc24004ad0 con 0x7efc2c107630 2024-01-11T17:57:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.159+0000 7efc22ffd700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc180a1ff0 0x7efc180a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.159+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc240ce3a0 con 0x7efc2c107630 2024-01-11T17:57:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.159+0000 7efc319fa700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc180a1ff0 0x7efc180a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.160+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7efc24097060 con 0x7efc2c107630 2024-01-11T17:57:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.160+0000 7efc319fa700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc180a1ff0 0x7efc180a44b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b2e0 tx=0x7efc28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3211416019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 msgr2=0x7fc1d0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3211416019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d0107a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009990 tx=0x7fc1c0009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3211416019 shutdown_connections 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3211416019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3211416019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d0107a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3211416019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 0x7fc1d0069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3211416019 >> 172.21.15.7:0/3211416019 conn(0x7fc1d0077440 msgr2=0x7fc1d0077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3211416019 shutdown_connections 2024-01-11T17:57:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.278+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3211416019 wait complete. 2024-01-11T17:57:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.279+0000 7fc1d65a8700 1 Processor -- start 2024-01-11T17:57:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.281+0000 7fc1d65a8700 1 -- start start 2024-01-11T17:57:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d65a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 0x7fc1d019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d65a8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d65a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d65a8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0199f40 con 0x7fc1d00692c0 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d65a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d019a080 con 0x7fc1d0107640 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d65a8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d019a240 con 0x7fc1d0069c80 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1cf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1d4b45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1cf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52642/0 (socket says 172.21.15.7:52642) 2024-01-11T17:57:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1cf7fe700 1 -- 172.21.15.7:0/3064732773 learned_addr learned my addr 172.21.15.7:0/3064732773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.282+0000 7fc1cffff700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 0x7fc1d019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1d4b45700 1 -- 172.21.15.7:0/3064732773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 msgr2=0x7fc1d019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1d4b45700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1d4b45700 1 -- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 msgr2=0x7fc1d019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1d4b45700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 0x7fc1d019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1d4b45700 1 -- 172.21.15.7:0/3064732773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c00095f0 con 0x7fc1d0107640 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1cffff700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 0x7fc1d019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1cf7fe700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1d4b45700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d01998e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800efc0 tx=0x7fc1c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.283+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8018070 con 0x7fc1d0107640 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.284+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c800f040 con 0x7fc1d0107640 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.284+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800cdf0 con 0x7fc1d0107640 2024-01-11T17:57:08.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.284+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d019a4c0 con 0x7fc1d0107640 2024-01-11T17:57:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.284+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d01a67c0 con 0x7fc1d0107640 2024-01-11T17:57:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.285+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d00682d0 con 0x7fc1d0107640 2024-01-11T17:57:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.289+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc1c8003a00 con 0x7fc1d0107640 2024-01-11T17:57:08.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.290+0000 7fc1cd7fa700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc1b80a1ff0 0x7fc1b80a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.290+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1c80cdad0 con 0x7fc1d0107640 2024-01-11T17:57:08.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.291+0000 7fc1cffff700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc1b80a1ff0 0x7fc1b80a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.292+0000 7fc1cffff700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc1b80a1ff0 0x7fc1b80a44b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000c00 tx=0x7fc1c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:08.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.295+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc1c8096730 con 0x7fc1d0107640 2024-01-11T17:57:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.416+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7efc2c1115b0 con 0x7efc2c107630 2024-01-11T17:57:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.417+0000 7efc22ffd700 1 -- 172.21.15.7:0/520729913 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc2409a680 con 0x7efc2c107630 2024-01-11T17:57:08.419 INFO:teuthology.orchestra.run.smithi007.stdout:210453397542 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc180a1ff0 msgr2=0x7efc180a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc180a1ff0 0x7efc180a44b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b2e0 tx=0x7efc28005fd0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 msgr2=0x7efc2c1a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 0x7efc2c1a2980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc2400ef30 tx=0x7efc2400c450 comp rx=0 tx=0).stop 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 shutdown_connections 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc180a1ff0 0x7efc180a44b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2c107630 0x7efc2c1a2980 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c101630 0x7efc2c10d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 --2- 172.21.15.7:0/520729913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc2c101ff0 0x7efc2c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 >> 172.21.15.7:0/520729913 conn(0x7efc2c0fd0e0 msgr2=0x7efc2c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 shutdown_connections 2024-01-11T17:57:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.420+0000 7efc33c5e700 1 -- 172.21.15.7:0/520729913 wait complete. 2024-01-11T17:57:08.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.534+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc1d01114e0 con 0x7fc1d0107640 2024-01-11T17:57:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.534+0000 7fc1cd7fa700 1 -- 172.21.15.7:0/3064732773 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc1c8099d50 con 0x7fc1d0107640 2024-01-11T17:57:08.536 INFO:teuthology.orchestra.run.smithi007.stdout:188978561066 2024-01-11T17:57:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.536+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc1b80a1ff0 msgr2=0x7fc1b80a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.536+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc1b80a1ff0 0x7fc1b80a44b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000c00 tx=0x7fc1c0005fb0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.536+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 msgr2=0x7fc1d01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.536+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d01998e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800efc0 tx=0x7fc1c800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 shutdown_connections 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc1b80a1ff0 0x7fc1b80a44b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d0107640 0x7fc1d01998e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1d00692c0 0x7fc1d019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 --2- 172.21.15.7:0/3064732773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1d0069c80 0x7fc1d019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 >> 172.21.15.7:0/3064732773 conn(0x7fc1d0077440 msgr2=0x7fc1d010c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 shutdown_connections 2024-01-11T17:57:08.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.537+0000 7fc1d65a8700 1 -- 172.21.15.7:0/3064732773 wait complete. 2024-01-11T17:57:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:08 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/814010387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-11T17:57:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:08 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4227258659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-11T17:57:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:08 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/520729913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-11T17:57:08.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:08 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3064732773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.018+0000 7fa82720e700 1 -- 172.21.15.7:0/3105776016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820107640 msgr2=0x7fa820107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.018+0000 7fa82720e700 1 --2- 172.21.15.7:0/3105776016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820107640 0x7fa820107a20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009920 tx=0x7fa81c009c30 comp rx=0 tx=0).stop 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 -- 172.21.15.7:0/3105776016 shutdown_connections 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 --2- 172.21.15.7:0/3105776016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa820069c80 0x7fa82010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 --2- 172.21.15.7:0/3105776016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820107640 0x7fa820107a20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 --2- 172.21.15.7:0/3105776016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8200692c0 0x7fa820069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 -- 172.21.15.7:0/3105776016 >> 172.21.15.7:0/3105776016 conn(0x7fa820077440 msgr2=0x7fa820077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 -- 172.21.15.7:0/3105776016 shutdown_connections 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.019+0000 7fa82720e700 1 -- 172.21.15.7:0/3105776016 wait complete. 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.020+0000 7fa82720e700 1 Processor -- start 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.020+0000 7fa82720e700 1 -- start start 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.020+0000 7fa82720e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8200692c0 0x7fa82019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.020+0000 7fa82720e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 0x7fa82019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa82720e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa820107640 0x7fa820199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa82720e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82019a090 con 0x7fa820069c80 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa82720e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82019a1d0 con 0x7fa820107640 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa82720e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82019a390 con 0x7fa8200692c0 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 0x7fa82019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 0x7fa82019fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44600/0 (socket says 172.21.15.7:44600) 2024-01-11T17:57:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 -- 172.21.15.7:0/4227258659 learned_addr learned my addr 172.21.15.7:0/4227258659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa8257ab700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa820107640 0x7fa820199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa824faa700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8200692c0 0x7fa82019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 -- 172.21.15.7:0/4227258659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8200692c0 msgr2=0x7fa82019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8200692c0 0x7fa82019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 -- 172.21.15.7:0/4227258659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa820107640 msgr2=0x7fa820199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa820107640 0x7fa820199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.021+0000 7fa817fff700 1 -- 172.21.15.7:0/4227258659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c009520 con 0x7fa820069c80 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.022+0000 7fa817fff700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 0x7fa82019fdb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e9a0 tx=0x7fa81000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.022+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810009790 con 0x7fa820069c80 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.022+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa810010430 con 0x7fa820069c80 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.022+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82019a670 con 0x7fa820069c80 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.022+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810018680 con 0x7fa820069c80 2024-01-11T17:57:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.022+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8201a6910 con 0x7fa820069c80 2024-01-11T17:57:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.023+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8200682d0 con 0x7fa820069c80 2024-01-11T17:57:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.026+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fa8100105a0 con 0x7fa820069c80 2024-01-11T17:57:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.027+0000 7fa815ffb700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa8080a1f70 0x7fa8080a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.027+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa810014070 con 0x7fa820069c80 2024-01-11T17:57:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.028+0000 7fa824faa700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa8080a1f70 0x7fa8080a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:08.826 INFO:teuthology.orchestra.run.smithi007.stdout:34359738434 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.029+0000 7fa824faa700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa8080a1f70 0x7fa8080a4430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009810 tx=0x7fa81c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.032+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa810052380 con 0x7fa820069c80 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.258+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8201114e0 con 0x7fa820069c80 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.259+0000 7fa815ffb700 1 -- 172.21.15.7:0/4227258659 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa810096c30 con 0x7fa820069c80 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa8080a1f70 msgr2=0x7fa8080a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa8080a1f70 0x7fa8080a4430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009810 tx=0x7fa81c019040 comp rx=0 tx=0).stop 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 msgr2=0x7fa82019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 0x7fa82019fdb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e9a0 tx=0x7fa81000ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 shutdown_connections 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa8080a1f70 0x7fa8080a4430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8200692c0 0x7fa82019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa820107640 0x7fa820199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 --2- 172.21.15.7:0/4227258659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa820069c80 0x7fa82019fdb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.262+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 >> 172.21.15.7:0/4227258659 conn(0x7fa820077440 msgr2=0x7fa82010c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.263+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 shutdown_connections 2024-01-11T17:57:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:08.263+0000 7fa82720e700 1 -- 172.21.15.7:0/4227258659 wait complete. 2024-01-11T17:57:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:08 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/814010387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-11T17:57:08.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:08 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4227258659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-11T17:57:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:08 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/520729913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-11T17:57:08.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:08 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3064732773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-11T17:57:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:08 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/814010387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-11T17:57:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:08 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4227258659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-11T17:57:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:08 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/520729913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-11T17:57:09.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:08 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3064732773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-11T17:57:09.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939674 got 270582939677 for osd.9 2024-01-11T17:57:09.131 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:09.211 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547223 for osd.11 2024-01-11T17:57:09.212 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:09.344 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738434 for osd.0 2024-01-11T17:57:09.344 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:09.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776152 got 292057776154 for osd.10 2024-01-11T17:57:09.490 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:09.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561064 got 188978561066 for osd.6 2024-01-11T17:57:09.708 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:09.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:09 smithi007 ceph-mon[33361]: pgmap v278: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:09.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:09 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:57:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:09 smithi016 ceph-mon[43195]: pgmap v278: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:09 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:57:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:09 smithi038 ceph-mon[43582]: pgmap v278: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:09 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:57:10.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397540 got 210453397542 for osd.7 2024-01-11T17:57:10.086 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:10.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574908 got 55834574911 for osd.1 2024-01-11T17:57:10.863 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 -- 172.21.15.7:0/94391716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c101660 msgr2=0x7fdd5c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 --2- 172.21.15.7:0/94391716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c101660 0x7fdd5c101ae0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009990 tx=0x7fdd48009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 -- 172.21.15.7:0/94391716 shutdown_connections 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 --2- 172.21.15.7:0/94391716 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c102020 0x7fdd5c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 --2- 172.21.15.7:0/94391716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c107650 0x7fdd5c107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 --2- 172.21.15.7:0/94391716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c101660 0x7fdd5c101ae0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 -- 172.21.15.7:0/94391716 >> 172.21.15.7:0/94391716 conn(0x7fdd5c078770 msgr2=0x7fdd5c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 -- 172.21.15.7:0/94391716 shutdown_connections 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.108+0000 7fdd63189700 1 -- 172.21.15.7:0/94391716 wait complete. 2024-01-11T17:57:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.109+0000 7fdd63189700 1 Processor -- start 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 -- start start 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c101660 0x7fdd5c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c102020 0x7fdd5c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c107650 0x7fdd5c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19ec60 con 0x7fdd5c102020 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19edd0 con 0x7fdd5c107650 2024-01-11T17:57:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd63189700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c19ef90 con 0x7fdd5c101660 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd5bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c102020 0x7fdd5c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd60f25700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c101660 0x7fdd5c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd61726700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c107650 0x7fdd5c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd60f25700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c101660 0x7fdd5c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52662/0 (socket says 172.21.15.7:52662) 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.110+0000 7fdd60f25700 1 -- 172.21.15.7:0/2486470146 learned_addr learned my addr 172.21.15.7:0/2486470146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd5bfff700 1 -- 172.21.15.7:0/2486470146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c101660 msgr2=0x7fdd5c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd5bfff700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c101660 0x7fdd5c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd5bfff700 1 -- 172.21.15.7:0/2486470146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c107650 msgr2=0x7fdd5c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd5bfff700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c107650 0x7fdd5c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd5bfff700 1 -- 172.21.15.7:0/2486470146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480095f0 con 0x7fdd5c102020 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd5bfff700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c102020 0x7fdd5c19a310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098e0 tx=0x7fdd48005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4801d070 con 0x7fdd5c102020 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd48022470 con 0x7fdd5c102020 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.111+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c19f210 con 0x7fdd5c102020 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.112+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48017650 con 0x7fdd5c102020 2024-01-11T17:57:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.112+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1a6ad0 con 0x7fdd5c102020 2024-01-11T17:57:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.113+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c102d90 con 0x7fdd5c102020 2024-01-11T17:57:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.116+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fdd480225e0 con 0x7fdd5c102020 2024-01-11T17:57:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.118+0000 7fdd59ffb700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdd4c0a1ff0 0x7fdd4c0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.118+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdd480cf870 con 0x7fdd5c102020 2024-01-11T17:57:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.118+0000 7fdd60f25700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdd4c0a1ff0 0x7fdd4c0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.119+0000 7fdd60f25700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdd4c0a1ff0 0x7fdd4c0a44b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009ca0 tx=0x7fdd50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.120+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdd480984d0 con 0x7fdd5c102020 2024-01-11T17:57:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.403+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdd5c19b0e0 con 0x7fdd5c102020 2024-01-11T17:57:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.404+0000 7fdd59ffb700 1 -- 172.21.15.7:0/2486470146 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd48027090 con 0x7fdd5c102020 2024-01-11T17:57:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:137438953522 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.407+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdd4c0a1ff0 msgr2=0x7fdd4c0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.407+0000 7fdd63189700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdd4c0a1ff0 0x7fdd4c0a44b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009ca0 tx=0x7fdd50009260 comp rx=0 tx=0).stop 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.407+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c102020 msgr2=0x7fdd5c19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.407+0000 7fdd63189700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c102020 0x7fdd5c19a310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098e0 tx=0x7fdd48005740 comp rx=0 tx=0).stop 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 shutdown_connections 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdd4c0a1ff0 0x7fdd4c0a44b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd5c101660 0x7fdd5c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c107650 0x7fdd5c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 --2- 172.21.15.7:0/2486470146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c102020 0x7fdd5c19a310 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 >> 172.21.15.7:0/2486470146 conn(0x7fdd5c078770 msgr2=0x7fdd5c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 shutdown_connections 2024-01-11T17:57:11.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.408+0000 7fdd63189700 1 -- 172.21.15.7:0/2486470146 wait complete. 2024-01-11T17:57:11.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:11 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:11 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:11 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:11 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:11 smithi016 ceph-mon[43195]: pgmap v279: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-01-11T17:57:11.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:11 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2486470146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-11T17:57:11.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:11 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:11 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:11 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:11 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:11 smithi038 ceph-mon[43582]: pgmap v279: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-01-11T17:57:11.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:11 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2486470146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-11T17:57:11.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.814+0000 7ff61dfbb700 1 -- 172.21.15.7:0/3897858494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618069800 msgr2=0x7ff618069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.814+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/3897858494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618069800 0x7ff618069c60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff608009990 tx=0x7ff608009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 -- 172.21.15.7:0/3897858494 shutdown_connections 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/3897858494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff618104010 0x7ff6181044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/3897858494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618106500 0x7ff6180692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/3897858494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618069800 0x7ff618069c60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 -- 172.21.15.7:0/3897858494 >> 172.21.15.7:0/3897858494 conn(0x7ff618077440 msgr2=0x7ff618077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 -- 172.21.15.7:0/3897858494 shutdown_connections 2024-01-11T17:57:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.815+0000 7ff61dfbb700 1 -- 172.21.15.7:0/3897858494 wait complete. 2024-01-11T17:57:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.816+0000 7ff61dfbb700 1 Processor -- start 2024-01-11T17:57:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.816+0000 7ff61dfbb700 1 -- start start 2024-01-11T17:57:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff61dfbb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 0x7ff61819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff61dfbb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff618104010 0x7ff61819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff61dfbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618106500 0x7ff6181a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff61dfbb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181a2f60 con 0x7ff618106500 2024-01-11T17:57:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff61dfbb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181a30d0 con 0x7ff618069800 2024-01-11T17:57:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff61dfbb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181a3290 con 0x7ff618104010 2024-01-11T17:57:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 0x7ff61819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff6177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 0x7ff61819e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43198/0 (socket says 172.21.15.7:43198) 2024-01-11T17:57:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff617fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618106500 0x7ff6181a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff6177fe700 1 -- 172.21.15.7:0/300286578 learned_addr learned my addr 172.21.15.7:0/300286578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.817+0000 7ff616ffd700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff618104010 0x7ff61819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff6177fe700 1 -- 172.21.15.7:0/300286578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff618104010 msgr2=0x7ff61819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff6177fe700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff618104010 0x7ff61819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff6177fe700 1 -- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618106500 msgr2=0x7ff6181a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff6177fe700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618106500 0x7ff6181a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff6177fe700 1 -- 172.21.15.7:0/300286578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080095f0 con 0x7ff618069800 2024-01-11T17:57:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff617fff700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618106500 0x7ff6181a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.818+0000 7ff6177fe700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 0x7ff61819e080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6100060b0 tx=0x7ff61000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.819+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000ddd0 con 0x7ff618069800 2024-01-11T17:57:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.819+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6181a3570 con 0x7ff618069800 2024-01-11T17:57:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.819+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61000f040 con 0x7ff618069800 2024-01-11T17:57:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.819+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6181aadd0 con 0x7ff618069800 2024-01-11T17:57:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.819+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610014690 con 0x7ff618069800 2024-01-11T17:57:11.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:11 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:11 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:11 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:11 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:11 smithi007 ceph-mon[33361]: pgmap v279: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-01-11T17:57:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:11 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2486470146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-11T17:57:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.821+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604005360 con 0x7ff618069800 2024-01-11T17:57:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.822+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7ff6100147f0 con 0x7ff618069800 2024-01-11T17:57:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.824+0000 7ff614ff9700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6000a1ff0 0x7ff6000a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.824+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6100cd670 con 0x7ff618069800 2024-01-11T17:57:11.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.824+0000 7ff616ffd700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6000a1ff0 0x7ff6000a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:11.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.825+0000 7ff616ffd700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6000a1ff0 0x7ff6000a44b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff60800b5c0 tx=0x7ff60801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:11.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:11.828+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff610096240 con 0x7ff618069800 2024-01-11T17:57:12.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.073+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff6040058e0 con 0x7ff618069800 2024-01-11T17:57:12.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.073+0000 7ff614ff9700 1 -- 172.21.15.7:0/300286578 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff610099860 con 0x7ff618069800 2024-01-11T17:57:12.075 INFO:teuthology.orchestra.run.smithi007.stdout:158913790000 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6000a1ff0 msgr2=0x7ff6000a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6000a1ff0 0x7ff6000a44b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff60800b5c0 tx=0x7ff60801a040 comp rx=0 tx=0).stop 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 msgr2=0x7ff61819e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 0x7ff61819e080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6100060b0 tx=0x7ff61000d3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 shutdown_connections 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff6000a1ff0 0x7ff6000a44b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff618106500 0x7ff6181a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff618069800 0x7ff61819e080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 --2- 172.21.15.7:0/300286578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff618104010 0x7ff61819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 >> 172.21.15.7:0/300286578 conn(0x7ff618077440 msgr2=0x7ff618102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.076+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 shutdown_connections 2024-01-11T17:57:12.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.077+0000 7ff61dfbb700 1 -- 172.21.15.7:0/300286578 wait complete. 2024-01-11T17:57:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.478+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3686401915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208108be0 msgr2=0x7fb208109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.478+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3686401915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208108be0 0x7fb208109060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009990 tx=0x7fb1fc009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3686401915 shutdown_connections 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3686401915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208116740 0x7fb208118b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3686401915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 0x7fb208108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3686401915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208108be0 0x7fb208109060 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3686401915 >> 172.21.15.7:0/3686401915 conn(0x7fb20806cb30 msgr2=0x7fb20806cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3686401915 shutdown_connections 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.479+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3686401915 wait complete. 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 Processor -- start 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 -- start start 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208108be0 0x7fb2081a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 0x7fb2081a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208116740 0x7fb2081a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081a7340 con 0x7fb208108be0 2024-01-11T17:57:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081a74b0 con 0x7fb20810bdc0 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.481+0000 7fb20d0b6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081a7670 con 0x7fb208116740 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208108be0 0x7fb2081a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208116740 0x7fb2081a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208108be0 0x7fb2081a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51894/0 (socket says 172.21.15.7:51894) 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb206d9d700 1 -- 172.21.15.7:0/3605882570 learned_addr learned my addr 172.21.15.7:0/3605882570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20659c700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 0x7fb2081a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20659c700 1 -- 172.21.15.7:0/3605882570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208116740 msgr2=0x7fb2081a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20659c700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208116740 0x7fb2081a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20659c700 1 -- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208108be0 msgr2=0x7fb2081a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20659c700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208108be0 0x7fb2081a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.482+0000 7fb20659c700 1 -- 172.21.15.7:0/3605882570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc0095f0 con 0x7fb20810bdc0 2024-01-11T17:57:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.483+0000 7fb20659c700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 0x7fb2081a29a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0098e0 tx=0x7fb1fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.483+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc01d070 con 0x7fb20810bdc0 2024-01-11T17:57:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.483+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2081a78f0 con 0x7fb20810bdc0 2024-01-11T17:57:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.483+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1fc022470 con 0x7fb20810bdc0 2024-01-11T17:57:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.483+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc017650 con 0x7fb20810bdc0 2024-01-11T17:57:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.483+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2081af1b0 con 0x7fb20810bdc0 2024-01-11T17:57:12.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.485+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20806dca0 con 0x7fb20810bdc0 2024-01-11T17:57:12.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.487+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb1fc0225e0 con 0x7fb20810bdc0 2024-01-11T17:57:12.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.488+0000 7fb1effff700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1f00a1f70 0x7fb1f00a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:12.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.488+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb1fc0cf070 con 0x7fb20810bdc0 2024-01-11T17:57:12.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.488+0000 7fb206d9d700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1f00a1f70 0x7fb1f00a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:12.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.489+0000 7fb206d9d700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1f00a1f70 0x7fb1f00a4430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009d20 tx=0x7fb1f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:12.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.492+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb1fc097d60 con 0x7fb20810bdc0 2024-01-11T17:57:12.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:12 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/300286578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-11T17:57:12.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953520 got 137438953522 for osd.4 2024-01-11T17:57:12.672 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.723+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb2081a37c0 con 0x7fb20810bdc0 2024-01-11T17:57:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.723+0000 7fb1effff700 1 -- 172.21.15.7:0/3605882570 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1fc09b380 con 0x7fb20810bdc0 2024-01-11T17:57:12.725 INFO:teuthology.orchestra.run.smithi007.stdout:107374182454 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1f00a1f70 msgr2=0x7fb1f00a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1f00a1f70 0x7fb1f00a4430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009d20 tx=0x7fb1f8009260 comp rx=0 tx=0).stop 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 msgr2=0x7fb2081a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 0x7fb2081a29a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0098e0 tx=0x7fb1fc004730 comp rx=0 tx=0).stop 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 shutdown_connections 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb1f00a1f70 0x7fb1f00a4430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb208116740 0x7fb2081a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb208108be0 0x7fb2081a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 --2- 172.21.15.7:0/3605882570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20810bdc0 0x7fb2081a29a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 >> 172.21.15.7:0/3605882570 conn(0x7fb20806cb30 msgr2=0x7fb208117eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 shutdown_connections 2024-01-11T17:57:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:12.726+0000 7fb20d0b6700 1 -- 172.21.15.7:0/3605882570 wait complete. 2024-01-11T17:57:12.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:12 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/300286578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 -- 172.21.15.7:0/1383472222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c073840 msgr2=0x7f218c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 --2- 172.21.15.7:0/1383472222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c073840 0x7f218c073c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f217c009990 tx=0x7f217c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 -- 172.21.15.7:0/1383472222 shutdown_connections 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 --2- 172.21.15.7:0/1383472222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c1127e0 0x7f218c114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 --2- 172.21.15.7:0/1383472222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c073840 0x7f218c073c20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 --2- 172.21.15.7:0/1383472222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 0x7f218c1122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 -- 172.21.15.7:0/1383472222 >> 172.21.15.7:0/1383472222 conn(0x7f218c078770 msgr2=0x7f218c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 -- 172.21.15.7:0/1383472222 shutdown_connections 2024-01-11T17:57:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.003+0000 7f2191af5700 1 -- 172.21.15.7:0/1383472222 wait complete. 2024-01-11T17:57:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.005+0000 7f2191af5700 1 Processor -- start 2024-01-11T17:57:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.005+0000 7f2191af5700 1 -- start start 2024-01-11T17:57:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.005+0000 7f2191af5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 0x7f218c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f2191af5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 0x7f218c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f2191af5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c1127e0 0x7f218c1a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f2191af5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c1a30c0 con 0x7f218c1127e0 2024-01-11T17:57:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f2191af5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c1a3230 con 0x7f218c074160 2024-01-11T17:57:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f2191af5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c1a33f0 con 0x7f218c073840 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 0x7f218c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 0x7f218c19e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52704/0 (socket says 172.21.15.7:52704) 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 -- 172.21.15.7:0/3785909061 learned_addr learned my addr 172.21.15.7:0/3785909061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218bfff700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c1127e0 0x7f218c1a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218affd700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 0x7f218c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 -- 172.21.15.7:0/3785909061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 msgr2=0x7f218c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 0x7f218c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 -- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c1127e0 msgr2=0x7f218c1a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c1127e0 0x7f218c1a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218b7fe700 1 -- 172.21.15.7:0/3785909061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0095f0 con 0x7f218c073840 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.006+0000 7f218affd700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 0x7f218c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.007+0000 7f218b7fe700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 0x7f218c19e1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f217c005910 tx=0x7f217c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.007+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c01d070 con 0x7f218c073840 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.007+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c1a3670 con 0x7f218c073840 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.007+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f217c022470 con 0x7f218c073840 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.007+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c017650 con 0x7f218c073840 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.007+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c1aaf30 con 0x7f218c073840 2024-01-11T17:57:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.008+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c074f70 con 0x7f218c073840 2024-01-11T17:57:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.010+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f217c0225e0 con 0x7f218c073840 2024-01-11T17:57:13.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.012+0000 7f2188ff9700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21740a1fd0 0x7f21740a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.012+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f217c098600 con 0x7f218c073840 2024-01-11T17:57:13.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.012+0000 7f218affd700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21740a1fd0 0x7f21740a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.012+0000 7f218affd700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21740a1fd0 0x7f21740a4490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f218c10f130 tx=0x7f2180009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.014+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f217c0981d0 con 0x7f218c073840 2024-01-11T17:57:13.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:12 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/300286578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-11T17:57:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 -- 172.21.15.7:0/1045577996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 msgr2=0x7f6e40069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 --2- 172.21.15.7:0/1045577996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e40069740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e340099c0 tx=0x7f6e34009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 -- 172.21.15.7:0/1045577996 shutdown_connections 2024-01-11T17:57:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 --2- 172.21.15.7:0/1045577996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 0x7f6e4010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 --2- 172.21.15.7:0/1045577996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40107640 0x7f6e40107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 --2- 172.21.15.7:0/1045577996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e40069740 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.053+0000 7f6e46750700 1 -- 172.21.15.7:0/1045577996 >> 172.21.15.7:0/1045577996 conn(0x7f6e40077440 msgr2=0x7f6e40077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.054+0000 7f6e46750700 1 -- 172.21.15.7:0/1045577996 shutdown_connections 2024-01-11T17:57:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.054+0000 7f6e46750700 1 -- 172.21.15.7:0/1045577996 wait complete. 2024-01-11T17:57:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 Processor -- start 2024-01-11T17:57:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 -- start start 2024-01-11T17:57:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e4019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 0x7f6e4019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40107640 0x7f6e4019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019eab0 con 0x7f6e400692c0 2024-01-11T17:57:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019ec20 con 0x7f6e40107640 2024-01-11T17:57:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.055+0000 7f6e46750700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019ede0 con 0x7f6e40069c80 2024-01-11T17:57:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e4019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e4019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51926/0 (socket says 172.21.15.7:51926) 2024-01-11T17:57:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3ffff700 1 -- 172.21.15.7:0/78182408 learned_addr learned my addr 172.21.15.7:0/78182408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 0x7f6e4019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 -- 172.21.15.7:0/78182408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40107640 msgr2=0x7f6e4019e4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T17:57:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40107640 0x7f6e4019e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 -- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 msgr2=0x7f6e4019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e4019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 -- 172.21.15.7:0/78182408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340095f0 con 0x7f6e40069c80 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.056+0000 7f6e3f7fe700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 0x7f6e4019a770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098e0 tx=0x7f6e3400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.057+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3401c070 con 0x7f6e40069c80 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.057+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4019f060 con 0x7f6e40069c80 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.057+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e401a6920 con 0x7f6e40069c80 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.057+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e3400fe60 con 0x7f6e40069c80 2024-01-11T17:57:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.057+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340173f0 con 0x7f6e40069c80 2024-01-11T17:57:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.058+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c005330 con 0x7f6e40069c80 2024-01-11T17:57:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.061+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f6e34020070 con 0x7f6e40069c80 2024-01-11T17:57:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.062+0000 7f6e3d7fa700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6e280a1f70 0x7f6e280a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.063+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6e340ce840 con 0x7f6e40069c80 2024-01-11T17:57:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.063+0000 7f6e3ffff700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6e280a1f70 0x7f6e280a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:13.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.064+0000 7f6e3ffff700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6e280a1f70 0x7f6e280a4430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009c90 tx=0x7f6e30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.067+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6e340974a0 con 0x7f6e40069c80 2024-01-11T17:57:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.242+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f218c0682d0 con 0x7f218c073840 2024-01-11T17:57:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.243+0000 7f2188ff9700 1 -- 172.21.15.7:0/3785909061 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f217c09be90 con 0x7f218c073840 2024-01-11T17:57:13.245 INFO:teuthology.orchestra.run.smithi007.stdout:77309411387 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21740a1fd0 msgr2=0x7f21740a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21740a1fd0 0x7f21740a4490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f218c10f130 tx=0x7f2180009260 comp rx=0 tx=0).stop 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 msgr2=0x7f218c19e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 0x7f218c19e1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f217c005910 tx=0x7f217c004730 comp rx=0 tx=0).stop 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 shutdown_connections 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21740a1fd0 0x7f21740a4490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f218c073840 0x7f218c19e1e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f218c1127e0 0x7f218c1a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 --2- 172.21.15.7:0/3785909061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c074160 0x7f218c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 >> 172.21.15.7:0/3785909061 conn(0x7f218c078770 msgr2=0x7f218c111610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 shutdown_connections 2024-01-11T17:57:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.246+0000 7f2191af5700 1 -- 172.21.15.7:0/3785909061 wait complete. 2024-01-11T17:57:13.313 INFO:teuthology.orchestra.run.smithi007.stdout:240518168611 2024-01-11T17:57:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.298+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f6e2c0058b0 con 0x7f6e40069c80 2024-01-11T17:57:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.299+0000 7f6e3d7fa700 1 -- 172.21.15.7:0/78182408 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e3400f9a0 con 0x7f6e40069c80 2024-01-11T17:57:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6e280a1f70 msgr2=0x7f6e280a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6e280a1f70 0x7f6e280a4430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009c90 tx=0x7f6e30009260 comp rx=0 tx=0).stop 2024-01-11T17:57:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 msgr2=0x7f6e4019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 0x7f6e4019a770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098e0 tx=0x7f6e3400f740 comp rx=0 tx=0).stop 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 shutdown_connections 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6e280a1f70 0x7f6e280a4430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40107640 0x7f6e4019e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.301+0000 7f6e46750700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e400692c0 0x7f6e4019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.302+0000 7f6e46750700 1 --2- 172.21.15.7:0/78182408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e40069c80 0x7f6e4019a770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.302+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 >> 172.21.15.7:0/78182408 conn(0x7f6e40077440 msgr2=0x7f6e400fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.302+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 shutdown_connections 2024-01-11T17:57:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:13.302+0000 7f6e46750700 1 -- 172.21.15.7:0/78182408 wait complete. 2024-01-11T17:57:13.363 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182452 got 107374182454 for osd.3 2024-01-11T17:57:13.363 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:13 smithi007 ceph-mon[33361]: pgmap v280: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3605882570' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-11T17:57:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3785909061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-11T17:57:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/78182408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-11T17:57:13.693 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789998 got 158913790000 for osd.5 2024-01-11T17:57:13.693 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:13 smithi016 ceph-mon[43195]: pgmap v280: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:13.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:13 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3605882570' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-11T17:57:13.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:13 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3785909061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-11T17:57:13.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:13 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/78182408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-11T17:57:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:13 smithi038 ceph-mon[43582]: pgmap v280: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:13 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3605882570' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-11T17:57:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:13 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3785909061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-11T17:57:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:13 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/78182408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-11T17:57:14.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168609 got 240518168611 for osd.8 2024-01-11T17:57:14.168 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:14.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411384 got 77309411387 for osd.2 2024-01-11T17:57:14.341 DEBUG:teuthology.parallel:result is None 2024-01-11T17:57:14.341 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-01-11T17:57:14.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph pg dump --format=json 2024-01-11T17:57:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:15 smithi016 ceph-mon[43195]: pgmap v281: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:15.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:15 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:15.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:15 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:15.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:15 smithi038 ceph-mon[43582]: pgmap v281: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:15.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:15 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:15.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:15 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:15.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:15 smithi007 ceph-mon[33361]: pgmap v281: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-01-11T17:57:15.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:15 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:15.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:15 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 -- 172.21.15.7:0/3881365570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388101660 msgr2=0x7f5388101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 --2- 172.21.15.7:0/3881365570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388101660 0x7f5388101ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f537c009990 tx=0x7f537c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 -- 172.21.15.7:0/3881365570 shutdown_connections 2024-01-11T17:57:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 --2- 172.21.15.7:0/3881365570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388107650 0x7f5388107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 --2- 172.21.15.7:0/3881365570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388102020 0x7f538810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 --2- 172.21.15.7:0/3881365570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388101660 0x7f5388101ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 -- 172.21.15.7:0/3881365570 >> 172.21.15.7:0/3881365570 conn(0x7f5388078770 msgr2=0x7f5388078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 -- 172.21.15.7:0/3881365570 shutdown_connections 2024-01-11T17:57:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.032+0000 7f538e82c700 1 -- 172.21.15.7:0/3881365570 wait complete. 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.033+0000 7f538e82c700 1 Processor -- start 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.033+0000 7f538e82c700 1 -- start start 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538e82c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 0x7f5388199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538e82c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388102020 0x7f538819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538e82c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388107650 0x7f538819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538e82c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538819ec50 con 0x7f5388102020 2024-01-11T17:57:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538e82c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538819edc0 con 0x7f5388107650 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538e82c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538819ef80 con 0x7f5388101660 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 0x7f5388199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 0x7f5388199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52736/0 (socket says 172.21.15.7:52736) 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 -- 172.21.15.7:0/1110910050 learned_addr learned my addr 172.21.15.7:0/1110910050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f53877fe700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388102020 0x7f538819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f538cdc9700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388107650 0x7f538819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 -- 172.21.15.7:0/1110910050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388107650 msgr2=0x7f538819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388107650 0x7f538819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 -- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388102020 msgr2=0x7f538819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388102020 0x7f538819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.034+0000 7f5387fff700 1 -- 172.21.15.7:0/1110910050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0095f0 con 0x7f5388101660 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.035+0000 7f5387fff700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 0x7f5388199d70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f537800e9a0 tx=0x7f537800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.035+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378009790 con 0x7f5388101660 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.035+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537800cbd0 con 0x7f5388101660 2024-01-11T17:57:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.035+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538819f260 con 0x7f5388101660 2024-01-11T17:57:16.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.035+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378018680 con 0x7f5388101660 2024-01-11T17:57:16.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.035+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53881a6ac0 con 0x7f5388101660 2024-01-11T17:57:16.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.036+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880682d0 con 0x7f5388101660 2024-01-11T17:57:16.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.038+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f537800cd40 con 0x7f5388101660 2024-01-11T17:57:16.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.039+0000 7f53857fa700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53700a1f70 0x7f53700a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:16.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.040+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5378014070 con 0x7f5388101660 2024-01-11T17:57:16.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.040+0000 7f53877fe700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53700a1f70 0x7f53700a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.040+0000 7f53877fe700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53700a1f70 0x7f53700a4430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f537c0098e0 tx=0x7f537c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:16.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.044+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f5378052380 con 0x7f5388101660 2024-01-11T17:57:16.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.262+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53881115e0 con 0x7f53700a1f70 2024-01-11T17:57:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.276+0000 7f53857fa700 1 -- 172.21.15.7:0/1110910050 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+425534 (secure 0 0 0) 0x7f53881115e0 con 0x7f53700a1f70 2024-01-11T17:57:16.285 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53700a1f70 msgr2=0x7f53700a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53700a1f70 0x7f53700a4430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f537c0098e0 tx=0x7f537c005960 comp rx=0 tx=0).stop 2024-01-11T17:57:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 msgr2=0x7f5388199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 0x7f5388199d70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f537800e9a0 tx=0x7f537800ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 shutdown_connections 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53700a1f70 0x7f53700a4430 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5388101660 0x7f5388199d70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388107650 0x7f538819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 --2- 172.21.15.7:0/1110910050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388102020 0x7f538819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 >> 172.21.15.7:0/1110910050 conn(0x7f5388078770 msgr2=0x7f538810c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.281+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 shutdown_connections 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:16.282+0000 7f538e82c700 1 -- 172.21.15.7:0/1110910050 wait complete. 2024-01-11T17:57:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:57:16.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:57:16.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:57:16.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-01-11T17:57:14.583198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658346,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":378,"num_read_kb":305,"num_write":338,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385592,"kb_used_data":11480,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556232,"statfs":{"total":1151940427776,"available":1151545581568,"internally_reserved":0,"allocated":11755520,"data_stored":7628144,"data_compressed":325332,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":32,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003593"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338271+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:56:25.338271+0000","last_peered":"2024-01-11T17:56:25.338271+0000","last_clean":"2024-01-11T17:56:25.338271+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:56:25.338271+0000","last_undegraded":"2024-01-11T17:56:25.338271+0000","last_fullsized":"2024-01-11T17:56:25.338271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336434+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:56:25.336434+0000","last_peered":"2024-01-11T17:56:25.336434+0000","last_clean":"2024-01-11T17:56:25.336434+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:56:25.336434+0000","last_undegraded":"2024-01-11T17:56:25.336434+0000","last_fullsized":"2024-01-11T17:56:25.336434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336251+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:56:25.336251+0000","last_peered":"2024-01-11T17:56:25.336251+0000","last_clean":"2024-01-11T17:56:25.336251+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:56:25.336251+0000","last_undegraded":"2024-01-11T17:56:25.336251+0000","last_fullsized":"2024-01-11T17:56:25.336251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332674+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:56:25.332674+0000","last_peered":"2024-01-11T17:56:25.332674+0000","last_clean":"2024-01-11T17:56:25.332674+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:56:25.332674+0000","last_undegraded":"2024-01-11T17:56:25.332674+0000","last_fullsized":"2024-01-11T17:56:25.332674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355950+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:56:26.355950+0000","last_peered":"2024-01-11T17:56:26.355950+0000","last_clean":"2024-01-11T17:56:26.355950+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:56:26.355950+0000","last_undegraded":"2024-01-11T17:56:26.355950+0000","last_fullsized":"2024-01-11T17:56:26.355950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355667+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:56:26.355667+0000","last_peered":"2024-01-11T17:56:26.355667+0000","last_clean":"2024-01-11T17:56:26.355667+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:56:26.355667+0000","last_undegraded":"2024-01-11T17:56:26.355667+0000","last_fullsized":"2024-01-11T17:56:26.355667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341507+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:56:25.341507+0000","last_peered":"2024-01-11T17:56:25.341507+0000","last_clean":"2024-01-11T17:56:25.341507+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:56:25.341507+0000","last_undegraded":"2024-01-11T17:56:25.341507+0000","last_fullsized":"2024-01-11T17:56:25.341507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338751+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:56:25.338751+0000","last_peered":"2024-01-11T17:56:25.338751+0000","last_clean":"2024-01-11T17:56:25.338751+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:56:25.338751+0000","last_undegraded":"2024-01-11T17:56:25.338751+0000","last_fullsized":"2024-01-11T17:56:25.338751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338413+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:56:25.338413+0000","last_peered":"2024-01-11T17:56:25.338413+0000","last_clean":"2024-01-11T17:56:25.338413+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:56:25.338413+0000","last_undegraded":"2024-01-11T17:56:25.338413+0000","last_fullsized":"2024-01-11T17:56:25.338413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335441+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:56:25.335441+0000","last_peered":"2024-01-11T17:56:25.335441+0000","last_clean":"2024-01-11T17:56:25.335441+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:56:25.335441+0000","last_undegraded":"2024-01-11T17:56:25.335441+0000","last_fullsized":"2024-01-11T17:56:25.335441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341624+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:56:25.341624+0000","last_peered":"2024-01-11T17:56:25.341624+0000","last_clean":"2024-01-11T17:56:25.341624+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:56:25.341624+0000","last_undegraded":"2024-01-11T17:56:25.341624+0000","last_fullsized":"2024-01-11T17:56:25.341624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332662+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:56:25.332662+0000","last_peered":"2024-01-11T17:56:25.332662+0000","last_clean":"2024-01-11T17:56:25.332662+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:56:25.332662+0000","last_undegraded":"2024-01-11T17:56:25.332662+0000","last_fullsized":"2024-01-11T17:56:25.332662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332139+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:56:25.332139+0000","last_peered":"2024-01-11T17:56:25.332139+0000","last_clean":"2024-01-11T17:56:25.332139+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:56:25.332139+0000","last_undegraded":"2024-01-11T17:56:25.332139+0000","last_fullsized":"2024-01-11T17:56:25.332139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335487+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:56:25.335487+0000","last_peered":"2024-01-11T17:56:25.335487+0000","last_clean":"2024-01-11T17:56:25.335487+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:56:25.335487+0000","last_undegraded":"2024-01-11T17:56:25.335487+0000","last_fullsized":"2024-01-11T17:56:25.335487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355728+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:56:26.355728+0000","last_peered":"2024-01-11T17:56:26.355728+0000","last_clean":"2024-01-11T17:56:26.355728+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:56:26.355728+0000","last_undegraded":"2024-01-11T17:56:26.355728+0000","last_fullsized":"2024-01-11T17:56:26.355728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337407+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:56:25.337407+0000","last_peered":"2024-01-11T17:56:25.337407+0000","last_clean":"2024-01-11T17:56:25.337407+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:56:25.337407+0000","last_undegraded":"2024-01-11T17:56:25.337407+0000","last_fullsized":"2024-01-11T17:56:25.337407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337914+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:56:25.337914+0000","last_peered":"2024-01-11T17:56:25.337914+0000","last_clean":"2024-01-11T17:56:25.337914+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:56:25.337914+0000","last_undegraded":"2024-01-11T17:56:25.337914+0000","last_fullsized":"2024-01-11T17:56:25.337914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336121+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:56:25.336121+0000","last_peered":"2024-01-11T17:56:25.336121+0000","last_clean":"2024-01-11T17:56:25.336121+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:56:25.336121+0000","last_undegraded":"2024-01-11T17:56:25.336121+0000","last_fullsized":"2024-01-11T17:56:25.336121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338989+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:56:25.338989+0000","last_peered":"2024-01-11T17:56:25.338989+0000","last_clean":"2024-01-11T17:56:25.338989+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:56:25.338989+0000","last_undegraded":"2024-01-11T17:56:25.338989+0000","last_fullsized":"2024-01-11T17:56:25.338989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340885+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:56:25.340885+0000","last_peered":"2024-01-11T17:56:25.340885+0000","last_clean":"2024-01-11T17:56:25.340885+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:56:25.340885+0000","last_undegraded":"2024-01-11T17:56:25.340885+0000","last_fullsized":"2024-01-11T17:56:25.340885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397873+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:56:33.397873+0000","last_peered":"2024-01-11T17:56:33.397873+0000","last_clean":"2024-01-11T17:56:33.397873+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:56:33.397873+0000","last_undegraded":"2024-01-11T17:56:33.397873+0000","last_fullsized":"2024-01-11T17:56:33.397873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339633+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:56:25.339633+0000","last_peered":"2024-01-11T17:56:25.339633+0000","last_clean":"2024-01-11T17:56:25.339633+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:56:25.339633+0000","last_undegraded":"2024-01-11T17:56:25.339633+0000","last_fullsized":"2024-01-11T17:56:25.339633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331322+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:56:25.331322+0000","last_peered":"2024-01-11T17:56:25.331322+0000","last_clean":"2024-01-11T17:56:25.331322+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:56:25.331322+0000","last_undegraded":"2024-01-11T17:56:25.331322+0000","last_fullsized":"2024-01-11T17:56:25.331322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.333494+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:56:26.333494+0000","last_peered":"2024-01-11T17:56:26.333494+0000","last_clean":"2024-01-11T17:56:26.333494+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:56:26.333494+0000","last_undegraded":"2024-01-11T17:56:26.333494+0000","last_fullsized":"2024-01-11T17:56:26.333494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336511+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:56:25.336511+0000","last_peered":"2024-01-11T17:56:25.336511+0000","last_clean":"2024-01-11T17:56:25.336511+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:56:25.336511+0000","last_undegraded":"2024-01-11T17:56:25.336511+0000","last_fullsized":"2024-01-11T17:56:25.336511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331455+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:56:25.331455+0000","last_peered":"2024-01-11T17:56:25.331455+0000","last_clean":"2024-01-11T17:56:25.331455+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:56:25.331455+0000","last_undegraded":"2024-01-11T17:56:25.331455+0000","last_fullsized":"2024-01-11T17:56:25.331455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395899+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:56:33.395899+0000","last_peered":"2024-01-11T17:56:33.395899+0000","last_clean":"2024-01-11T17:56:33.395899+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:56:33.395899+0000","last_undegraded":"2024-01-11T17:56:33.395899+0000","last_fullsized":"2024-01-11T17:56:33.395899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337771+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:56:25.337771+0000","last_peered":"2024-01-11T17:56:25.337771+0000","last_clean":"2024-01-11T17:56:25.337771+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:56:25.337771+0000","last_undegraded":"2024-01-11T17:56:25.337771+0000","last_fullsized":"2024-01-11T17:56:25.337771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331648+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:56:25.331648+0000","last_peered":"2024-01-11T17:56:25.331648+0000","last_clean":"2024-01-11T17:56:25.331648+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:56:25.331648+0000","last_undegraded":"2024-01-11T17:56:25.331648+0000","last_fullsized":"2024-01-11T17:56:25.331648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.351148+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:56:26.351148+0000","last_peered":"2024-01-11T17:56:26.351148+0000","last_clean":"2024-01-11T17:56:26.351148+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:56:26.351148+0000","last_undegraded":"2024-01-11T17:56:26.351148+0000","last_fullsized":"2024-01-11T17:56:26.351148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335605+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:56:25.335605+0000","last_peered":"2024-01-11T17:56:25.335605+0000","last_clean":"2024-01-11T17:56:25.335605+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:56:25.335605+0000","last_undegraded":"2024-01-11T17:56:25.335605+0000","last_fullsized":"2024-01-11T17:56:25.335605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338781+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:56:25.338781+0000","last_peered":"2024-01-11T17:56:25.338781+0000","last_clean":"2024-01-11T17:56:25.338781+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:56:25.338781+0000","last_undegraded":"2024-01-11T17:56:25.338781+0000","last_fullsized":"2024-01-11T17:56:25.338781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:13.387002+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:57:13.387002+0000","last_peered":"2024-01-11T17:57:13.387002+0000","last_clean":"2024-01-11T17:57:13.387002+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:57:13.387002+0000","last_undegraded":"2024-01-11T17:57:13.387002+0000","last_fullsized":"2024-01-11T17:57:13.387002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335788+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:56:25.335788+0000","last_peered":"2024-01-11T17:56:25.335788+0000","last_clean":"2024-01-11T17:56:25.335788+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:56:25.335788+0000","last_undegraded":"2024-01-11T17:56:25.335788+0000","last_fullsized":"2024-01-11T17:56:25.335788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337755+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:56:25.337755+0000","last_peered":"2024-01-11T17:56:25.337755+0000","last_clean":"2024-01-11T17:56:25.337755+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:56:25.337755+0000","last_undegraded":"2024-01-11T17:56:25.337755+0000","last_fullsized":"2024-01-11T17:56:25.337755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395256+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:56:33.395256+0000","last_peered":"2024-01-11T17:56:33.395256+0000","last_clean":"2024-01-11T17:56:33.395256+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:56:33.395256+0000","last_undegraded":"2024-01-11T17:56:33.395256+0000","last_fullsized":"2024-01-11T17:56:33.395256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337679+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:56:25.337679+0000","last_peered":"2024-01-11T17:56:25.337679+0000","last_clean":"2024-01-11T17:56:25.337679+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:56:25.337679+0000","last_undegraded":"2024-01-11T17:56:25.337679+0000","last_fullsized":"2024-01-11T17:56:25.337679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337811+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:56:25.337811+0000","last_peered":"2024-01-11T17:56:25.337811+0000","last_clean":"2024-01-11T17:56:25.337811+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:56:25.337811+0000","last_undegraded":"2024-01-11T17:56:25.337811+0000","last_fullsized":"2024-01-11T17:56:25.337811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397732+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:56:33.397732+0000","last_peered":"2024-01-11T17:56:33.397732+0000","last_clean":"2024-01-11T17:56:33.397732+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:56:33.397732+0000","last_undegraded":"2024-01-11T17:56:33.397732+0000","last_fullsized":"2024-01-11T17:56:33.397732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338662+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:56:25.338662+0000","last_peered":"2024-01-11T17:56:25.338662+0000","last_clean":"2024-01-11T17:56:25.338662+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:56:25.338662+0000","last_undegraded":"2024-01-11T17:56:25.338662+0000","last_fullsized":"2024-01-11T17:56:25.338662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336435+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:56:25.336435+0000","last_peered":"2024-01-11T17:56:25.336435+0000","last_clean":"2024-01-11T17:56:25.336435+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:56:25.336435+0000","last_undegraded":"2024-01-11T17:56:25.336435+0000","last_fullsized":"2024-01-11T17:56:25.336435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.344358+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:56:26.344358+0000","last_peered":"2024-01-11T17:56:26.344358+0000","last_clean":"2024-01-11T17:56:26.344358+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:56:26.344358+0000","last_undegraded":"2024-01-11T17:56:26.344358+0000","last_fullsized":"2024-01-11T17:56:26.344358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335428+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:56:25.335428+0000","last_peered":"2024-01-11T17:56:25.335428+0000","last_clean":"2024-01-11T17:56:25.335428+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:56:25.335428+0000","last_undegraded":"2024-01-11T17:56:25.335428+0000","last_fullsized":"2024-01-11T17:56:25.335428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338974+0000","last_change":"2024-01-11T17:56:12.824145+0000","last_active":"2024-01-11T17:56:25.338974+0000","last_peered":"2024-01-11T17:56:25.338974+0000","last_clean":"2024-01-11T17:56:25.338974+0000","last_became_active":"2024-01-11T17:56:12.823443+0000","last_became_peered":"2024-01-11T17:56:12.823443+0000","last_unstale":"2024-01-11T17:56:25.338974+0000","last_undegraded":"2024-01-11T17:56:25.338974+0000","last_fullsized":"2024-01-11T17:56:25.338974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:34:19.317304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":58,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:08.379771+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:57:08.379771+0000","last_peered":"2024-01-11T17:57:08.379771+0000","last_clean":"2024-01-11T17:57:08.379771+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:57:08.379771+0000","last_undegraded":"2024-01-11T17:57:08.379771+0000","last_fullsized":"2024-01-11T17:57:08.379771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338217+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:56:25.338217+0000","last_peered":"2024-01-11T17:56:25.338217+0000","last_clean":"2024-01-11T17:56:25.338217+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:56:25.338217+0000","last_undegraded":"2024-01-11T17:56:25.338217+0000","last_fullsized":"2024-01-11T17:56:25.338217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334836+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:56:25.334836+0000","last_peered":"2024-01-11T17:56:25.334836+0000","last_clean":"2024-01-11T17:56:25.334836+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:56:25.334836+0000","last_undegraded":"2024-01-11T17:56:25.334836+0000","last_fullsized":"2024-01-11T17:56:25.334836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388337+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:56:32.388337+0000","last_peered":"2024-01-11T17:56:32.388337+0000","last_clean":"2024-01-11T17:56:32.388337+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:56:32.388337+0000","last_undegraded":"2024-01-11T17:56:32.388337+0000","last_fullsized":"2024-01-11T17:56:32.388337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336605+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:56:25.336605+0000","last_peered":"2024-01-11T17:56:25.336605+0000","last_clean":"2024-01-11T17:56:25.336605+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:56:25.336605+0000","last_undegraded":"2024-01-11T17:56:25.336605+0000","last_fullsized":"2024-01-11T17:56:25.336605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331894+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:56:25.331894+0000","last_peered":"2024-01-11T17:56:25.331894+0000","last_clean":"2024-01-11T17:56:25.331894+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:56:25.331894+0000","last_undegraded":"2024-01-11T17:56:25.331894+0000","last_fullsized":"2024-01-11T17:56:25.331894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391761+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:56:32.391761+0000","last_peered":"2024-01-11T17:56:32.391761+0000","last_clean":"2024-01-11T17:56:32.391761+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:56:32.391761+0000","last_undegraded":"2024-01-11T17:56:32.391761+0000","last_fullsized":"2024-01-11T17:56:32.391761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337845+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:56:25.337845+0000","last_peered":"2024-01-11T17:56:25.337845+0000","last_clean":"2024-01-11T17:56:25.337845+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:56:25.337845+0000","last_undegraded":"2024-01-11T17:56:25.337845+0000","last_fullsized":"2024-01-11T17:56:25.337845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336863+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:56:25.336863+0000","last_peered":"2024-01-11T17:56:25.336863+0000","last_clean":"2024-01-11T17:56:25.336863+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:56:25.336863+0000","last_undegraded":"2024-01-11T17:56:25.336863+0000","last_fullsized":"2024-01-11T17:56:25.336863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'32","reported_seq":132,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:09.380977+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:57:09.380977+0000","last_peered":"2024-01-11T17:57:09.380977+0000","last_clean":"2024-01-11T17:57:09.380977+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:57:09.380977+0000","last_undegraded":"2024-01-11T17:57:09.380977+0000","last_fullsized":"2024-01-11T17:57:09.380977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63354,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":81,"num_read_kb":103,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336092+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:56:25.336092+0000","last_peered":"2024-01-11T17:56:25.336092+0000","last_clean":"2024-01-11T17:56:25.336092+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:56:25.336092+0000","last_undegraded":"2024-01-11T17:56:25.336092+0000","last_fullsized":"2024-01-11T17:56:25.336092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397159+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:56:33.397159+0000","last_peered":"2024-01-11T17:56:33.397159+0000","last_clean":"2024-01-11T17:56:33.397159+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:56:33.397159+0000","last_undegraded":"2024-01-11T17:56:33.397159+0000","last_fullsized":"2024-01-11T17:56:33.397159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331833+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:56:25.331833+0000","last_peered":"2024-01-11T17:56:25.331833+0000","last_clean":"2024-01-11T17:56:25.331833+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:56:25.331833+0000","last_undegraded":"2024-01-11T17:56:25.331833+0000","last_fullsized":"2024-01-11T17:56:25.331833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337590+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:56:25.337590+0000","last_peered":"2024-01-11T17:56:25.337590+0000","last_clean":"2024-01-11T17:56:25.337590+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:56:25.337590+0000","last_undegraded":"2024-01-11T17:56:25.337590+0000","last_fullsized":"2024-01-11T17:56:25.337590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391050+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:56:32.391050+0000","last_peered":"2024-01-11T17:56:32.391050+0000","last_clean":"2024-01-11T17:56:32.391050+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:56:32.391050+0000","last_undegraded":"2024-01-11T17:56:32.391050+0000","last_fullsized":"2024-01-11T17:56:32.391050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341538+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:56:25.341538+0000","last_peered":"2024-01-11T17:56:25.341538+0000","last_clean":"2024-01-11T17:56:25.341538+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:56:25.341538+0000","last_undegraded":"2024-01-11T17:56:25.341538+0000","last_fullsized":"2024-01-11T17:56:25.341538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.343158+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:56:26.343158+0000","last_peered":"2024-01-11T17:56:26.343158+0000","last_clean":"2024-01-11T17:56:26.343158+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:56:26.343158+0000","last_undegraded":"2024-01-11T17:56:26.343158+0000","last_fullsized":"2024-01-11T17:56:26.343158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:09.381662+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:57:09.381662+0000","last_peered":"2024-01-11T17:57:09.381662+0000","last_clean":"2024-01-11T17:57:09.381662+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:57:09.381662+0000","last_undegraded":"2024-01-11T17:57:09.381662+0000","last_fullsized":"2024-01-11T17:57:09.381662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341549+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:56:25.341549+0000","last_peered":"2024-01-11T17:56:25.341549+0000","last_clean":"2024-01-11T17:56:25.341549+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:56:25.341549+0000","last_undegraded":"2024-01-11T17:56:25.341549+0000","last_fullsized":"2024-01-11T17:56:25.341549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331924+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:56:25.331924+0000","last_peered":"2024-01-11T17:56:25.331924+0000","last_clean":"2024-01-11T17:56:25.331924+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:56:25.331924+0000","last_undegraded":"2024-01-11T17:56:25.331924+0000","last_fullsized":"2024-01-11T17:56:25.331924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.398418+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:56:33.398418+0000","last_peered":"2024-01-11T17:56:33.398418+0000","last_clean":"2024-01-11T17:56:33.398418+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:56:33.398418+0000","last_undegraded":"2024-01-11T17:56:33.398418+0000","last_fullsized":"2024-01-11T17:56:33.398418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336728+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:56:25.336728+0000","last_peered":"2024-01-11T17:56:25.336728+0000","last_clean":"2024-01-11T17:56:25.336728+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:56:25.336728+0000","last_undegraded":"2024-01-11T17:56:25.336728+0000","last_fullsized":"2024-01-11T17:56:25.336728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336598+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:56:25.336598+0000","last_peered":"2024-01-11T17:56:25.336598+0000","last_clean":"2024-01-11T17:56:25.336598+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:56:25.336598+0000","last_undegraded":"2024-01-11T17:56:25.336598+0000","last_fullsized":"2024-01-11T17:56:25.336598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324929+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:56:34.324929+0000","last_peered":"2024-01-11T17:56:34.324929+0000","last_clean":"2024-01-11T17:56:34.324929+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:56:34.324929+0000","last_undegraded":"2024-01-11T17:56:34.324929+0000","last_fullsized":"2024-01-11T17:56:34.324929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335418+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:56:25.335418+0000","last_peered":"2024-01-11T17:56:25.335418+0000","last_clean":"2024-01-11T17:56:25.335418+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:56:25.335418+0000","last_undegraded":"2024-01-11T17:56:25.335418+0000","last_fullsized":"2024-01-11T17:56:25.335418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335675+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:56:25.335675+0000","last_peered":"2024-01-11T17:56:25.335675+0000","last_clean":"2024-01-11T17:56:25.335675+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:56:25.335675+0000","last_undegraded":"2024-01-11T17:56:25.335675+0000","last_fullsized":"2024-01-11T17:56:25.335675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337326+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:56:25.337326+0000","last_peered":"2024-01-11T17:56:25.337326+0000","last_clean":"2024-01-11T17:56:25.337326+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:56:25.337326+0000","last_undegraded":"2024-01-11T17:56:25.337326+0000","last_fullsized":"2024-01-11T17:56:25.337326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335306+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:56:25.335306+0000","last_peered":"2024-01-11T17:56:25.335306+0000","last_clean":"2024-01-11T17:56:25.335306+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:56:25.335306+0000","last_undegraded":"2024-01-11T17:56:25.335306+0000","last_fullsized":"2024-01-11T17:56:25.335306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331851+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:56:25.331851+0000","last_peered":"2024-01-11T17:56:25.331851+0000","last_clean":"2024-01-11T17:56:25.331851+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:56:25.331851+0000","last_undegraded":"2024-01-11T17:56:25.331851+0000","last_fullsized":"2024-01-11T17:56:25.331851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338402+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:56:26.338402+0000","last_peered":"2024-01-11T17:56:26.338402+0000","last_clean":"2024-01-11T17:56:26.338402+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:56:26.338402+0000","last_undegraded":"2024-01-11T17:56:26.338402+0000","last_fullsized":"2024-01-11T17:56:26.338402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335017+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:56:25.335017+0000","last_peered":"2024-01-11T17:56:25.335017+0000","last_clean":"2024-01-11T17:56:25.335017+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:56:25.335017+0000","last_undegraded":"2024-01-11T17:56:25.335017+0000","last_fullsized":"2024-01-11T17:56:25.335017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331773+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:56:25.331773+0000","last_peered":"2024-01-11T17:56:25.331773+0000","last_clean":"2024-01-11T17:56:25.331773+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:56:25.331773+0000","last_undegraded":"2024-01-11T17:56:25.331773+0000","last_fullsized":"2024-01-11T17:56:25.331773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338194+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:56:25.338194+0000","last_peered":"2024-01-11T17:56:25.338194+0000","last_clean":"2024-01-11T17:56:25.338194+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:56:25.338194+0000","last_undegraded":"2024-01-11T17:56:25.338194+0000","last_fullsized":"2024-01-11T17:56:25.338194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:09.381235+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:57:09.381235+0000","last_peered":"2024-01-11T17:57:09.381235+0000","last_clean":"2024-01-11T17:57:09.381235+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:57:09.381235+0000","last_undegraded":"2024-01-11T17:57:09.381235+0000","last_fullsized":"2024-01-11T17:57:09.381235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338722+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:56:26.338722+0000","last_peered":"2024-01-11T17:56:26.338722+0000","last_clean":"2024-01-11T17:56:26.338722+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:56:26.338722+0000","last_undegraded":"2024-01-11T17:56:26.338722+0000","last_fullsized":"2024-01-11T17:56:26.338722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338909+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:56:25.338909+0000","last_peered":"2024-01-11T17:56:25.338909+0000","last_clean":"2024-01-11T17:56:25.338909+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:56:25.338909+0000","last_undegraded":"2024-01-11T17:56:25.338909+0000","last_fullsized":"2024-01-11T17:56:25.338909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324345+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:56:34.324345+0000","last_peered":"2024-01-11T17:56:34.324345+0000","last_clean":"2024-01-11T17:56:34.324345+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:56:34.324345+0000","last_undegraded":"2024-01-11T17:56:34.324345+0000","last_fullsized":"2024-01-11T17:56:34.324345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341318+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:56:25.341318+0000","last_peered":"2024-01-11T17:56:25.341318+0000","last_clean":"2024-01-11T17:56:25.341318+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:56:25.341318+0000","last_undegraded":"2024-01-11T17:56:25.341318+0000","last_fullsized":"2024-01-11T17:56:25.341318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331435+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:56:25.331435+0000","last_peered":"2024-01-11T17:56:25.331435+0000","last_clean":"2024-01-11T17:56:25.331435+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:56:25.331435+0000","last_undegraded":"2024-01-11T17:56:25.331435+0000","last_fullsized":"2024-01-11T17:56:25.331435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.394917+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:56:33.394917+0000","last_peered":"2024-01-11T17:56:33.394917+0000","last_clean":"2024-01-11T17:56:33.394917+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:56:33.394917+0000","last_undegraded":"2024-01-11T17:56:33.394917+0000","last_fullsized":"2024-01-11T17:56:33.394917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335272+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:56:25.335272+0000","last_peered":"2024-01-11T17:56:25.335272+0000","last_clean":"2024-01-11T17:56:25.335272+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:56:25.335272+0000","last_undegraded":"2024-01-11T17:56:25.335272+0000","last_fullsized":"2024-01-11T17:56:25.335272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388565+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:56:32.388565+0000","last_peered":"2024-01-11T17:56:32.388565+0000","last_clean":"2024-01-11T17:56:32.388565+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:56:32.388565+0000","last_undegraded":"2024-01-11T17:56:32.388565+0000","last_fullsized":"2024-01-11T17:56:32.388565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332040+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:56:25.332040+0000","last_peered":"2024-01-11T17:56:25.332040+0000","last_clean":"2024-01-11T17:56:25.332040+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:56:25.332040+0000","last_undegraded":"2024-01-11T17:56:25.332040+0000","last_fullsized":"2024-01-11T17:56:25.332040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340003+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:56:25.340003+0000","last_peered":"2024-01-11T17:56:25.340003+0000","last_clean":"2024-01-11T17:56:25.340003+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:56:25.340003+0000","last_undegraded":"2024-01-11T17:56:25.340003+0000","last_fullsized":"2024-01-11T17:56:25.340003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:38.268914+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:56:38.268914+0000","last_peered":"2024-01-11T17:56:38.268914+0000","last_clean":"2024-01-11T17:56:38.268914+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:56:38.268914+0000","last_undegraded":"2024-01-11T17:56:38.268914+0000","last_fullsized":"2024-01-11T17:56:38.268914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339772+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:56:25.339772+0000","last_peered":"2024-01-11T17:56:25.339772+0000","last_clean":"2024-01-11T17:56:25.339772+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:56:25.339772+0000","last_undegraded":"2024-01-11T17:56:25.339772+0000","last_fullsized":"2024-01-11T17:56:25.339772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340165+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:56:25.340165+0000","last_peered":"2024-01-11T17:56:25.340165+0000","last_clean":"2024-01-11T17:56:25.340165+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:56:25.340165+0000","last_undegraded":"2024-01-11T17:56:25.340165+0000","last_fullsized":"2024-01-11T17:56:25.340165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.386925+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:56:32.386925+0000","last_peered":"2024-01-11T17:56:32.386925+0000","last_clean":"2024-01-11T17:56:32.386925+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:56:32.386925+0000","last_undegraded":"2024-01-11T17:56:32.386925+0000","last_fullsized":"2024-01-11T17:56:32.386925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336540+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:56:25.336540+0000","last_peered":"2024-01-11T17:56:25.336540+0000","last_clean":"2024-01-11T17:56:25.336540+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:56:25.336540+0000","last_undegraded":"2024-01-11T17:56:25.336540+0000","last_fullsized":"2024-01-11T17:56:25.336540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.346059+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:56:26.346059+0000","last_peered":"2024-01-11T17:56:26.346059+0000","last_clean":"2024-01-11T17:56:26.346059+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:56:26.346059+0000","last_undegraded":"2024-01-11T17:56:26.346059+0000","last_fullsized":"2024-01-11T17:56:26.346059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388647+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:56:32.388647+0000","last_peered":"2024-01-11T17:56:32.388647+0000","last_clean":"2024-01-11T17:56:32.388647+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:56:32.388647+0000","last_undegraded":"2024-01-11T17:56:32.388647+0000","last_fullsized":"2024-01-11T17:56:32.388647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334285+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:56:25.334285+0000","last_peered":"2024-01-11T17:56:25.334285+0000","last_clean":"2024-01-11T17:56:25.334285+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:56:25.334285+0000","last_undegraded":"2024-01-11T17:56:25.334285+0000","last_fullsized":"2024-01-11T17:56:25.334285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338275+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:56:25.338275+0000","last_peered":"2024-01-11T17:56:25.338275+0000","last_clean":"2024-01-11T17:56:25.338275+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:56:25.338275+0000","last_undegraded":"2024-01-11T17:56:25.338275+0000","last_fullsized":"2024-01-11T17:56:25.338275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389005+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:56:32.389005+0000","last_peered":"2024-01-11T17:56:32.389005+0000","last_clean":"2024-01-11T17:56:32.389005+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:56:32.389005+0000","last_undegraded":"2024-01-11T17:56:32.389005+0000","last_fullsized":"2024-01-11T17:56:32.389005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334931+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:56:25.334931+0000","last_peered":"2024-01-11T17:56:25.334931+0000","last_clean":"2024-01-11T17:56:25.334931+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:56:25.334931+0000","last_undegraded":"2024-01-11T17:56:25.334931+0000","last_fullsized":"2024-01-11T17:56:25.334931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.333647+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:56:26.333647+0000","last_peered":"2024-01-11T17:56:26.333647+0000","last_clean":"2024-01-11T17:56:26.333647+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:56:26.333647+0000","last_undegraded":"2024-01-11T17:56:26.333647+0000","last_fullsized":"2024-01-11T17:56:26.333647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:29.433711+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:56:29.433711+0000","last_peered":"2024-01-11T17:56:29.433711+0000","last_clean":"2024-01-11T17:56:29.433711+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:56:29.433711+0000","last_undegraded":"2024-01-11T17:56:29.433711+0000","last_fullsized":"2024-01-11T17:56:29.433711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339412+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:56:25.339412+0000","last_peered":"2024-01-11T17:56:25.339412+0000","last_clean":"2024-01-11T17:56:25.339412+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:56:25.339412+0000","last_undegraded":"2024-01-11T17:56:25.339412+0000","last_fullsized":"2024-01-11T17:56:25.339412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331945+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:56:25.331945+0000","last_peered":"2024-01-11T17:56:25.331945+0000","last_clean":"2024-01-11T17:56:25.331945+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:56:25.331945+0000","last_undegraded":"2024-01-11T17:56:25.331945+0000","last_fullsized":"2024-01-11T17:56:25.331945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389594+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:56:32.389594+0000","last_peered":"2024-01-11T17:56:32.389594+0000","last_clean":"2024-01-11T17:56:32.389594+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:56:32.389594+0000","last_undegraded":"2024-01-11T17:56:32.389594+0000","last_fullsized":"2024-01-11T17:56:32.389594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341965+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:56:25.341965+0000","last_peered":"2024-01-11T17:56:25.341965+0000","last_clean":"2024-01-11T17:56:25.341965+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:56:25.341965+0000","last_undegraded":"2024-01-11T17:56:25.341965+0000","last_fullsized":"2024-01-11T17:56:25.341965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337490+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:56:25.337490+0000","last_peered":"2024-01-11T17:56:25.337490+0000","last_clean":"2024-01-11T17:56:25.337490+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:56:25.337490+0000","last_undegraded":"2024-01-11T17:56:25.337490+0000","last_fullsized":"2024-01-11T17:56:25.337490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.396839+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:56:33.396839+0000","last_peered":"2024-01-11T17:56:33.396839+0000","last_clean":"2024-01-11T17:56:33.396839+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:56:33.396839+0000","last_undegraded":"2024-01-11T17:56:33.396839+0000","last_fullsized":"2024-01-11T17:56:33.396839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341851+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:56:25.341851+0000","last_peered":"2024-01-11T17:56:25.341851+0000","last_clean":"2024-01-11T17:56:25.341851+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:56:25.341851+0000","last_undegraded":"2024-01-11T17:56:25.341851+0000","last_fullsized":"2024-01-11T17:56:25.341851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345480+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:56:26.345480+0000","last_peered":"2024-01-11T17:56:26.345480+0000","last_clean":"2024-01-11T17:56:26.345480+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:56:26.345480+0000","last_undegraded":"2024-01-11T17:56:26.345480+0000","last_fullsized":"2024-01-11T17:56:26.345480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345371+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:56:26.345371+0000","last_peered":"2024-01-11T17:56:26.345371+0000","last_clean":"2024-01-11T17:56:26.345371+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:56:26.345371+0000","last_undegraded":"2024-01-11T17:56:26.345371+0000","last_fullsized":"2024-01-11T17:56:26.345371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341451+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:56:25.341451+0000","last_peered":"2024-01-11T17:56:25.341451+0000","last_clean":"2024-01-11T17:56:25.341451+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:56:25.341451+0000","last_undegraded":"2024-01-11T17:56:25.341451+0000","last_fullsized":"2024-01-11T17:56:25.341451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337848+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:56:25.337848+0000","last_peered":"2024-01-11T17:56:25.337848+0000","last_clean":"2024-01-11T17:56:25.337848+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:56:25.337848+0000","last_undegraded":"2024-01-11T17:56:25.337848+0000","last_fullsized":"2024-01-11T17:56:25.337848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332029+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:56:25.332029+0000","last_peered":"2024-01-11T17:56:25.332029+0000","last_clean":"2024-01-11T17:56:25.332029+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:56:25.332029+0000","last_undegraded":"2024-01-11T17:56:25.332029+0000","last_fullsized":"2024-01-11T17:56:25.332029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341977+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:56:25.341977+0000","last_peered":"2024-01-11T17:56:25.341977+0000","last_clean":"2024-01-11T17:56:25.341977+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:56:25.341977+0000","last_undegraded":"2024-01-11T17:56:25.341977+0000","last_fullsized":"2024-01-11T17:56:25.341977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335030+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:56:25.335030+0000","last_peered":"2024-01-11T17:56:25.335030+0000","last_clean":"2024-01-11T17:56:25.335030+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:56:25.335030+0000","last_undegraded":"2024-01-11T17:56:25.335030+0000","last_fullsized":"2024-01-11T17:56:25.335030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355996+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:56:26.355996+0000","last_peered":"2024-01-11T17:56:26.355996+0000","last_clean":"2024-01-11T17:56:26.355996+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:56:26.355996+0000","last_undegraded":"2024-01-11T17:56:26.355996+0000","last_fullsized":"2024-01-11T17:56:26.355996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336409+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:56:25.336409+0000","last_peered":"2024-01-11T17:56:25.336409+0000","last_clean":"2024-01-11T17:56:25.336409+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:56:25.336409+0000","last_undegraded":"2024-01-11T17:56:25.336409+0000","last_fullsized":"2024-01-11T17:56:25.336409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336125+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:56:25.336125+0000","last_peered":"2024-01-11T17:56:25.336125+0000","last_clean":"2024-01-11T17:56:25.336125+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:56:25.336125+0000","last_undegraded":"2024-01-11T17:56:25.336125+0000","last_fullsized":"2024-01-11T17:56:25.336125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332015+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:56:25.332015+0000","last_peered":"2024-01-11T17:56:25.332015+0000","last_clean":"2024-01-11T17:56:25.332015+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:56:25.332015+0000","last_undegraded":"2024-01-11T17:56:25.332015+0000","last_fullsized":"2024-01-11T17:56:25.332015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336086+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:56:25.336086+0000","last_peered":"2024-01-11T17:56:25.336086+0000","last_clean":"2024-01-11T17:56:25.336086+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:56:25.336086+0000","last_undegraded":"2024-01-11T17:56:25.336086+0000","last_fullsized":"2024-01-11T17:56:25.336086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338479+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:56:25.338479+0000","last_peered":"2024-01-11T17:56:25.338479+0000","last_clean":"2024-01-11T17:56:25.338479+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:56:25.338479+0000","last_undegraded":"2024-01-11T17:56:25.338479+0000","last_fullsized":"2024-01-11T17:56:25.338479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341422+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:56:25.341422+0000","last_peered":"2024-01-11T17:56:25.341422+0000","last_clean":"2024-01-11T17:56:25.341422+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:56:25.341422+0000","last_undegraded":"2024-01-11T17:56:25.341422+0000","last_fullsized":"2024-01-11T17:56:25.341422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338107+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:56:25.338107+0000","last_peered":"2024-01-11T17:56:25.338107+0000","last_clean":"2024-01-11T17:56:25.338107+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:56:25.338107+0000","last_undegraded":"2024-01-11T17:56:25.338107+0000","last_fullsized":"2024-01-11T17:56:25.338107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339143+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:56:25.339143+0000","last_peered":"2024-01-11T17:56:25.339143+0000","last_clean":"2024-01-11T17:56:25.339143+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:56:25.339143+0000","last_undegraded":"2024-01-11T17:56:25.339143+0000","last_fullsized":"2024-01-11T17:56:25.339143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337858+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:56:25.337858+0000","last_peered":"2024-01-11T17:56:25.337858+0000","last_clean":"2024-01-11T17:56:25.337858+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:56:25.337858+0000","last_undegraded":"2024-01-11T17:56:25.337858+0000","last_fullsized":"2024-01-11T17:56:25.337858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332218+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:56:25.332218+0000","last_peered":"2024-01-11T17:56:25.332218+0000","last_clean":"2024-01-11T17:56:25.332218+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:56:25.332218+0000","last_undegraded":"2024-01-11T17:56:25.332218+0000","last_fullsized":"2024-01-11T17:56:25.332218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355928+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:56:26.355928+0000","last_peered":"2024-01-11T17:56:26.355928+0000","last_clean":"2024-01-11T17:56:26.355928+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:56:26.355928+0000","last_undegraded":"2024-01-11T17:56:26.355928+0000","last_fullsized":"2024-01-11T17:56:26.355928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67744,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":252,"num_read_kb":196,"num_write":155,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203232,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":473098,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.879}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.86499999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.98899999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.89100000000000001}]}]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":1124110,"data_compressed":55398,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.433}]},{"osd":5,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.75},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.94999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.254,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.097,"15min":1.097},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.117,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.099,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0529999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79000000000000004}]}]},{"osd":9,"up_from":63,"seq":270582939678,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":477430,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82999999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.502}]},{"osd":1,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.42099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.45800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90700000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84799999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82899999999999996,"15min":0.83199999999999996},"min":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.83099999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.04,"5min":1.198,"15min":1.198},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0269999999999999,"5min":1.224,"15min":1.224},"last":0.873}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.92300000000000004,"15min":0.94199999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.079,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":0.98299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.73699999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98999999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.89100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168612,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":473254,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747}]},{"osd":1,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.31}]},{"osd":3,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.502}]},{"osd":4,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94199999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.872,"5min":1.097,"15min":1.097},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78300000000000003,"15min":0.78700000000000003},"min":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94799999999999995,"5min":1,"15min":1},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93300000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.70299999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.89500000000000002,"15min":0.89700000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.871},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.83099999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85599999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.624},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.63800000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397543,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1069552,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.432},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.63400000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.627,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84499999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.86199999999999999,"5min":1.137,"15min":1.137},"last":0.497}]},{"osd":5,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.753},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.65100000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.748},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.004,"5min":1.04,"15min":1.04},"last":0.82899999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0960000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.145,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.85899999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83999999999999997,"15min":0.83199999999999996},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.86799999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":468510,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.504,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72899999999999998,"15min":0.70699999999999996},"min":{"1min":0.59099999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.008,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.64000000000000001,"15min":0.60799999999999998},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.04,"5min":1.161,"15min":1.161},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.645,"5min":2.645,"15min":2.645},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.214},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.59499999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.048,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80900000000000005,"15min":0.80100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89500000000000002,"15min":0.89400000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.292},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.59399999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0669999999999999,"5min":1.131,"15min":1.131},"last":1.3540000000000001}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.407},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0980000000000001,"5min":1.155,"15min":1.155},"last":0.877}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.024,"15min":1.0309999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.341},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92100000000000004,"15min":0.91700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.98699999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.4450000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":531774,"data_compressed":19305,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.03,"5min":1.077,"15min":1.077},"last":0.439},{"interface":"front","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89400000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.442,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.58199999999999996,"15min":0.54400000000000004},"min":{"1min":0.41799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.54200000000000004},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.97299999999999998,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.44400000000000001,"15min":0.40000000000000002},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.48199999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0049999999999999,"5min":1.012,"15min":1.012},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.54200000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.04,"5min":1.048,"15min":1.048},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92700000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.52600000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79800000000000004,"5min":1.048,"15min":1.048},"last":0.746}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.628},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.80800000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.61499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.87},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.72499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.056,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Thu Jan 11 17:56:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.006,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84399999999999997},"min":{"1min":0.66000000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.997,"5min":1.127,"15min":1.127},"last":0.92600000000000005}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.98699999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.008}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":469898,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.55500000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.67300000000000004,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.65800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.508,"5min":3.508,"15min":3.508},"last":0.628}]},{"osd":3,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.70099999999999996,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.71199999999999997,"15min":0.68000000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.68200000000000005,"15min":0.63300000000000001},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.442},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.67300000000000004},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 11 17:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.875,"15min":0.878},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.746},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":0.64400000000000002}]},{"osd":9,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95999999999999996,"15min":0.95599999999999996},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.042,"5min":0.96999999999999997,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.2599999999999998,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.93300000000000005}]},{"osd":10,"last update":"Thu Jan 11 17:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 11 17:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":0.78700000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":533514,"data_compressed":19305,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.753},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60299999999999998,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.075},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.92600000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81499999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.022,"15min":1.022},"last":1.159},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.198}]},{"osd":8,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.25},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.97399999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.78000000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98599999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.111}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.129},"last":0.998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.73499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0469999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0469999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.095}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":468704,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.40100000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.249},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.492},{"interface":"front","average":{"1min":0.502,"5min":0.46700000000000003,"15min":0.42599999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.623,"15min":0.53700000000000003},"min":{"1min":0.56299999999999994,"5min":0.308,"15min":0.308},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.64000000000000001,"15min":0.55400000000000005},"min":{"1min":0.59099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65500000000000003,"15min":0.60499999999999998},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.56699999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.4079999999999999}]},{"osd":4,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.65300000000000002,"15min":0.59399999999999997},"min":{"1min":0.50700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.68700000000000006,"15min":0.628},"min":{"1min":0.57399999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.3680000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.71399999999999997,"15min":0.67800000000000005},"min":{"1min":0.60199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.72099999999999997,"15min":0.68600000000000005},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.343},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.65100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0580000000000001,"5min":1.379,"15min":1.379},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.499,"15min":0.499},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.1970000000000001}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":0.93600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.52,"15min":1.52},"last":1.03}]},{"osd":10,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.2330000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":473550,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.374},"min":{"1min":0.36099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.33600000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.91700000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.379}]},{"osd":3,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.60099999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1,"5min":1.034,"15min":1.034},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.61899999999999999,"15min":0.59099999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.57699999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89800000000000002,"5min":1.129,"15min":1.129},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 11 17:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68400000000000005,"15min":0.65400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96999999999999997,"5min":1.163,"15min":1.163},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.435},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.499,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.44900000000000001}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93200000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.024,"5min":1.238,"15min":1.238},"last":0.75800000000000001}]},{"osd":9,"last update":"Thu Jan 11 17:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1359999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.74399999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.72199999999999998}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79000000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":1064750,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88500000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.76100000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.374},{"interface":"front","average":{"1min":0.5,"5min":0.47599999999999998,"15min":0.45900000000000002},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.439}]},{"osd":3,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.42399999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73299999999999998,"5min":0.999,"15min":0.999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76000000000000001,"5min":1.548,"15min":1.548},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.22,"15min":1.22},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.88300000000000001}]},{"osd":8,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.56499999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93999999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53300000000000003}]},{"osd":9,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":1.097,"15min":1.097},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79700000000000004,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0189999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":63354,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65094,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65322,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:57:16.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph pg dump --format=json 2024-01-11T17:57:16.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:57:16.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:57:16.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:16.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:57:16.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:57:16.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:57:17.112 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:17.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:17 smithi007 ceph-mon[33361]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:57:17.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:17 smithi007 ceph-mon[33361]: pgmap v282: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-01-11T17:57:17.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:17 smithi016 ceph-mon[43195]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:57:17.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:17 smithi016 ceph-mon[43195]: pgmap v282: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-01-11T17:57:17.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:17 smithi038 ceph-mon[43582]: from='client.25078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:57:17.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:17 smithi038 ceph-mon[43582]: pgmap v282: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-01-11T17:57:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 -- 172.21.15.7:0/590890443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 msgr2=0x7f3b7c101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 --2- 172.21.15.7:0/590890443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c101a70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009990 tx=0x7f3b70009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 -- 172.21.15.7:0/590890443 shutdown_connections 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 --2- 172.21.15.7:0/590890443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c101fb0 0x7f3b7c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 --2- 172.21.15.7:0/590890443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b7c107680 0x7f3b7c107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 --2- 172.21.15.7:0/590890443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c101a70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 -- 172.21.15.7:0/590890443 >> 172.21.15.7:0/590890443 conn(0x7f3b7c0fd0c0 msgr2=0x7f3b7c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.268+0000 7f3b81b82700 1 -- 172.21.15.7:0/590890443 shutdown_connections 2024-01-11T17:57:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.269+0000 7f3b81b82700 1 -- 172.21.15.7:0/590890443 wait complete. 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.269+0000 7f3b81b82700 1 Processor -- start 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 -- start start 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b7c101fb0 0x7f3b7c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c107680 0x7f3b7c074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1a5e40 con 0x7f3b7c1015f0 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1a5fb0 con 0x7f3b7c101fb0 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b81b82700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1a6170 con 0x7f3b7c107680 2024-01-11T17:57:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51956/0 (socket says 172.21.15.7:51956) 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.270+0000 7f3b7b7fe700 1 -- 172.21.15.7:0/67321441 learned_addr learned my addr 172.21.15.7:0/67321441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7affd700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b7c101fb0 0x7f3b7c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7bfff700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c107680 0x7f3b7c074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7b7fe700 1 -- 172.21.15.7:0/67321441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c107680 msgr2=0x7f3b7c074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7b7fe700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c107680 0x7f3b7c074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7b7fe700 1 -- 172.21.15.7:0/67321441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b7c101fb0 msgr2=0x7f3b7c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7b7fe700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b7c101fb0 0x7f3b7c073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7b7fe700 1 -- 172.21.15.7:0/67321441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b700095f0 con 0x7f3b7c1015f0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7bfff700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c107680 0x7f3b7c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b7b7fe700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c073910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e970 tx=0x7f3b6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c009790 con 0x7f3b7c1015f0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b6c00eec0 con 0x7f3b7c1015f0 2024-01-11T17:57:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c1a6450 con 0x7f3b7c1015f0 2024-01-11T17:57:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.271+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c003cf0 con 0x7f3b7c1015f0 2024-01-11T17:57:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.272+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c1a69a0 con 0x7f3b7c1015f0 2024-01-11T17:57:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.273+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c109c10 con 0x7f3b7c1015f0 2024-01-11T17:57:18.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.274+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f3b6c00ccd0 con 0x7f3b7c1015f0 2024-01-11T17:57:18.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.276+0000 7f3b78ff9700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3b640a1ff0 0x7f3b640a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:18.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.276+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3b6c014070 con 0x7f3b7c1015f0 2024-01-11T17:57:18.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.276+0000 7f3b7affd700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3b640a1ff0 0x7f3b640a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:18.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.277+0000 7f3b7affd700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3b640a1ff0 0x7f3b640a44b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c106d00 tx=0x7f3b7000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:18.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.280+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3b6c096bd0 con 0x7f3b7c1015f0 2024-01-11T17:57:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.490+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b7c111580 con 0x7f3b640a1ff0 2024-01-11T17:57:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.504+0000 7f3b78ff9700 1 -- 172.21.15.7:0/67321441 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426736 (secure 0 0 0) 0x7f3b7c111580 con 0x7f3b640a1ff0 2024-01-11T17:57:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.509+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3b640a1ff0 msgr2=0x7f3b640a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.509+0000 7f3b81b82700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3b640a1ff0 0x7f3b640a44b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c106d00 tx=0x7f3b7000b260 comp rx=0 tx=0).stop 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.509+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 msgr2=0x7f3b7c073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.509+0000 7f3b81b82700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c073910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00e970 tx=0x7f3b6c00ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 shutdown_connections 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3b640a1ff0 0x7f3b640a44b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b7c107680 0x7f3b7c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b7c1015f0 0x7f3b7c073910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 --2- 172.21.15.7:0/67321441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b7c101fb0 0x7f3b7c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 >> 172.21.15.7:0/67321441 conn(0x7f3b7c0fd0c0 msgr2=0x7f3b7c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 shutdown_connections 2024-01-11T17:57:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:18.510+0000 7f3b81b82700 1 -- 172.21.15.7:0/67321441 wait complete. 2024-01-11T17:57:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:57:18.920 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-01-11T17:57:16.583911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658346,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":403,"num_read_kb":320,"num_write":338,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385592,"kb_used_data":11480,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556232,"statfs":{"total":1151940427776,"available":1151545581568,"internally_reserved":0,"allocated":11755520,"data_stored":7628144,"data_compressed":325332,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":47,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338271+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:56:25.338271+0000","last_peered":"2024-01-11T17:56:25.338271+0000","last_clean":"2024-01-11T17:56:25.338271+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:56:25.338271+0000","last_undegraded":"2024-01-11T17:56:25.338271+0000","last_fullsized":"2024-01-11T17:56:25.338271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336434+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:56:25.336434+0000","last_peered":"2024-01-11T17:56:25.336434+0000","last_clean":"2024-01-11T17:56:25.336434+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:56:25.336434+0000","last_undegraded":"2024-01-11T17:56:25.336434+0000","last_fullsized":"2024-01-11T17:56:25.336434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336251+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:56:25.336251+0000","last_peered":"2024-01-11T17:56:25.336251+0000","last_clean":"2024-01-11T17:56:25.336251+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:56:25.336251+0000","last_undegraded":"2024-01-11T17:56:25.336251+0000","last_fullsized":"2024-01-11T17:56:25.336251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332674+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:56:25.332674+0000","last_peered":"2024-01-11T17:56:25.332674+0000","last_clean":"2024-01-11T17:56:25.332674+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:56:25.332674+0000","last_undegraded":"2024-01-11T17:56:25.332674+0000","last_fullsized":"2024-01-11T17:56:25.332674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355950+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:56:26.355950+0000","last_peered":"2024-01-11T17:56:26.355950+0000","last_clean":"2024-01-11T17:56:26.355950+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:56:26.355950+0000","last_undegraded":"2024-01-11T17:56:26.355950+0000","last_fullsized":"2024-01-11T17:56:26.355950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355667+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:56:26.355667+0000","last_peered":"2024-01-11T17:56:26.355667+0000","last_clean":"2024-01-11T17:56:26.355667+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:56:26.355667+0000","last_undegraded":"2024-01-11T17:56:26.355667+0000","last_fullsized":"2024-01-11T17:56:26.355667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341507+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:56:25.341507+0000","last_peered":"2024-01-11T17:56:25.341507+0000","last_clean":"2024-01-11T17:56:25.341507+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:56:25.341507+0000","last_undegraded":"2024-01-11T17:56:25.341507+0000","last_fullsized":"2024-01-11T17:56:25.341507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338751+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:56:25.338751+0000","last_peered":"2024-01-11T17:56:25.338751+0000","last_clean":"2024-01-11T17:56:25.338751+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:56:25.338751+0000","last_undegraded":"2024-01-11T17:56:25.338751+0000","last_fullsized":"2024-01-11T17:56:25.338751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338413+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:56:25.338413+0000","last_peered":"2024-01-11T17:56:25.338413+0000","last_clean":"2024-01-11T17:56:25.338413+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:56:25.338413+0000","last_undegraded":"2024-01-11T17:56:25.338413+0000","last_fullsized":"2024-01-11T17:56:25.338413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335441+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:56:25.335441+0000","last_peered":"2024-01-11T17:56:25.335441+0000","last_clean":"2024-01-11T17:56:25.335441+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:56:25.335441+0000","last_undegraded":"2024-01-11T17:56:25.335441+0000","last_fullsized":"2024-01-11T17:56:25.335441+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341624+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:56:25.341624+0000","last_peered":"2024-01-11T17:56:25.341624+0000","last_clean":"2024-01-11T17:56:25.341624+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:56:25.341624+0000","last_undegraded":"2024-01-11T17:56:25.341624+0000","last_fullsized":"2024-01-11T17:56:25.341624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332662+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:56:25.332662+0000","last_peered":"2024-01-11T17:56:25.332662+0000","last_clean":"2024-01-11T17:56:25.332662+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:56:25.332662+0000","last_undegraded":"2024-01-11T17:56:25.332662+0000","last_fullsized":"2024-01-11T17:56:25.332662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332139+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:56:25.332139+0000","last_peered":"2024-01-11T17:56:25.332139+0000","last_clean":"2024-01-11T17:56:25.332139+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:56:25.332139+0000","last_undegraded":"2024-01-11T17:56:25.332139+0000","last_fullsized":"2024-01-11T17:56:25.332139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335487+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:56:25.335487+0000","last_peered":"2024-01-11T17:56:25.335487+0000","last_clean":"2024-01-11T17:56:25.335487+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:56:25.335487+0000","last_undegraded":"2024-01-11T17:56:25.335487+0000","last_fullsized":"2024-01-11T17:56:25.335487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355728+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:56:26.355728+0000","last_peered":"2024-01-11T17:56:26.355728+0000","last_clean":"2024-01-11T17:56:26.355728+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:56:26.355728+0000","last_undegraded":"2024-01-11T17:56:26.355728+0000","last_fullsized":"2024-01-11T17:56:26.355728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337407+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:56:25.337407+0000","last_peered":"2024-01-11T17:56:25.337407+0000","last_clean":"2024-01-11T17:56:25.337407+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:56:25.337407+0000","last_undegraded":"2024-01-11T17:56:25.337407+0000","last_fullsized":"2024-01-11T17:56:25.337407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337914+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:56:25.337914+0000","last_peered":"2024-01-11T17:56:25.337914+0000","last_clean":"2024-01-11T17:56:25.337914+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:56:25.337914+0000","last_undegraded":"2024-01-11T17:56:25.337914+0000","last_fullsized":"2024-01-11T17:56:25.337914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336121+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:56:25.336121+0000","last_peered":"2024-01-11T17:56:25.336121+0000","last_clean":"2024-01-11T17:56:25.336121+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:56:25.336121+0000","last_undegraded":"2024-01-11T17:56:25.336121+0000","last_fullsized":"2024-01-11T17:56:25.336121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338989+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:56:25.338989+0000","last_peered":"2024-01-11T17:56:25.338989+0000","last_clean":"2024-01-11T17:56:25.338989+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:56:25.338989+0000","last_undegraded":"2024-01-11T17:56:25.338989+0000","last_fullsized":"2024-01-11T17:56:25.338989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340885+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:56:25.340885+0000","last_peered":"2024-01-11T17:56:25.340885+0000","last_clean":"2024-01-11T17:56:25.340885+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:56:25.340885+0000","last_undegraded":"2024-01-11T17:56:25.340885+0000","last_fullsized":"2024-01-11T17:56:25.340885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397873+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:56:33.397873+0000","last_peered":"2024-01-11T17:56:33.397873+0000","last_clean":"2024-01-11T17:56:33.397873+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:56:33.397873+0000","last_undegraded":"2024-01-11T17:56:33.397873+0000","last_fullsized":"2024-01-11T17:56:33.397873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339633+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:56:25.339633+0000","last_peered":"2024-01-11T17:56:25.339633+0000","last_clean":"2024-01-11T17:56:25.339633+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:56:25.339633+0000","last_undegraded":"2024-01-11T17:56:25.339633+0000","last_fullsized":"2024-01-11T17:56:25.339633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331322+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:56:25.331322+0000","last_peered":"2024-01-11T17:56:25.331322+0000","last_clean":"2024-01-11T17:56:25.331322+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:56:25.331322+0000","last_undegraded":"2024-01-11T17:56:25.331322+0000","last_fullsized":"2024-01-11T17:56:25.331322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.333494+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:56:26.333494+0000","last_peered":"2024-01-11T17:56:26.333494+0000","last_clean":"2024-01-11T17:56:26.333494+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:56:26.333494+0000","last_undegraded":"2024-01-11T17:56:26.333494+0000","last_fullsized":"2024-01-11T17:56:26.333494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336511+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:56:25.336511+0000","last_peered":"2024-01-11T17:56:25.336511+0000","last_clean":"2024-01-11T17:56:25.336511+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:56:25.336511+0000","last_undegraded":"2024-01-11T17:56:25.336511+0000","last_fullsized":"2024-01-11T17:56:25.336511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331455+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:56:25.331455+0000","last_peered":"2024-01-11T17:56:25.331455+0000","last_clean":"2024-01-11T17:56:25.331455+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:56:25.331455+0000","last_undegraded":"2024-01-11T17:56:25.331455+0000","last_fullsized":"2024-01-11T17:56:25.331455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395899+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:56:33.395899+0000","last_peered":"2024-01-11T17:56:33.395899+0000","last_clean":"2024-01-11T17:56:33.395899+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:56:33.395899+0000","last_undegraded":"2024-01-11T17:56:33.395899+0000","last_fullsized":"2024-01-11T17:56:33.395899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337771+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:56:25.337771+0000","last_peered":"2024-01-11T17:56:25.337771+0000","last_clean":"2024-01-11T17:56:25.337771+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:56:25.337771+0000","last_undegraded":"2024-01-11T17:56:25.337771+0000","last_fullsized":"2024-01-11T17:56:25.337771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331648+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:56:25.331648+0000","last_peered":"2024-01-11T17:56:25.331648+0000","last_clean":"2024-01-11T17:56:25.331648+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:56:25.331648+0000","last_undegraded":"2024-01-11T17:56:25.331648+0000","last_fullsized":"2024-01-11T17:56:25.331648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.351148+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:56:26.351148+0000","last_peered":"2024-01-11T17:56:26.351148+0000","last_clean":"2024-01-11T17:56:26.351148+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:56:26.351148+0000","last_undegraded":"2024-01-11T17:56:26.351148+0000","last_fullsized":"2024-01-11T17:56:26.351148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335605+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:56:25.335605+0000","last_peered":"2024-01-11T17:56:25.335605+0000","last_clean":"2024-01-11T17:56:25.335605+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:56:25.335605+0000","last_undegraded":"2024-01-11T17:56:25.335605+0000","last_fullsized":"2024-01-11T17:56:25.335605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338781+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:56:25.338781+0000","last_peered":"2024-01-11T17:56:25.338781+0000","last_clean":"2024-01-11T17:56:25.338781+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:56:25.338781+0000","last_undegraded":"2024-01-11T17:56:25.338781+0000","last_fullsized":"2024-01-11T17:56:25.338781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:13.387002+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:57:13.387002+0000","last_peered":"2024-01-11T17:57:13.387002+0000","last_clean":"2024-01-11T17:57:13.387002+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:57:13.387002+0000","last_undegraded":"2024-01-11T17:57:13.387002+0000","last_fullsized":"2024-01-11T17:57:13.387002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335788+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:56:25.335788+0000","last_peered":"2024-01-11T17:56:25.335788+0000","last_clean":"2024-01-11T17:56:25.335788+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:56:25.335788+0000","last_undegraded":"2024-01-11T17:56:25.335788+0000","last_fullsized":"2024-01-11T17:56:25.335788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337755+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:56:25.337755+0000","last_peered":"2024-01-11T17:56:25.337755+0000","last_clean":"2024-01-11T17:56:25.337755+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:56:25.337755+0000","last_undegraded":"2024-01-11T17:56:25.337755+0000","last_fullsized":"2024-01-11T17:56:25.337755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.395256+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:56:33.395256+0000","last_peered":"2024-01-11T17:56:33.395256+0000","last_clean":"2024-01-11T17:56:33.395256+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:56:33.395256+0000","last_undegraded":"2024-01-11T17:56:33.395256+0000","last_fullsized":"2024-01-11T17:56:33.395256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337679+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:56:25.337679+0000","last_peered":"2024-01-11T17:56:25.337679+0000","last_clean":"2024-01-11T17:56:25.337679+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:56:25.337679+0000","last_undegraded":"2024-01-11T17:56:25.337679+0000","last_fullsized":"2024-01-11T17:56:25.337679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337811+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:56:25.337811+0000","last_peered":"2024-01-11T17:56:25.337811+0000","last_clean":"2024-01-11T17:56:25.337811+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:56:25.337811+0000","last_undegraded":"2024-01-11T17:56:25.337811+0000","last_fullsized":"2024-01-11T17:56:25.337811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397732+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:56:33.397732+0000","last_peered":"2024-01-11T17:56:33.397732+0000","last_clean":"2024-01-11T17:56:33.397732+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:56:33.397732+0000","last_undegraded":"2024-01-11T17:56:33.397732+0000","last_fullsized":"2024-01-11T17:56:33.397732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338662+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:56:25.338662+0000","last_peered":"2024-01-11T17:56:25.338662+0000","last_clean":"2024-01-11T17:56:25.338662+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:56:25.338662+0000","last_undegraded":"2024-01-11T17:56:25.338662+0000","last_fullsized":"2024-01-11T17:56:25.338662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336435+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:56:25.336435+0000","last_peered":"2024-01-11T17:56:25.336435+0000","last_clean":"2024-01-11T17:56:25.336435+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:56:25.336435+0000","last_undegraded":"2024-01-11T17:56:25.336435+0000","last_fullsized":"2024-01-11T17:56:25.336435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.344358+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:56:26.344358+0000","last_peered":"2024-01-11T17:56:26.344358+0000","last_clean":"2024-01-11T17:56:26.344358+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:56:26.344358+0000","last_undegraded":"2024-01-11T17:56:26.344358+0000","last_fullsized":"2024-01-11T17:56:26.344358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335428+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:56:25.335428+0000","last_peered":"2024-01-11T17:56:25.335428+0000","last_clean":"2024-01-11T17:56:25.335428+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:56:25.335428+0000","last_undegraded":"2024-01-11T17:56:25.335428+0000","last_fullsized":"2024-01-11T17:56:25.335428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338974+0000","last_change":"2024-01-11T17:56:12.824145+0000","last_active":"2024-01-11T17:56:25.338974+0000","last_peered":"2024-01-11T17:56:25.338974+0000","last_clean":"2024-01-11T17:56:25.338974+0000","last_became_active":"2024-01-11T17:56:12.823443+0000","last_became_peered":"2024-01-11T17:56:12.823443+0000","last_unstale":"2024-01-11T17:56:25.338974+0000","last_undegraded":"2024-01-11T17:56:25.338974+0000","last_fullsized":"2024-01-11T17:56:25.338974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:34:19.317304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:13.386288+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:57:13.386288+0000","last_peered":"2024-01-11T17:57:13.386288+0000","last_clean":"2024-01-11T17:57:13.386288+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:57:13.386288+0000","last_undegraded":"2024-01-11T17:57:13.386288+0000","last_fullsized":"2024-01-11T17:57:13.386288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338217+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:56:25.338217+0000","last_peered":"2024-01-11T17:56:25.338217+0000","last_clean":"2024-01-11T17:56:25.338217+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:56:25.338217+0000","last_undegraded":"2024-01-11T17:56:25.338217+0000","last_fullsized":"2024-01-11T17:56:25.338217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334836+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:56:25.334836+0000","last_peered":"2024-01-11T17:56:25.334836+0000","last_clean":"2024-01-11T17:56:25.334836+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:56:25.334836+0000","last_undegraded":"2024-01-11T17:56:25.334836+0000","last_fullsized":"2024-01-11T17:56:25.334836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388337+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:56:32.388337+0000","last_peered":"2024-01-11T17:56:32.388337+0000","last_clean":"2024-01-11T17:56:32.388337+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:56:32.388337+0000","last_undegraded":"2024-01-11T17:56:32.388337+0000","last_fullsized":"2024-01-11T17:56:32.388337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336605+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:56:25.336605+0000","last_peered":"2024-01-11T17:56:25.336605+0000","last_clean":"2024-01-11T17:56:25.336605+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:56:25.336605+0000","last_undegraded":"2024-01-11T17:56:25.336605+0000","last_fullsized":"2024-01-11T17:56:25.336605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331894+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:56:25.331894+0000","last_peered":"2024-01-11T17:56:25.331894+0000","last_clean":"2024-01-11T17:56:25.331894+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:56:25.331894+0000","last_undegraded":"2024-01-11T17:56:25.331894+0000","last_fullsized":"2024-01-11T17:56:25.331894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391761+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:56:32.391761+0000","last_peered":"2024-01-11T17:56:32.391761+0000","last_clean":"2024-01-11T17:56:32.391761+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:56:32.391761+0000","last_undegraded":"2024-01-11T17:56:32.391761+0000","last_fullsized":"2024-01-11T17:56:32.391761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337845+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:56:25.337845+0000","last_peered":"2024-01-11T17:56:25.337845+0000","last_clean":"2024-01-11T17:56:25.337845+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:56:25.337845+0000","last_undegraded":"2024-01-11T17:56:25.337845+0000","last_fullsized":"2024-01-11T17:56:25.337845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336863+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:56:25.336863+0000","last_peered":"2024-01-11T17:56:25.336863+0000","last_clean":"2024-01-11T17:56:25.336863+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:56:25.336863+0000","last_undegraded":"2024-01-11T17:56:25.336863+0000","last_fullsized":"2024-01-11T17:56:25.336863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'32","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:14.387567+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:57:14.387567+0000","last_peered":"2024-01-11T17:57:14.387567+0000","last_clean":"2024-01-11T17:57:14.387567+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:57:14.387567+0000","last_undegraded":"2024-01-11T17:57:14.387567+0000","last_fullsized":"2024-01-11T17:57:14.387567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63354,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":91,"num_read_kb":108,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336092+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:56:25.336092+0000","last_peered":"2024-01-11T17:56:25.336092+0000","last_clean":"2024-01-11T17:56:25.336092+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:56:25.336092+0000","last_undegraded":"2024-01-11T17:56:25.336092+0000","last_fullsized":"2024-01-11T17:56:25.336092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.397159+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:56:33.397159+0000","last_peered":"2024-01-11T17:56:33.397159+0000","last_clean":"2024-01-11T17:56:33.397159+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:56:33.397159+0000","last_undegraded":"2024-01-11T17:56:33.397159+0000","last_fullsized":"2024-01-11T17:56:33.397159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331833+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:56:25.331833+0000","last_peered":"2024-01-11T17:56:25.331833+0000","last_clean":"2024-01-11T17:56:25.331833+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:56:25.331833+0000","last_undegraded":"2024-01-11T17:56:25.331833+0000","last_fullsized":"2024-01-11T17:56:25.331833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337590+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:56:25.337590+0000","last_peered":"2024-01-11T17:56:25.337590+0000","last_clean":"2024-01-11T17:56:25.337590+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:56:25.337590+0000","last_undegraded":"2024-01-11T17:56:25.337590+0000","last_fullsized":"2024-01-11T17:56:25.337590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.391050+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:56:32.391050+0000","last_peered":"2024-01-11T17:56:32.391050+0000","last_clean":"2024-01-11T17:56:32.391050+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:56:32.391050+0000","last_undegraded":"2024-01-11T17:56:32.391050+0000","last_fullsized":"2024-01-11T17:56:32.391050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341538+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:56:25.341538+0000","last_peered":"2024-01-11T17:56:25.341538+0000","last_clean":"2024-01-11T17:56:25.341538+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:56:25.341538+0000","last_undegraded":"2024-01-11T17:56:25.341538+0000","last_fullsized":"2024-01-11T17:56:25.341538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.343158+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:56:26.343158+0000","last_peered":"2024-01-11T17:56:26.343158+0000","last_clean":"2024-01-11T17:56:26.343158+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:56:26.343158+0000","last_undegraded":"2024-01-11T17:56:26.343158+0000","last_fullsized":"2024-01-11T17:56:26.343158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:14.388403+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:57:14.388403+0000","last_peered":"2024-01-11T17:57:14.388403+0000","last_clean":"2024-01-11T17:57:14.388403+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:57:14.388403+0000","last_undegraded":"2024-01-11T17:57:14.388403+0000","last_fullsized":"2024-01-11T17:57:14.388403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341549+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:56:25.341549+0000","last_peered":"2024-01-11T17:56:25.341549+0000","last_clean":"2024-01-11T17:56:25.341549+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:56:25.341549+0000","last_undegraded":"2024-01-11T17:56:25.341549+0000","last_fullsized":"2024-01-11T17:56:25.341549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331924+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:56:25.331924+0000","last_peered":"2024-01-11T17:56:25.331924+0000","last_clean":"2024-01-11T17:56:25.331924+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:56:25.331924+0000","last_undegraded":"2024-01-11T17:56:25.331924+0000","last_fullsized":"2024-01-11T17:56:25.331924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.398418+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:56:33.398418+0000","last_peered":"2024-01-11T17:56:33.398418+0000","last_clean":"2024-01-11T17:56:33.398418+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:56:33.398418+0000","last_undegraded":"2024-01-11T17:56:33.398418+0000","last_fullsized":"2024-01-11T17:56:33.398418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336728+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:56:25.336728+0000","last_peered":"2024-01-11T17:56:25.336728+0000","last_clean":"2024-01-11T17:56:25.336728+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:56:25.336728+0000","last_undegraded":"2024-01-11T17:56:25.336728+0000","last_fullsized":"2024-01-11T17:56:25.336728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336598+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:56:25.336598+0000","last_peered":"2024-01-11T17:56:25.336598+0000","last_clean":"2024-01-11T17:56:25.336598+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:56:25.336598+0000","last_undegraded":"2024-01-11T17:56:25.336598+0000","last_fullsized":"2024-01-11T17:56:25.336598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324929+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:56:34.324929+0000","last_peered":"2024-01-11T17:56:34.324929+0000","last_clean":"2024-01-11T17:56:34.324929+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:56:34.324929+0000","last_undegraded":"2024-01-11T17:56:34.324929+0000","last_fullsized":"2024-01-11T17:56:34.324929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335418+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:56:25.335418+0000","last_peered":"2024-01-11T17:56:25.335418+0000","last_clean":"2024-01-11T17:56:25.335418+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:56:25.335418+0000","last_undegraded":"2024-01-11T17:56:25.335418+0000","last_fullsized":"2024-01-11T17:56:25.335418+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335675+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:56:25.335675+0000","last_peered":"2024-01-11T17:56:25.335675+0000","last_clean":"2024-01-11T17:56:25.335675+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:56:25.335675+0000","last_undegraded":"2024-01-11T17:56:25.335675+0000","last_fullsized":"2024-01-11T17:56:25.335675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337326+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:56:25.337326+0000","last_peered":"2024-01-11T17:56:25.337326+0000","last_clean":"2024-01-11T17:56:25.337326+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:56:25.337326+0000","last_undegraded":"2024-01-11T17:56:25.337326+0000","last_fullsized":"2024-01-11T17:56:25.337326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335306+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:56:25.335306+0000","last_peered":"2024-01-11T17:56:25.335306+0000","last_clean":"2024-01-11T17:56:25.335306+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:56:25.335306+0000","last_undegraded":"2024-01-11T17:56:25.335306+0000","last_fullsized":"2024-01-11T17:56:25.335306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331851+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:56:25.331851+0000","last_peered":"2024-01-11T17:56:25.331851+0000","last_clean":"2024-01-11T17:56:25.331851+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:56:25.331851+0000","last_undegraded":"2024-01-11T17:56:25.331851+0000","last_fullsized":"2024-01-11T17:56:25.331851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338402+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:56:26.338402+0000","last_peered":"2024-01-11T17:56:26.338402+0000","last_clean":"2024-01-11T17:56:26.338402+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:56:26.338402+0000","last_undegraded":"2024-01-11T17:56:26.338402+0000","last_fullsized":"2024-01-11T17:56:26.338402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335017+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:56:25.335017+0000","last_peered":"2024-01-11T17:56:25.335017+0000","last_clean":"2024-01-11T17:56:25.335017+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:56:25.335017+0000","last_undegraded":"2024-01-11T17:56:25.335017+0000","last_fullsized":"2024-01-11T17:56:25.335017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331773+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:56:25.331773+0000","last_peered":"2024-01-11T17:56:25.331773+0000","last_clean":"2024-01-11T17:56:25.331773+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:56:25.331773+0000","last_undegraded":"2024-01-11T17:56:25.331773+0000","last_fullsized":"2024-01-11T17:56:25.331773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338194+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:56:25.338194+0000","last_peered":"2024-01-11T17:56:25.338194+0000","last_clean":"2024-01-11T17:56:25.338194+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:56:25.338194+0000","last_undegraded":"2024-01-11T17:56:25.338194+0000","last_fullsized":"2024-01-11T17:56:25.338194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":65,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:57:14.387954+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:57:14.387954+0000","last_peered":"2024-01-11T17:57:14.387954+0000","last_clean":"2024-01-11T17:57:14.387954+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:57:14.387954+0000","last_undegraded":"2024-01-11T17:57:14.387954+0000","last_fullsized":"2024-01-11T17:57:14.387954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.338722+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:56:26.338722+0000","last_peered":"2024-01-11T17:56:26.338722+0000","last_clean":"2024-01-11T17:56:26.338722+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:56:26.338722+0000","last_undegraded":"2024-01-11T17:56:26.338722+0000","last_fullsized":"2024-01-11T17:56:26.338722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338909+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:56:25.338909+0000","last_peered":"2024-01-11T17:56:25.338909+0000","last_clean":"2024-01-11T17:56:25.338909+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:56:25.338909+0000","last_undegraded":"2024-01-11T17:56:25.338909+0000","last_fullsized":"2024-01-11T17:56:25.338909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:34.324345+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:56:34.324345+0000","last_peered":"2024-01-11T17:56:34.324345+0000","last_clean":"2024-01-11T17:56:34.324345+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:56:34.324345+0000","last_undegraded":"2024-01-11T17:56:34.324345+0000","last_fullsized":"2024-01-11T17:56:34.324345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":311,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341318+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:56:25.341318+0000","last_peered":"2024-01-11T17:56:25.341318+0000","last_clean":"2024-01-11T17:56:25.341318+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:56:25.341318+0000","last_undegraded":"2024-01-11T17:56:25.341318+0000","last_fullsized":"2024-01-11T17:56:25.341318+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335975+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:56:25.335975+0000","last_peered":"2024-01-11T17:56:25.335975+0000","last_clean":"2024-01-11T17:56:25.335975+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:56:25.335975+0000","last_undegraded":"2024-01-11T17:56:25.335975+0000","last_fullsized":"2024-01-11T17:56:25.335975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331435+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:56:25.331435+0000","last_peered":"2024-01-11T17:56:25.331435+0000","last_clean":"2024-01-11T17:56:25.331435+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:56:25.331435+0000","last_undegraded":"2024-01-11T17:56:25.331435+0000","last_fullsized":"2024-01-11T17:56:25.331435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.394917+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:56:33.394917+0000","last_peered":"2024-01-11T17:56:33.394917+0000","last_clean":"2024-01-11T17:56:33.394917+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:56:33.394917+0000","last_undegraded":"2024-01-11T17:56:33.394917+0000","last_fullsized":"2024-01-11T17:56:33.394917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335272+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:56:25.335272+0000","last_peered":"2024-01-11T17:56:25.335272+0000","last_clean":"2024-01-11T17:56:25.335272+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:56:25.335272+0000","last_undegraded":"2024-01-11T17:56:25.335272+0000","last_fullsized":"2024-01-11T17:56:25.335272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388565+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:56:32.388565+0000","last_peered":"2024-01-11T17:56:32.388565+0000","last_clean":"2024-01-11T17:56:32.388565+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:56:32.388565+0000","last_undegraded":"2024-01-11T17:56:32.388565+0000","last_fullsized":"2024-01-11T17:56:32.388565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332040+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:56:25.332040+0000","last_peered":"2024-01-11T17:56:25.332040+0000","last_clean":"2024-01-11T17:56:25.332040+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:56:25.332040+0000","last_undegraded":"2024-01-11T17:56:25.332040+0000","last_fullsized":"2024-01-11T17:56:25.332040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340003+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:56:25.340003+0000","last_peered":"2024-01-11T17:56:25.340003+0000","last_clean":"2024-01-11T17:56:25.340003+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:56:25.340003+0000","last_undegraded":"2024-01-11T17:56:25.340003+0000","last_fullsized":"2024-01-11T17:56:25.340003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:38.268914+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:56:38.268914+0000","last_peered":"2024-01-11T17:56:38.268914+0000","last_clean":"2024-01-11T17:56:38.268914+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:56:38.268914+0000","last_undegraded":"2024-01-11T17:56:38.268914+0000","last_fullsized":"2024-01-11T17:56:38.268914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339772+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:56:25.339772+0000","last_peered":"2024-01-11T17:56:25.339772+0000","last_clean":"2024-01-11T17:56:25.339772+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:56:25.339772+0000","last_undegraded":"2024-01-11T17:56:25.339772+0000","last_fullsized":"2024-01-11T17:56:25.339772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.340165+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:56:25.340165+0000","last_peered":"2024-01-11T17:56:25.340165+0000","last_clean":"2024-01-11T17:56:25.340165+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:56:25.340165+0000","last_undegraded":"2024-01-11T17:56:25.340165+0000","last_fullsized":"2024-01-11T17:56:25.340165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.386925+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:56:32.386925+0000","last_peered":"2024-01-11T17:56:32.386925+0000","last_clean":"2024-01-11T17:56:32.386925+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:56:32.386925+0000","last_undegraded":"2024-01-11T17:56:32.386925+0000","last_fullsized":"2024-01-11T17:56:32.386925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336540+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:56:25.336540+0000","last_peered":"2024-01-11T17:56:25.336540+0000","last_clean":"2024-01-11T17:56:25.336540+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:56:25.336540+0000","last_undegraded":"2024-01-11T17:56:25.336540+0000","last_fullsized":"2024-01-11T17:56:25.336540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.346059+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:56:26.346059+0000","last_peered":"2024-01-11T17:56:26.346059+0000","last_clean":"2024-01-11T17:56:26.346059+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:56:26.346059+0000","last_undegraded":"2024-01-11T17:56:26.346059+0000","last_fullsized":"2024-01-11T17:56:26.346059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.388647+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:56:32.388647+0000","last_peered":"2024-01-11T17:56:32.388647+0000","last_clean":"2024-01-11T17:56:32.388647+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:56:32.388647+0000","last_undegraded":"2024-01-11T17:56:32.388647+0000","last_fullsized":"2024-01-11T17:56:32.388647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334285+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:56:25.334285+0000","last_peered":"2024-01-11T17:56:25.334285+0000","last_clean":"2024-01-11T17:56:25.334285+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:56:25.334285+0000","last_undegraded":"2024-01-11T17:56:25.334285+0000","last_fullsized":"2024-01-11T17:56:25.334285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338275+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:56:25.338275+0000","last_peered":"2024-01-11T17:56:25.338275+0000","last_clean":"2024-01-11T17:56:25.338275+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:56:25.338275+0000","last_undegraded":"2024-01-11T17:56:25.338275+0000","last_fullsized":"2024-01-11T17:56:25.338275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389005+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:56:32.389005+0000","last_peered":"2024-01-11T17:56:32.389005+0000","last_clean":"2024-01-11T17:56:32.389005+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:56:32.389005+0000","last_undegraded":"2024-01-11T17:56:32.389005+0000","last_fullsized":"2024-01-11T17:56:32.389005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.334931+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:56:25.334931+0000","last_peered":"2024-01-11T17:56:25.334931+0000","last_clean":"2024-01-11T17:56:25.334931+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:56:25.334931+0000","last_undegraded":"2024-01-11T17:56:25.334931+0000","last_fullsized":"2024-01-11T17:56:25.334931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.333647+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:56:26.333647+0000","last_peered":"2024-01-11T17:56:26.333647+0000","last_clean":"2024-01-11T17:56:26.333647+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:56:26.333647+0000","last_undegraded":"2024-01-11T17:56:26.333647+0000","last_fullsized":"2024-01-11T17:56:26.333647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:29.433711+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:56:29.433711+0000","last_peered":"2024-01-11T17:56:29.433711+0000","last_clean":"2024-01-11T17:56:29.433711+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:56:29.433711+0000","last_undegraded":"2024-01-11T17:56:29.433711+0000","last_fullsized":"2024-01-11T17:56:29.433711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339412+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:56:25.339412+0000","last_peered":"2024-01-11T17:56:25.339412+0000","last_clean":"2024-01-11T17:56:25.339412+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:56:25.339412+0000","last_undegraded":"2024-01-11T17:56:25.339412+0000","last_fullsized":"2024-01-11T17:56:25.339412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.331945+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:56:25.331945+0000","last_peered":"2024-01-11T17:56:25.331945+0000","last_clean":"2024-01-11T17:56:25.331945+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:56:25.331945+0000","last_undegraded":"2024-01-11T17:56:25.331945+0000","last_fullsized":"2024-01-11T17:56:25.331945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:32.389594+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:56:32.389594+0000","last_peered":"2024-01-11T17:56:32.389594+0000","last_clean":"2024-01-11T17:56:32.389594+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:56:32.389594+0000","last_undegraded":"2024-01-11T17:56:32.389594+0000","last_fullsized":"2024-01-11T17:56:32.389594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341965+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:56:25.341965+0000","last_peered":"2024-01-11T17:56:25.341965+0000","last_clean":"2024-01-11T17:56:25.341965+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:56:25.341965+0000","last_undegraded":"2024-01-11T17:56:25.341965+0000","last_fullsized":"2024-01-11T17:56:25.341965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337490+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:56:25.337490+0000","last_peered":"2024-01-11T17:56:25.337490+0000","last_clean":"2024-01-11T17:56:25.337490+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:56:25.337490+0000","last_undegraded":"2024-01-11T17:56:25.337490+0000","last_fullsized":"2024-01-11T17:56:25.337490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:33.396839+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:56:33.396839+0000","last_peered":"2024-01-11T17:56:33.396839+0000","last_clean":"2024-01-11T17:56:33.396839+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:56:33.396839+0000","last_undegraded":"2024-01-11T17:56:33.396839+0000","last_fullsized":"2024-01-11T17:56:33.396839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341851+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:56:25.341851+0000","last_peered":"2024-01-11T17:56:25.341851+0000","last_clean":"2024-01-11T17:56:25.341851+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:56:25.341851+0000","last_undegraded":"2024-01-11T17:56:25.341851+0000","last_fullsized":"2024-01-11T17:56:25.341851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345480+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:56:26.345480+0000","last_peered":"2024-01-11T17:56:26.345480+0000","last_clean":"2024-01-11T17:56:26.345480+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:56:26.345480+0000","last_undegraded":"2024-01-11T17:56:26.345480+0000","last_fullsized":"2024-01-11T17:56:26.345480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.345371+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:56:26.345371+0000","last_peered":"2024-01-11T17:56:26.345371+0000","last_clean":"2024-01-11T17:56:26.345371+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:56:26.345371+0000","last_undegraded":"2024-01-11T17:56:26.345371+0000","last_fullsized":"2024-01-11T17:56:26.345371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341451+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:56:25.341451+0000","last_peered":"2024-01-11T17:56:25.341451+0000","last_clean":"2024-01-11T17:56:25.341451+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:56:25.341451+0000","last_undegraded":"2024-01-11T17:56:25.341451+0000","last_fullsized":"2024-01-11T17:56:25.341451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337848+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:56:25.337848+0000","last_peered":"2024-01-11T17:56:25.337848+0000","last_clean":"2024-01-11T17:56:25.337848+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:56:25.337848+0000","last_undegraded":"2024-01-11T17:56:25.337848+0000","last_fullsized":"2024-01-11T17:56:25.337848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332029+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:56:25.332029+0000","last_peered":"2024-01-11T17:56:25.332029+0000","last_clean":"2024-01-11T17:56:25.332029+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:56:25.332029+0000","last_undegraded":"2024-01-11T17:56:25.332029+0000","last_fullsized":"2024-01-11T17:56:25.332029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341977+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:56:25.341977+0000","last_peered":"2024-01-11T17:56:25.341977+0000","last_clean":"2024-01-11T17:56:25.341977+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:56:25.341977+0000","last_undegraded":"2024-01-11T17:56:25.341977+0000","last_fullsized":"2024-01-11T17:56:25.341977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.335030+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:56:25.335030+0000","last_peered":"2024-01-11T17:56:25.335030+0000","last_clean":"2024-01-11T17:56:25.335030+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:56:25.335030+0000","last_undegraded":"2024-01-11T17:56:25.335030+0000","last_fullsized":"2024-01-11T17:56:25.335030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355996+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:56:26.355996+0000","last_peered":"2024-01-11T17:56:26.355996+0000","last_clean":"2024-01-11T17:56:26.355996+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:56:26.355996+0000","last_undegraded":"2024-01-11T17:56:26.355996+0000","last_fullsized":"2024-01-11T17:56:26.355996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336409+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:56:25.336409+0000","last_peered":"2024-01-11T17:56:25.336409+0000","last_clean":"2024-01-11T17:56:25.336409+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:56:25.336409+0000","last_undegraded":"2024-01-11T17:56:25.336409+0000","last_fullsized":"2024-01-11T17:56:25.336409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336125+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:56:25.336125+0000","last_peered":"2024-01-11T17:56:25.336125+0000","last_clean":"2024-01-11T17:56:25.336125+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:56:25.336125+0000","last_undegraded":"2024-01-11T17:56:25.336125+0000","last_fullsized":"2024-01-11T17:56:25.336125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332015+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:56:25.332015+0000","last_peered":"2024-01-11T17:56:25.332015+0000","last_clean":"2024-01-11T17:56:25.332015+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:56:25.332015+0000","last_undegraded":"2024-01-11T17:56:25.332015+0000","last_fullsized":"2024-01-11T17:56:25.332015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.336086+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:56:25.336086+0000","last_peered":"2024-01-11T17:56:25.336086+0000","last_clean":"2024-01-11T17:56:25.336086+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:56:25.336086+0000","last_undegraded":"2024-01-11T17:56:25.336086+0000","last_fullsized":"2024-01-11T17:56:25.336086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338479+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:56:25.338479+0000","last_peered":"2024-01-11T17:56:25.338479+0000","last_clean":"2024-01-11T17:56:25.338479+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:56:25.338479+0000","last_undegraded":"2024-01-11T17:56:25.338479+0000","last_fullsized":"2024-01-11T17:56:25.338479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.341422+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:56:25.341422+0000","last_peered":"2024-01-11T17:56:25.341422+0000","last_clean":"2024-01-11T17:56:25.341422+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:56:25.341422+0000","last_undegraded":"2024-01-11T17:56:25.341422+0000","last_fullsized":"2024-01-11T17:56:25.341422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.338107+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:56:25.338107+0000","last_peered":"2024-01-11T17:56:25.338107+0000","last_clean":"2024-01-11T17:56:25.338107+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:56:25.338107+0000","last_undegraded":"2024-01-11T17:56:25.338107+0000","last_fullsized":"2024-01-11T17:56:25.338107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.339143+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:56:25.339143+0000","last_peered":"2024-01-11T17:56:25.339143+0000","last_clean":"2024-01-11T17:56:25.339143+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:56:25.339143+0000","last_undegraded":"2024-01-11T17:56:25.339143+0000","last_fullsized":"2024-01-11T17:56:25.339143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.337858+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:56:25.337858+0000","last_peered":"2024-01-11T17:56:25.337858+0000","last_clean":"2024-01-11T17:56:25.337858+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:56:25.337858+0000","last_undegraded":"2024-01-11T17:56:25.337858+0000","last_fullsized":"2024-01-11T17:56:25.337858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-11T17:56:25.332218+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:56:25.332218+0000","last_peered":"2024-01-11T17:56:25.332218+0000","last_clean":"2024-01-11T17:56:25.332218+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:56:25.332218+0000","last_undegraded":"2024-01-11T17:56:25.332218+0000","last_fullsized":"2024-01-11T17:56:25.332218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-11T17:56:26.355928+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:56:26.355928+0000","last_peered":"2024-01-11T17:56:26.355928+0000","last_clean":"2024-01-11T17:56:26.355928+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:56:26.355928+0000","last_undegraded":"2024-01-11T17:56:26.355928+0000","last_fullsized":"2024-01-11T17:56:26.355928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67744,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":277,"num_read_kb":211,"num_write":155,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203232,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":473098,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.879}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.86499999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.98899999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.89100000000000001}]}]},{"osd":10,"up_from":68,"seq":292057776156,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":1124110,"data_compressed":55398,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.433}]},{"osd":5,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.75},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.94999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.254,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.097,"15min":1.097},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.117,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.099,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0529999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66600000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.79000000000000004}]}]},{"osd":9,"up_from":63,"seq":270582939679,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":477430,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82999999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.42099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.433},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.45800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90700000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.749}]},{"osd":3,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84799999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82899999999999996,"15min":0.83199999999999996},"min":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.83099999999999996,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.04,"5min":1.198,"15min":1.198},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0269999999999999,"5min":1.224,"15min":1.224},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.92300000000000004,"15min":0.94199999999999995},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.079,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.139},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":0.98299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.52800000000000002}]},{"osd":8,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98999999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.96299999999999997,"15min":0.97899999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.022}]},{"osd":10,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.94799999999999995},"min":{"1min":0.622,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96099999999999997,"5min":1.284,"15min":1.284},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0309999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.089}]},{"osd":11,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77000000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168612,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":473254,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747}]},{"osd":1,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.31}]},{"osd":3,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.502}]},{"osd":4,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94199999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.872,"5min":1.097,"15min":1.097},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78300000000000003,"15min":0.78700000000000003},"min":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.94799999999999995,"5min":1,"15min":1},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93300000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.70299999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.89500000000000002,"15min":0.89700000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.871},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.83099999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85599999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.624},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59799999999999998}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.63800000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1069552,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.62},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.626,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90300000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81499999999999995,"5min":1.137,"15min":1.137},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.56000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.64500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98699999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":1.04,"15min":1.04},"last":0.746}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0960000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.145,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83999999999999997,"15min":0.83199999999999996},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.036},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.88200000000000001}]},{"osd":10,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80500000000000005,"15min":0.79600000000000004},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.82999999999999996},"min":{"1min":0.79500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0089999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0600000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561068,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":468510,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.504,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":1.149},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":2.149}]},{"osd":1,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72899999999999998,"15min":0.70699999999999996},"min":{"1min":0.59099999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.008,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.024},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":2.085}]},{"osd":2,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.64000000000000001,"15min":0.60799999999999998},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":1.907},{"interface":"front","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.04,"5min":1.161,"15min":1.161},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.645,"5min":2.645,"15min":2.645},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.8029999999999999}]},{"osd":4,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.52100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":2.1869999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.59499999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.048,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.93200000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80900000000000005,"15min":0.80100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.9770000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.9990000000000001}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.91600000000000004,"15min":0.90100000000000002},"min":{"1min":0.68799999999999994,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.79700000000000004,"15min":0.75600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":2.1099999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0980000000000001,"5min":1.155,"15min":1.155},"last":2.04}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.024,"15min":1.0309999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92100000000000004,"15min":0.91700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":2.2080000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.92700000000000005,"15min":0.89500000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":2.3069999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.093,"15min":1.095},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.2509999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.0880000000000001}]},{"osd":11,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":2.2490000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":531774,"data_compressed":19305,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.03,"5min":1.077,"15min":1.077},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89400000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.442,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.58199999999999996,"15min":0.54400000000000004},"min":{"1min":0.41799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.54200000000000004},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.97299999999999998,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 11 17:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.44400000000000001,"15min":0.40000000000000002},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.48199999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0049999999999999,"5min":1.012,"15min":1.012},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.54200000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.04,"5min":1.048,"15min":1.048},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92700000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.52600000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79800000000000004,"5min":1.048,"15min":1.048},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.64000000000000001},"min":{"1min":0.59699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.048,"15min":1.048},"last":0.59699999999999998}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.61499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.72499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.056,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80500000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:56:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.006,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.877},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84399999999999997},"min":{"1min":0.66000000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.997,"5min":1.127,"15min":1.127},"last":0.84199999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86199999999999999,"15min":0.86399999999999999},"min":{"1min":0.58299999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0880000000000001,"5min":1.21,"15min":1.21},"last":0.78000000000000003}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.016}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":469898,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.66600000000000004,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.911,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.502,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.85999999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.67300000000000004,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.65800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.508,"5min":3.508,"15min":3.508},"last":1.087}]},{"osd":3,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.70099999999999996,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.129}]},{"osd":5,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.71199999999999997,"15min":0.68000000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.024},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.68200000000000005,"15min":0.63300000000000001},"min":{"1min":0.50700000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.997}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.67300000000000004},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":3.1019999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":1.274}]},{"osd":8,"last update":"Thu Jan 11 17:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.875,"15min":0.878},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":1.1439999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95999999999999996,"15min":0.95599999999999996},"min":{"1min":0.46200000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.423},{"interface":"front","average":{"1min":1.042,"5min":0.96999999999999997,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.2599999999999998,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":1.2390000000000001}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.75800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0630000000000002,"5min":3.0630000000000002,"15min":3.0630000000000002},"last":1.319},{"interface":"front","average":{"1min":1.075,"5min":1.135,"15min":1.145},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":3.4199999999999999,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.163}]},{"osd":11,"last update":"Thu Jan 11 17:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.4849999999999999,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":1.339}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":533514,"data_compressed":19305,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.753},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.60299999999999998,"15min":0.54700000000000004},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.075},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.92600000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.51900000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81499999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.022,"15min":1.022},"last":1.159},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.198}]},{"osd":8,"last update":"Thu Jan 11 17:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.25},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.97399999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.78000000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98599999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.111}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.129},"last":0.998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.73499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0469999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.0469999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.095}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":468704,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.40100000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.249},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.46700000000000003,"15min":0.42599999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.623,"15min":0.53700000000000003},"min":{"1min":0.56299999999999994,"5min":0.308,"15min":0.308},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.64000000000000001,"15min":0.55400000000000005},"min":{"1min":0.59099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65500000000000003,"15min":0.60499999999999998},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.56699999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.65300000000000002,"15min":0.59399999999999997},"min":{"1min":0.50700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.68700000000000006,"15min":0.628},"min":{"1min":0.57399999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.71399999999999997,"15min":0.67800000000000005},"min":{"1min":0.60199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.72099999999999997,"15min":0.68600000000000005},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.65100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0580000000000001,"5min":1.379,"15min":1.379},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.499,"15min":0.499},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.91800000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":0.93600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4259999999999999,"5min":1.52,"15min":1.52},"last":0.47599999999999998}]},{"osd":10,"last update":"Thu Jan 11 17:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.879}]},{"osd":11,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.82999999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":473550,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.41199999999999998,"15min":0.374},"min":{"1min":0.36099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.33600000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.91700000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Jan 11 17:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.60099999999999998,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1,"5min":1.034,"15min":1.034},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.61899999999999999,"15min":0.59099999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.57699999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89800000000000002,"5min":1.129,"15min":1.129},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68400000000000005,"15min":0.65400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96999999999999997,"5min":1.163,"15min":1.163},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.746}]},{"osd":8,"last update":"Thu Jan 11 17:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93200000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.623},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.024,"5min":1.238,"15min":1.238},"last":0.79100000000000004}]},{"osd":9,"last update":"Thu Jan 11 17:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1359999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.90300000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.629,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.95299999999999996,"5min":1.248,"15min":1.248},"last":0.87},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.84899999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.92500000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.88600000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":1064750,"data_compressed":53775,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88500000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.76100000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.374},{"interface":"front","average":{"1min":0.5,"5min":0.47599999999999998,"15min":0.45900000000000002},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.439}]},{"osd":3,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.42399999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.73299999999999998,"5min":0.999,"15min":0.999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76000000000000001,"5min":1.548,"15min":1.548},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.22,"15min":1.22},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.88300000000000001}]},{"osd":8,"last update":"Thu Jan 11 17:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.56499999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93999999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53300000000000003}]},{"osd":9,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":1.097,"15min":1.097},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79700000000000004,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0189999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":63354,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65094,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65322,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:57:18.928 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-01-11T17:57:18.928 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-11T17:57:18.928 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-01-11T17:57:18.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph health --format=json 2024-01-11T17:57:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:19 smithi007 ceph-mon[33361]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:57:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:19 smithi007 ceph-mon[33361]: pgmap v283: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:19 smithi016 ceph-mon[43195]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:57:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:19 smithi016 ceph-mon[43195]: pgmap v283: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:20.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:19 smithi038 ceph-mon[43582]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:57:20.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:19 smithi038 ceph-mon[43582]: pgmap v283: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 -- 172.21.15.7:0/1387099254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e4107640 msgr2=0x7fd6e4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 --2- 172.21.15.7:0/1387099254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e4107640 0x7fd6e4107a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00098f0 tx=0x7fd6e0009c00 comp rx=0 tx=0).stop 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 -- 172.21.15.7:0/1387099254 shutdown_connections 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 --2- 172.21.15.7:0/1387099254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e410e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 --2- 172.21.15.7:0/1387099254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e4107640 0x7fd6e4107a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 --2- 172.21.15.7:0/1387099254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e40692c0 0x7fd6e4069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 -- 172.21.15.7:0/1387099254 >> 172.21.15.7:0/1387099254 conn(0x7fd6e4077440 msgr2=0x7fd6e4077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 -- 172.21.15.7:0/1387099254 shutdown_connections 2024-01-11T17:57:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.542+0000 7fd6eb809700 1 -- 172.21.15.7:0/1387099254 wait complete. 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.543+0000 7fd6eb809700 1 Processor -- start 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 -- start start 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e40692c0 0x7fd6e419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e4107640 0x7fd6e4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4199ff0 con 0x7fd6e4069c80 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e419a130 con 0x7fd6e40692c0 2024-01-11T17:57:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6eb809700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e419a2f0 con 0x7fd6e4107640 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6e8da4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6e8da4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51968/0 (socket says 172.21.15.7:51968) 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6e95a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e40692c0 0x7fd6e419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6e8da4700 1 -- 172.21.15.7:0/3008797259 learned_addr learned my addr 172.21.15.7:0/3008797259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.544+0000 7fd6e9da6700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e4107640 0x7fd6e4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6e8da4700 1 -- 172.21.15.7:0/3008797259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e4107640 msgr2=0x7fd6e4199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6e8da4700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e4107640 0x7fd6e4199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6e8da4700 1 -- 172.21.15.7:0/3008797259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e40692c0 msgr2=0x7fd6e419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6e8da4700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e40692c0 0x7fd6e419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6e8da4700 1 -- 172.21.15.7:0/3008797259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc009d00 con 0x7fd6e4069c80 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6e8da4700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e419fd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e9a0 tx=0x7fd6d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4009790 con 0x7fd6e4069c80 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6d4010430 con 0x7fd6e4069c80 2024-01-11T17:57:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e0009520 con 0x7fd6e4069c80 2024-01-11T17:57:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4018680 con 0x7fd6e4069c80 2024-01-11T17:57:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.545+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e41a66a0 con 0x7fd6e4069c80 2024-01-11T17:57:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.547+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e40682d0 con 0x7fd6e4069c80 2024-01-11T17:57:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.548+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fd6d40105a0 con 0x7fd6e4069c80 2024-01-11T17:57:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.550+0000 7fd6da7fc700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd6d00a1ff0 0x7fd6d00a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.550+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd6d4014070 con 0x7fd6e4069c80 2024-01-11T17:57:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.550+0000 7fd6e95a5700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd6d00a1ff0 0x7fd6d00a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.551+0000 7fd6e95a5700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd6d00a1ff0 0x7fd6d00a44b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009810 tx=0x7fd6e0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:20.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.554+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd6d4096bd0 con 0x7fd6e4069c80 2024-01-11T17:57:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.818+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd6e4111490 con 0x7fd6e4069c80 2024-01-11T17:57:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.819+0000 7fd6da7fc700 1 -- 172.21.15.7:0/3008797259 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd6d409a1f0 con 0x7fd6e4069c80 2024-01-11T17:57:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:20.821 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-11T17:57:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd6d00a1ff0 msgr2=0x7fd6d00a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd6d00a1ff0 0x7fd6d00a44b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009810 tx=0x7fd6e0019040 comp rx=0 tx=0).stop 2024-01-11T17:57:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 msgr2=0x7fd6e419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e419fd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e9a0 tx=0x7fd6d400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 shutdown_connections 2024-01-11T17:57:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd6d00a1ff0 0x7fd6d00a44b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6e4107640 0x7fd6e4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e40692c0 0x7fd6e419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 --2- 172.21.15.7:0/3008797259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4069c80 0x7fd6e419fd10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:20.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.821+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 >> 172.21.15.7:0/3008797259 conn(0x7fd6e4077440 msgr2=0x7fd6e410c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:20.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.822+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 shutdown_connections 2024-01-11T17:57:20.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:20.822+0000 7fd6eb809700 1 -- 172.21.15.7:0/3008797259 wait complete. 2024-01-11T17:57:21.201 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-01-11T17:57:21.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:21.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.408+0000 7fd76359e700 1 -- 172.21.15.7:0/1322026165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 msgr2=0x7fd764102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.408+0000 7fd76359e700 1 --2- 172.21.15.7:0/1322026165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764102a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd7580099c0 tx=0x7fd758009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 -- 172.21.15.7:0/1322026165 shutdown_connections 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 --2- 172.21.15.7:0/1322026165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7641013c0 0x7fd7641017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 --2- 172.21.15.7:0/1322026165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd764102f50 0x7fd764109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 --2- 172.21.15.7:0/1322026165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764102a10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 -- 172.21.15.7:0/1322026165 >> 172.21.15.7:0/1322026165 conn(0x7fd7640fc930 msgr2=0x7fd7640fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 -- 172.21.15.7:0/1322026165 shutdown_connections 2024-01-11T17:57:21.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.409+0000 7fd76359e700 1 -- 172.21.15.7:0/1322026165 wait complete. 2024-01-11T17:57:21.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 Processor -- start 2024-01-11T17:57:21.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 -- start start 2024-01-11T17:57:21.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7641013c0 0x7fd7640753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd764102f50 0x7fd764073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640746e0 con 0x7fd7641025b0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd764074850 con 0x7fd7641013c0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.410+0000 7fd76359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641a12c0 con 0x7fd764102f50 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51982/0 (socket says 172.21.15.7:51982) 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 -- 172.21.15.7:0/4120483097 learned_addr learned my addr 172.21.15.7:0/4120483097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd76259c700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7641013c0 0x7fd7640753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd762d9d700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd764102f50 0x7fd764073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 -- 172.21.15.7:0/4120483097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd764102f50 msgr2=0x7fd764073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd764102f50 0x7fd764073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 -- 172.21.15.7:0/4120483097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7641013c0 msgr2=0x7fd7640753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7641013c0 0x7fd7640753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 -- 172.21.15.7:0/4120483097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580095f0 con 0x7fd7641025b0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd761d9b700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764073a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd758006010 tx=0x7fd758004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75801d070 con 0x7fd7641025b0 2024-01-11T17:57:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd758004050 con 0x7fd7641025b0 2024-01-11T17:57:21.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.411+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758005290 con 0x7fd7641025b0 2024-01-11T17:57:21.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.412+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7641a1540 con 0x7fd7641025b0 2024-01-11T17:57:21.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.412+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7641a1910 con 0x7fd7641025b0 2024-01-11T17:57:21.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.413+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd764108770 con 0x7fd7641025b0 2024-01-11T17:57:21.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.414+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fd7580048e0 con 0x7fd7641025b0 2024-01-11T17:57:21.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.416+0000 7fd7537fe700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd74c0a1ff0 0x7fd74c0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.416+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7580cec80 con 0x7fd7641025b0 2024-01-11T17:57:21.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.416+0000 7fd76259c700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd74c0a1ff0 0x7fd74c0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.417+0000 7fd76259c700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd74c0a1ff0 0x7fd74c0a44b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd764102410 tx=0x7fd75400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:21.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.420+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fd7580978e0 con 0x7fd7641025b0 2024-01-11T17:57:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.632+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd76410cc30 con 0x7fd7641025b0 2024-01-11T17:57:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.634+0000 7fd7537fe700 1 -- 172.21.15.7:0/4120483097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7fd75809af00 con 0x7fd7641025b0 2024-01-11T17:57:21.635 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:21.636 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd74c0a1ff0 msgr2=0x7fd74c0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd74c0a1ff0 0x7fd74c0a44b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd764102410 tx=0x7fd75400b410 comp rx=0 tx=0).stop 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 msgr2=0x7fd764073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764073a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd758006010 tx=0x7fd758004730 comp rx=0 tx=0).stop 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 shutdown_connections 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd74c0a1ff0 0x7fd74c0a44b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd764102f50 0x7fd764073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7641013c0 0x7fd7640753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 --2- 172.21.15.7:0/4120483097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7641025b0 0x7fd764073a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 >> 172.21.15.7:0/4120483097 conn(0x7fd7640fc930 msgr2=0x7fd7641058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 shutdown_connections 2024-01-11T17:57:21.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.636+0000 7fd76359e700 1 -- 172.21.15.7:0/4120483097 wait complete. 2024-01-11T17:57:21.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:57:21.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:57:21.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 -- 172.21.15.7:0/2332711826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 msgr2=0x7fb054073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/2332711826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054073aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb050009990 tx=0x7fb050009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 -- 172.21.15.7:0/2332711826 shutdown_connections 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/2332711826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054109470 0x7fb05410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/2332711826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054073aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/2332711826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054074070 0x7fb0540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.865+0000 7fb05c5ff700 1 -- 172.21.15.7:0/2332711826 >> 172.21.15.7:0/2332711826 conn(0x7fb0540fcb00 msgr2=0x7fb0540fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.866+0000 7fb05c5ff700 1 -- 172.21.15.7:0/2332711826 shutdown_connections 2024-01-11T17:57:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.866+0000 7fb05c5ff700 1 -- 172.21.15.7:0/2332711826 wait complete. 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.866+0000 7fb05c5ff700 1 Processor -- start 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 -- start start 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054074070 0x7fb054199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054109470 0x7fb0541a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541a1200 con 0x7fb054075650 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541a1340 con 0x7fb054109470 2024-01-11T17:57:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05c5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541a14d0 con 0x7fb054074070 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb059b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05ab9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054109470 0x7fb0541a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05a39b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054074070 0x7fb054199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb059b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51996/0 (socket says 172.21.15.7:51996) 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb05ab9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054109470 0x7fb0541a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40396/0 (socket says 172.21.15.7:40396) 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.867+0000 7fb059b9a700 1 -- 172.21.15.7:0/1157095231 learned_addr learned my addr 172.21.15.7:0/1157095231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05a39b700 1 -- 172.21.15.7:0/1157095231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054109470 msgr2=0x7fb0541a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05a39b700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054109470 0x7fb0541a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05a39b700 1 -- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 msgr2=0x7fb054199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05a39b700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05a39b700 1 -- 172.21.15.7:0/1157095231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0500095f0 con 0x7fb054074070 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb059b9a700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05a39b700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054074070 0x7fb054199680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0500098e0 tx=0x7fb050009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05001d070 con 0x7fb054074070 2024-01-11T17:57:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0541a1750 con 0x7fb054074070 2024-01-11T17:57:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb050022470 con 0x7fb054074070 2024-01-11T17:57:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.868+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050017650 con 0x7fb054074070 2024-01-11T17:57:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.869+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0541a1bd0 con 0x7fb054074070 2024-01-11T17:57:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.869+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054108230 con 0x7fb054074070 2024-01-11T17:57:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.873+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb050022890 con 0x7fb054074070 2024-01-11T17:57:21.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.874+0000 7fb04b7fe700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb0400a1ff0 0x7fb0400a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:21.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.874+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb0500ceeb0 con 0x7fb054074070 2024-01-11T17:57:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.874+0000 7fb059b9a700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb0400a1ff0 0x7fb0400a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.874+0000 7fb059b9a700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb0400a1ff0 0x7fb0400a44b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb044005950 tx=0x7fb044009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:21.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:21.875+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb050097b10 con 0x7fb054074070 2024-01-11T17:57:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:21 smithi016 ceph-mon[43195]: pgmap v284: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:21 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3008797259' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-11T17:57:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:21 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4120483097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:22.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:21 smithi038 ceph-mon[43582]: pgmap v284: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:22.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:21 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3008797259' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-11T17:57:22.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:21 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4120483097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:21 smithi007 ceph-mon[33361]: pgmap v284: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:21 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3008797259' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-11T17:57:22.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:21 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4120483097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.092+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb05410f0a0 con 0x7fb054074070 2024-01-11T17:57:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.093+0000 7fb04b7fe700 1 -- 172.21.15.7:0/1157095231 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb050005c00 con 0x7fb054074070 2024-01-11T17:57:22.095 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb0400a1ff0 msgr2=0x7fb0400a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb0400a1ff0 0x7fb0400a44b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb044005950 tx=0x7fb044009260 comp rx=0 tx=0).stop 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054074070 msgr2=0x7fb054199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054074070 0x7fb054199680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb0500098e0 tx=0x7fb050009580 comp rx=0 tx=0).stop 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 shutdown_connections 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb0400a1ff0 0x7fb0400a44b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb054074070 0x7fb054199680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb054109470 0x7fb0541a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 --2- 172.21.15.7:0/1157095231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb054075650 0x7fb054199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 >> 172.21.15.7:0/1157095231 conn(0x7fb0540fcb00 msgr2=0x7fb054105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 shutdown_connections 2024-01-11T17:57:22.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.096+0000 7fb05c5ff700 1 -- 172.21.15.7:0/1157095231 wait complete. 2024-01-11T17:57:22.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:57:22.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 -- 172.21.15.7:0/3760449887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee8103140 msgr2=0x7f6ee81099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 --2- 172.21.15.7:0/3760449887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee8103140 0x7f6ee81099f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b0d0 tx=0x7f6ee000b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 -- 172.21.15.7:0/3760449887 shutdown_connections 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 --2- 172.21.15.7:0/3760449887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee8103140 0x7f6ee81099f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 --2- 172.21.15.7:0/3760449887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ee81015b0 0x7f6ee81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 --2- 172.21.15.7:0/3760449887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ee81027a0 0x7f6ee8102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 -- 172.21.15.7:0/3760449887 >> 172.21.15.7:0/3760449887 conn(0x7f6ee80fcb00 msgr2=0x7f6ee80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 -- 172.21.15.7:0/3760449887 shutdown_connections 2024-01-11T17:57:22.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.326+0000 7f6eefbff700 1 -- 172.21.15.7:0/3760449887 wait complete. 2024-01-11T17:57:22.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 Processor -- start 2024-01-11T17:57:22.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 -- start start 2024-01-11T17:57:22.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 0x7f6ee8197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ee81027a0 0x7f6ee8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ee8103140 0x7f6ee819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee819f190 con 0x7f6ee81015b0 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee819f2d0 con 0x7f6ee8103140 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eefbff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee819f460 con 0x7f6ee81027a0 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.327+0000 7f6eed99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 0x7f6ee8197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 0x7f6ee8197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46022/0 (socket says 172.21.15.7:46022) 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 -- 172.21.15.7:0/280958422 learned_addr learned my addr 172.21.15.7:0/280958422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed19a700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ee81027a0 0x7f6ee8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eee19c700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ee8103140 0x7f6ee819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 -- 172.21.15.7:0/280958422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ee81027a0 msgr2=0x7f6ee8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ee81027a0 0x7f6ee8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 -- 172.21.15.7:0/280958422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ee8103140 msgr2=0x7f6ee819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ee8103140 0x7f6ee819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 -- 172.21.15.7:0/280958422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4009520 con 0x7f6ee81015b0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6eed99b700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 0x7f6ee8197610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400ea50 tx=0x7f6ee400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.328+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400cbd0 con 0x7f6ee81015b0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.329+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee0009d00 con 0x7f6ee81015b0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.329+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee400cd30 con 0x7f6ee81015b0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.329+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4004a50 con 0x7f6ee81015b0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.329+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee819fac0 con 0x7f6ee81015b0 2024-01-11T17:57:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.330+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8108900 con 0x7f6ee81015b0 2024-01-11T17:57:22.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.332+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f6ee4004bb0 con 0x7f6ee81015b0 2024-01-11T17:57:22.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.334+0000 7f6edeffd700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ed40a1f70 0x7f6ed40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.334+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ee4014070 con 0x7f6ee81015b0 2024-01-11T17:57:22.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.334+0000 7f6eed19a700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ed40a1f70 0x7f6ed40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.334+0000 7f6eed19a700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ed40a1f70 0x7f6ed40a4430 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009c60 tx=0x7f6ed8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:22.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.337+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6ee4096fd0 con 0x7f6ee81015b0 2024-01-11T17:57:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.551+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ee810cec0 con 0x7f6ee81015b0 2024-01-11T17:57:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.551+0000 7f6edeffd700 1 -- 172.21.15.7:0/280958422 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ee4004e60 con 0x7f6ee81015b0 2024-01-11T17:57:22.553 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ed40a1f70 msgr2=0x7f6ed40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ed40a1f70 0x7f6ed40a4430 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009c60 tx=0x7f6ed8009260 comp rx=0 tx=0).stop 2024-01-11T17:57:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 msgr2=0x7f6ee8197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 0x7f6ee8197610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400ea50 tx=0x7f6ee400ed60 comp rx=0 tx=0).stop 2024-01-11T17:57:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 shutdown_connections 2024-01-11T17:57:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6ed40a1f70 0x7f6ed40a4430 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ee8103140 0x7f6ee819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ee81015b0 0x7f6ee8197610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 --2- 172.21.15.7:0/280958422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ee81027a0 0x7f6ee8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 >> 172.21.15.7:0/280958422 conn(0x7f6ee80fcb00 msgr2=0x7f6ee8105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 shutdown_connections 2024-01-11T17:57:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.554+0000 7f6eefbff700 1 -- 172.21.15.7:0/280958422 wait complete. 2024-01-11T17:57:22.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 -- 172.21.15.7:0/230888083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 msgr2=0x7ff3d81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/230888083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d81019d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40099c0 tx=0x7ff3d4009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 -- 172.21.15.7:0/230888083 shutdown_connections 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/230888083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 0x7ff3d81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/230888083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d81019d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/230888083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 0x7ff3d8102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 -- 172.21.15.7:0/230888083 >> 172.21.15.7:0/230888083 conn(0x7ff3d80fcb00 msgr2=0x7ff3d80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 -- 172.21.15.7:0/230888083 shutdown_connections 2024-01-11T17:57:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.783+0000 7ff3dfbff700 1 -- 172.21.15.7:0/230888083 wait complete. 2024-01-11T17:57:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.784+0000 7ff3dfbff700 1 Processor -- start 2024-01-11T17:57:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.784+0000 7ff3dfbff700 1 -- start start 2024-01-11T17:57:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.784+0000 7ff3dfbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d80753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.784+0000 7ff3dfbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 0x7ff3d8073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.784+0000 7ff3dfbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 0x7ff3d8073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.784+0000 7ff3dfbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80746e0 con 0x7ff3d81015b0 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dfbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8074850 con 0x7ff3d81027a0 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dfbff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81a1540 con 0x7ff3d8103140 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 0x7ff3d8073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dd99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d80753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dd99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d80753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46030/0 (socket says 172.21.15.7:46030) 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dd99b700 1 -- 172.21.15.7:0/3844329347 learned_addr learned my addr 172.21.15.7:0/3844329347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dd19a700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 0x7ff3d8073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 -- 172.21.15.7:0/3844329347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 msgr2=0x7ff3d8073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 0x7ff3d8073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 -- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 msgr2=0x7ff3d80753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d80753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 -- 172.21.15.7:0/3844329347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40095f0 con 0x7ff3d8103140 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dd99b700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d80753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3dd19a700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 0x7ff3d8073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.785+0000 7ff3de19c700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 0x7ff3d8073f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000efc0 tx=0x7ff3d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.786+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0018070 con 0x7ff3d8103140 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.786+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d81a17c0 con 0x7ff3d8103140 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.786+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3d000f040 con 0x7ff3d8103140 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.786+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000cdf0 con 0x7ff3d8103140 2024-01-11T17:57:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.786+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d81a1cd0 con 0x7ff3d8103140 2024-01-11T17:57:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.787+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d81089e0 con 0x7ff3d8103140 2024-01-11T17:57:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.789+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7ff3d0003a00 con 0x7ff3d8103140 2024-01-11T17:57:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.790+0000 7ff3ceffd700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3c40a1ff0 0x7ff3c40a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.790+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3d00cdc10 con 0x7ff3d8103140 2024-01-11T17:57:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.790+0000 7ff3dd99b700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3c40a1ff0 0x7ff3c40a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.791+0000 7ff3dd99b700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3c40a1ff0 0x7ff3c40a44b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000c00 tx=0x7ff3d4009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:22.793+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff3d0096870 con 0x7ff3d8103140 2024-01-11T17:57:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:22 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1157095231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:22.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:22 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/280958422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.004+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3d810cfc0 con 0x7ff3d8103140 2024-01-11T17:57:23.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.004+0000 7ff3ceffd700 1 -- 172.21.15.7:0/3844329347 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3d0099e90 con 0x7ff3d8103140 2024-01-11T17:57:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:57:23.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3c40a1ff0 msgr2=0x7ff3c40a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3c40a1ff0 0x7ff3c40a44b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000c00 tx=0x7ff3d4009530 comp rx=0 tx=0).stop 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 msgr2=0x7ff3d8073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 0x7ff3d8073f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000efc0 tx=0x7ff3d000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 shutdown_connections 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff3c40a1ff0 0x7ff3c40a44b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d8103140 0x7ff3d8073f40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d81015b0 0x7ff3d80753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 --2- 172.21.15.7:0/3844329347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d81027a0 0x7ff3d8073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 >> 172.21.15.7:0/3844329347 conn(0x7ff3d80fcb00 msgr2=0x7ff3d8105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 shutdown_connections 2024-01-11T17:57:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.007+0000 7ff3dfbff700 1 -- 172.21.15.7:0/3844329347 wait complete. 2024-01-11T17:57:23.024 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:22 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1157095231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.024 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:22 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/280958422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:57:23.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:22 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1157095231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:22 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/280958422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 -- 172.21.15.7:0/4041417004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 msgr2=0x7f5330102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 --2- 172.21.15.7:0/4041417004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 0x7f5330102c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5320009990 tx=0x7f5320009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 -- 172.21.15.7:0/4041417004 shutdown_connections 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 --2- 172.21.15.7:0/4041417004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 --2- 172.21.15.7:0/4041417004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53301015b0 0x7f53301019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 --2- 172.21.15.7:0/4041417004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 0x7f5330102c00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 -- 172.21.15.7:0/4041417004 >> 172.21.15.7:0/4041417004 conn(0x7f53300fcb00 msgr2=0x7f53300fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 -- 172.21.15.7:0/4041417004 shutdown_connections 2024-01-11T17:57:23.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.234+0000 7f5337bff700 1 -- 172.21.15.7:0/4041417004 wait complete. 2024-01-11T17:57:23.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 Processor -- start 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 -- start start 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53301015b0 0x7f53301996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 0x7f5330199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301a1270 con 0x7f5330103140 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301a13b0 con 0x7f53301015b0 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.235+0000 7f5337bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53301a1540 con 0x7f53301027a0 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533519a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 0x7f5330199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46050/0 (socket says 172.21.15.7:46050) 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 -- 172.21.15.7:0/1929028644 learned_addr learned my addr 172.21.15.7:0/1929028644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533599b700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53301015b0 0x7f53301996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 -- 172.21.15.7:0/1929028644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 msgr2=0x7f5330199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 0x7f5330199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 -- 172.21.15.7:0/1929028644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53301015b0 msgr2=0x7f53301996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53301015b0 0x7f53301996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 -- 172.21.15.7:0/1929028644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200095f0 con 0x7f5330103140 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.236+0000 7f533619c700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301a0ca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f532800ef30 tx=0x7f532800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.237+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328018070 con 0x7f5330103140 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.237+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53301a17c0 con 0x7f5330103140 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.237+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f532800f040 con 0x7f5330103140 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.237+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532800cd00 con 0x7f5330103140 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.237+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53301a1c80 con 0x7f5330103140 2024-01-11T17:57:23.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.238+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53300683f0 con 0x7f5330103140 2024-01-11T17:57:23.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.240+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f5328003a00 con 0x7f5330103140 2024-01-11T17:57:23.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.241+0000 7f5326ffd700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f531c0a1f70 0x7f531c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.241+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53280cd970 con 0x7f5330103140 2024-01-11T17:57:23.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.242+0000 7f533599b700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f531c0a1f70 0x7f531c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.242+0000 7f533599b700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f531c0a1f70 0x7f531c0a4430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5330102600 tx=0x7f532c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:23.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.245+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f53280965d0 con 0x7f5330103140 2024-01-11T17:57:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.458+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f533010cff0 con 0x7f5330103140 2024-01-11T17:57:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.458+0000 7f5326ffd700 1 -- 172.21.15.7:0/1929028644 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5328099bf0 con 0x7f5330103140 2024-01-11T17:57:23.459 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f531c0a1f70 msgr2=0x7f531c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f531c0a1f70 0x7f531c0a4430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5330102600 tx=0x7f532c007400 comp rx=0 tx=0).stop 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 msgr2=0x7f53301a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301a0ca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f532800ef30 tx=0x7f532800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 shutdown_connections 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f531c0a1f70 0x7f531c0a4430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5330103140 0x7f53301a0ca0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53301015b0 0x7f53301996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 --2- 172.21.15.7:0/1929028644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53301027a0 0x7f5330199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 >> 172.21.15.7:0/1929028644 conn(0x7f53300fcb00 msgr2=0x7f5330073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 shutdown_connections 2024-01-11T17:57:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.461+0000 7f5337bff700 1 -- 172.21.15.7:0/1929028644 wait complete. 2024-01-11T17:57:23.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:57:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 -- 172.21.15.7:0/1520082678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c101590 msgr2=0x7eff7c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 --2- 172.21.15.7:0/1520082678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c101590 0x7eff7c1019b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff780099c0 tx=0x7eff78009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 -- 172.21.15.7:0/1520082678 shutdown_connections 2024-01-11T17:57:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 --2- 172.21.15.7:0/1520082678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff7c103120 0x7eff7c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 --2- 172.21.15.7:0/1520082678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c101590 0x7eff7c1019b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 --2- 172.21.15.7:0/1520082678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c102780 0x7eff7c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 -- 172.21.15.7:0/1520082678 >> 172.21.15.7:0/1520082678 conn(0x7eff7c0fcb00 msgr2=0x7eff7c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 -- 172.21.15.7:0/1520082678 shutdown_connections 2024-01-11T17:57:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.685+0000 7eff845ff700 1 -- 172.21.15.7:0/1520082678 wait complete. 2024-01-11T17:57:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.686+0000 7eff845ff700 1 Processor -- start 2024-01-11T17:57:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.686+0000 7eff845ff700 1 -- start start 2024-01-11T17:57:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff845ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 0x7eff7c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff845ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c102780 0x7eff7c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff845ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff7c103120 0x7eff7c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff845ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1a1270 con 0x7eff7c101590 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff845ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1a13b0 con 0x7eff7c102780 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff845ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1a1540 con 0x7eff7c103120 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 0x7eff7c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 0x7eff7c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46072/0 (socket says 172.21.15.7:46072) 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 -- 172.21.15.7:0/872872615 learned_addr learned my addr 172.21.15.7:0/872872615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff82b9c700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff7c103120 0x7eff7c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff81b9a700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c102780 0x7eff7c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 -- 172.21.15.7:0/872872615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff7c103120 msgr2=0x7eff7c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff7c103120 0x7eff7c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 -- 172.21.15.7:0/872872615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c102780 msgr2=0x7eff7c199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c102780 0x7eff7c199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.687+0000 7eff8239b700 1 -- 172.21.15.7:0/872872615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c009520 con 0x7eff7c101590 2024-01-11T17:57:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.688+0000 7eff8239b700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 0x7eff7c199790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff78006010 tx=0x7eff7800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.688+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7801d070 con 0x7eff7c101590 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.688+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff78005020 con 0x7eff7c101590 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.688+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780095f0 con 0x7eff7c101590 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.688+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78004b10 con 0x7eff7c101590 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.688+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c1a1a50 con 0x7eff7c101590 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.689+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c0683f0 con 0x7eff7c101590 2024-01-11T17:57:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.691+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7eff78005190 con 0x7eff7c101590 2024-01-11T17:57:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.692+0000 7eff737fe700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff680a1ff0 0x7eff680a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.693+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff780ce810 con 0x7eff7c101590 2024-01-11T17:57:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.693+0000 7eff81b9a700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff680a1ff0 0x7eff680a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:23.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.693+0000 7eff81b9a700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff680a1ff0 0x7eff680a44b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009df0 tx=0x7eff6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:23.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.697+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7eff78097470 con 0x7eff7c101590 2024-01-11T17:57:23.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:23 smithi007 ceph-mon[33361]: pgmap v285: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:23.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:23 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3844329347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:23 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1929028644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.958+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff7c10ce10 con 0x7eff7c101590 2024-01-11T17:57:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.959+0000 7eff737fe700 1 -- 172.21.15.7:0/872872615 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11309 (secure 0 0 0) 0x7eff7809aa90 con 0x7eff7c101590 2024-01-11T17:57:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:23.960 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:34.314730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:57:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff680a1ff0 msgr2=0x7eff680a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff680a1ff0 0x7eff680a44b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009df0 tx=0x7eff6c009260 comp rx=0 tx=0).stop 2024-01-11T17:57:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 msgr2=0x7eff7c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 0x7eff7c199790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff78006010 tx=0x7eff7800f690 comp rx=0 tx=0).stop 2024-01-11T17:57:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 shutdown_connections 2024-01-11T17:57:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7eff680a1ff0 0x7eff680a44b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff7c103120 0x7eff7c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff7c101590 0x7eff7c199790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 --2- 172.21.15.7:0/872872615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff7c102780 0x7eff7c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.961+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 >> 172.21.15.7:0/872872615 conn(0x7eff7c0fcb00 msgr2=0x7eff7c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.962+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 shutdown_connections 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:23.962+0000 7eff845ff700 1 -- 172.21.15.7:0/872872615 wait complete. 2024-01-11T17:57:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:57:23.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:23 smithi016 ceph-mon[43195]: pgmap v285: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:23 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3844329347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:23 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1929028644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:24.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:23 smithi038 ceph-mon[43582]: pgmap v285: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:24.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:23 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3844329347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:24.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:23 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1929028644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:24.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 -- 172.21.15.7:0/484170094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c101590 msgr2=0x7fc34c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 --2- 172.21.15.7:0/484170094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c101590 0x7fc34c1019b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc348009990 tx=0x7fc348009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 -- 172.21.15.7:0/484170094 shutdown_connections 2024-01-11T17:57:24.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 --2- 172.21.15.7:0/484170094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c101590 0x7fc34c1019b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 --2- 172.21.15.7:0/484170094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c103120 0x7fc34c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 --2- 172.21.15.7:0/484170094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c102780 0x7fc34c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 -- 172.21.15.7:0/484170094 >> 172.21.15.7:0/484170094 conn(0x7fc34c0fcb00 msgr2=0x7fc34c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 -- 172.21.15.7:0/484170094 shutdown_connections 2024-01-11T17:57:24.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.186+0000 7fc3545ff700 1 -- 172.21.15.7:0/484170094 wait complete. 2024-01-11T17:57:24.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.187+0000 7fc3545ff700 1 Processor -- start 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 -- start start 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 0x7fc34c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c102780 0x7fc34c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c103120 0x7fc34c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19f030 con 0x7fc34c101590 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19f170 con 0x7fc34c102780 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc3545ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19f300 con 0x7fc34c103120 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 0x7fc34c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 0x7fc34c1974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46098/0 (socket says 172.21.15.7:46098) 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc351b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c102780 0x7fc34c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 -- 172.21.15.7:0/682934818 learned_addr learned my addr 172.21.15.7:0/682934818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc352b9c700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c103120 0x7fc34c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 -- 172.21.15.7:0/682934818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c103120 msgr2=0x7fc34c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c103120 0x7fc34c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 -- 172.21.15.7:0/682934818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c102780 msgr2=0x7fc34c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.188+0000 7fc35239b700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c102780 0x7fc34c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc35239b700 1 -- 172.21.15.7:0/682934818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3480095f0 con 0x7fc34c101590 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc352b9c700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c103120 0x7fc34c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc35239b700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 0x7fc34c1974b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc348000c00 tx=0x7fc348004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34801d070 con 0x7fc34c101590 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc348021410 con 0x7fc34c101590 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348017650 con 0x7fc34c101590 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c19f580 con 0x7fc34c101590 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.189+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c19f9a0 con 0x7fc34c101590 2024-01-11T17:57:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.190+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c0683f0 con 0x7fc34c101590 2024-01-11T17:57:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.192+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc348021580 con 0x7fc34c101590 2024-01-11T17:57:24.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.193+0000 7fc3437fe700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc3380a1f70 0x7fc3380a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.194+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3480cf9a0 con 0x7fc34c101590 2024-01-11T17:57:24.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.194+0000 7fc351b9a700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc3380a1f70 0x7fc3380a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.194+0000 7fc351b9a700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc3380a1f70 0x7fc3380a4430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009c60 tx=0x7fc33c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:24.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.196+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc348098600 con 0x7fc34c101590 2024-01-11T17:57:24.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.414+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc34c10ced0 con 0x7fc34c101590 2024-01-11T17:57:24.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.416+0000 7fc3437fe700 1 -- 172.21.15.7:0/682934818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24334 (secure 0 0 0) 0x7fc34809bc20 con 0x7fc34c101590 2024-01-11T17:57:24.416 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:24.417 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:56:26.328970+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc3380a1f70 msgr2=0x7fc3380a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc3380a1f70 0x7fc3380a4430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009c60 tx=0x7fc33c009260 comp rx=0 tx=0).stop 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 msgr2=0x7fc34c1974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 0x7fc34c1974b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc348000c00 tx=0x7fc348004730 comp rx=0 tx=0).stop 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 shutdown_connections 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc3380a1f70 0x7fc3380a4430 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc34c103120 0x7fc34c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc34c101590 0x7fc34c1974b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 --2- 172.21.15.7:0/682934818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c102780 0x7fc34c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 >> 172.21.15.7:0/682934818 conn(0x7fc34c0fcb00 msgr2=0x7fc34c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 shutdown_connections 2024-01-11T17:57:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.418+0000 7fc3545ff700 1 -- 172.21.15.7:0/682934818 wait complete. 2024-01-11T17:57:24.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:57:24.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:57:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 -- 172.21.15.7:0/169375197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 msgr2=0x7f81f4102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/169375197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4102c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81e00099c0 tx=0x7f81e0009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 -- 172.21.15.7:0/169375197 shutdown_connections 2024-01-11T17:57:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/169375197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f41015b0 0x7f81f41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/169375197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f4103140 0x7f81f41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/169375197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4102c00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 -- 172.21.15.7:0/169375197 >> 172.21.15.7:0/169375197 conn(0x7f81f40fcb00 msgr2=0x7f81f40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 -- 172.21.15.7:0/169375197 shutdown_connections 2024-01-11T17:57:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.647+0000 7f81fb1ff700 1 -- 172.21.15.7:0/169375197 wait complete. 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.648+0000 7f81fb1ff700 1 Processor -- start 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.648+0000 7f81fb1ff700 1 -- start start 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81fb1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f41015b0 0x7f81f4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81fb1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81fb1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f4103140 0x7f81f41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81fb1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41a11b0 con 0x7f81f41027a0 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81fb1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41a12f0 con 0x7f81f41015b0 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81fb1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41a1480 con 0x7f81f4103140 2024-01-11T17:57:24.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46110/0 (socket says 172.21.15.7:46110) 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 -- 172.21.15.7:0/3318779016 learned_addr learned my addr 172.21.15.7:0/3318779016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f979c700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f4103140 0x7f81f41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f8f9b700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f41015b0 0x7f81f4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 -- 172.21.15.7:0/3318779016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f4103140 msgr2=0x7f81f41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f4103140 0x7f81f41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 -- 172.21.15.7:0/3318779016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f41015b0 msgr2=0x7f81f4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f41015b0 0x7f81f4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.649+0000 7f81f3fff700 1 -- 172.21.15.7:0/3318779016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00095f0 con 0x7f81f41027a0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.650+0000 7f81f3fff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4199b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f81e0003730 tx=0x7f81e000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.650+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e001d070 con 0x7f81f41027a0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.650+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f41a1700 con 0x7f81f41027a0 2024-01-11T17:57:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.650+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81e00059a0 con 0x7f81f41027a0 2024-01-11T17:57:24.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.650+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0004a00 con 0x7f81f41027a0 2024-01-11T17:57:24.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.650+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f41a1c00 con 0x7f81f41027a0 2024-01-11T17:57:24.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.652+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f40683f0 con 0x7f81f41027a0 2024-01-11T17:57:24.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.655+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f81e0005210 con 0x7f81f41027a0 2024-01-11T17:57:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.656+0000 7f81f1ffb700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81e40a1f70 0x7f81e40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.656+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81e00ce9c0 con 0x7f81f41027a0 2024-01-11T17:57:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.657+0000 7f81f8f9b700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81e40a1f70 0x7f81e40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.657+0000 7f81f8f9b700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81e40a1f70 0x7f81e40a4430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81f4102600 tx=0x7f81e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.661+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f81e0097680 con 0x7f81f41027a0 2024-01-11T17:57:24.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:24 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/872872615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:24.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:24 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/682934818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:24.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.877+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81f41a1f40 con 0x7f81f41027a0 2024-01-11T17:57:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.877+0000 7f81f1ffb700 1 -- 172.21.15.7:0/3318779016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81e009aca0 con 0x7f81f41027a0 2024-01-11T17:57:24.878 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81e40a1f70 msgr2=0x7f81e40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81e40a1f70 0x7f81e40a4430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81f4102600 tx=0x7f81e8009260 comp rx=0 tx=0).stop 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 msgr2=0x7f81f4199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4199b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f81e0003730 tx=0x7f81e000f670 comp rx=0 tx=0).stop 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 shutdown_connections 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f81e40a1f70 0x7f81e40a4430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f4103140 0x7f81f41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81f41015b0 0x7f81f4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 --2- 172.21.15.7:0/3318779016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f41027a0 0x7f81f4199b70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 >> 172.21.15.7:0/3318779016 conn(0x7f81f40fcb00 msgr2=0x7f81f4073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 shutdown_connections 2024-01-11T17:57:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:24.880+0000 7f81fb1ff700 1 -- 172.21.15.7:0/3318779016 wait complete. 2024-01-11T17:57:24.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:57:24.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:24 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/872872615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:24.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:24 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/682934818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:25.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:24 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/872872615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:25.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:24 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/682934818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 -- 172.21.15.7:0/327999462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874075650 msgr2=0x7f2874073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 --2- 172.21.15.7:0/327999462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874075650 0x7f2874073aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2868009990 tx=0x7f2868009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 -- 172.21.15.7:0/327999462 shutdown_connections 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 --2- 172.21.15.7:0/327999462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874109470 0x7f287410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 --2- 172.21.15.7:0/327999462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874075650 0x7f2874073aa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 --2- 172.21.15.7:0/327999462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874074070 0x7f28740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 -- 172.21.15.7:0/327999462 >> 172.21.15.7:0/327999462 conn(0x7f28740fcb00 msgr2=0x7f28740fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 -- 172.21.15.7:0/327999462 shutdown_connections 2024-01-11T17:57:25.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.108+0000 7f287b1ff700 1 -- 172.21.15.7:0/327999462 wait complete. 2024-01-11T17:57:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 Processor -- start 2024-01-11T17:57:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 -- start start 2024-01-11T17:57:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 0x7f2874199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874075650 0x7f2874199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874109470 0x7f28741a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28741a11c0 con 0x7f2874074070 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28741a1300 con 0x7f2874075650 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.109+0000 7f287b1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28741a1490 con 0x7f2874109470 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 0x7f2874199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 0x7f2874199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46124/0 (socket says 172.21.15.7:46124) 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 -- 172.21.15.7:0/3501849240 learned_addr learned my addr 172.21.15.7:0/3501849240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f287979c700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874109470 0x7f28741a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2873fff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874075650 0x7f2874199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 -- 172.21.15.7:0/3501849240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874109470 msgr2=0x7f28741a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874109470 0x7f28741a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 -- 172.21.15.7:0/3501849240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874075650 msgr2=0x7f2874199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874075650 0x7f2874199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f2878f9b700 1 -- 172.21.15.7:0/3501849240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680095f0 con 0x7f2874074070 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.110+0000 7f287979c700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874109470 0x7f28741a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.111+0000 7f2878f9b700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 0x7f2874199640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2868005910 tx=0x7f2868009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.111+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286801d070 con 0x7f2874074070 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.111+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2868021410 con 0x7f2874074070 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.111+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28741a1710 con 0x7f2874074070 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.111+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868017650 con 0x7f2874074070 2024-01-11T17:57:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.111+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28741a1b30 con 0x7f2874074070 2024-01-11T17:57:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.112+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2874108230 con 0x7f2874074070 2024-01-11T17:57:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.114+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f2868021580 con 0x7f2874074070 2024-01-11T17:57:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.115+0000 7f2871ffb700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28640a1f70 0x7f28640a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.116+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28680cf130 con 0x7f2874074070 2024-01-11T17:57:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.116+0000 7f2873fff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28640a1f70 0x7f28640a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.116+0000 7f2873fff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28640a1f70 0x7f28640a4430 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2874075370 tx=0x7f2860009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.119+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2868097d90 con 0x7f2874074070 2024-01-11T17:57:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.333+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f287410f090 con 0x7f2874074070 2024-01-11T17:57:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.334+0000 7f2871ffb700 1 -- 172.21.15.7:0/3501849240 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2868021880 con 0x7f2874074070 2024-01-11T17:57:25.335 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28640a1f70 msgr2=0x7f28640a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28640a1f70 0x7f28640a4430 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2874075370 tx=0x7f2860009260 comp rx=0 tx=0).stop 2024-01-11T17:57:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 msgr2=0x7f2874199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 0x7f2874199640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2868005910 tx=0x7f2868009580 comp rx=0 tx=0).stop 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 shutdown_connections 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28640a1f70 0x7f28640a4430 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2874109470 0x7f28741a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2874074070 0x7f2874199640 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 --2- 172.21.15.7:0/3501849240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874075650 0x7f2874199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.336+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 >> 172.21.15.7:0/3501849240 conn(0x7f28740fcb00 msgr2=0x7f2874105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.337+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 shutdown_connections 2024-01-11T17:57:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.337+0000 7f287b1ff700 1 -- 172.21.15.7:0/3501849240 wait complete. 2024-01-11T17:57:25.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 -- 172.21.15.7:0/1230067008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01027a0 msgr2=0x7f04f0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 --2- 172.21.15.7:0/1230067008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01027a0 0x7f04f0102c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04e40099c0 tx=0x7f04e4009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 -- 172.21.15.7:0/1230067008 shutdown_connections 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 --2- 172.21.15.7:0/1230067008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f01015b0 0x7f04f01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 --2- 172.21.15.7:0/1230067008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0103140 0x7f04f01099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 --2- 172.21.15.7:0/1230067008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01027a0 0x7f04f0102c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 -- 172.21.15.7:0/1230067008 >> 172.21.15.7:0/1230067008 conn(0x7f04f00fcb00 msgr2=0x7f04f00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 -- 172.21.15.7:0/1230067008 shutdown_connections 2024-01-11T17:57:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.567+0000 7f04f4bff700 1 -- 172.21.15.7:0/1230067008 wait complete. 2024-01-11T17:57:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 Processor -- start 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 -- start start 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01015b0 0x7f04f01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 0x7f04f0199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f0103140 0x7f04f01a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01a1260 con 0x7f04f01027a0 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01a13a0 con 0x7f04f01015b0 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.568+0000 7f04f4bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01a1530 con 0x7f04f0103140 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 0x7f04f0199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 0x7f04f0199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46138/0 (socket says 172.21.15.7:46138) 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 -- 172.21.15.7:0/2460404647 learned_addr learned my addr 172.21.15.7:0/2460404647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04ee59c700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01015b0 0x7f04f01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04eed9d700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f0103140 0x7f04f01a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 -- 172.21.15.7:0/2460404647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f0103140 msgr2=0x7f04f01a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f0103140 0x7f04f01a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 -- 172.21.15.7:0/2460404647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01015b0 msgr2=0x7f04f01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01015b0 0x7f04f01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 -- 172.21.15.7:0/2460404647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e40095f0 con 0x7f04f01027a0 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04eed9d700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f0103140 0x7f04f01a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04edd9b700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 0x7f04f0199c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003730 tx=0x7f04e400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e401d070 con 0x7f04f01027a0 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04e400fe70 con 0x7f04f01027a0 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.569+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f01a17b0 con 0x7f04f01027a0 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.570+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4017650 con 0x7f04f01027a0 2024-01-11T17:57:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.570+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f01a1bd0 con 0x7f04f01027a0 2024-01-11T17:57:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.571+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f00683f0 con 0x7f04f01027a0 2024-01-11T17:57:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.573+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f04e4022470 con 0x7f04f01027a0 2024-01-11T17:57:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.574+0000 7f04df7fe700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f04d80a1f70 0x7f04d80a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.574+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f04e40cf250 con 0x7f04f01027a0 2024-01-11T17:57:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.574+0000 7f04ee59c700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f04d80a1f70 0x7f04d80a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.575+0000 7f04ee59c700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f04d80a1f70 0x7f04d80a4430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04f0102600 tx=0x7f04e000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.578+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f04e4097eb0 con 0x7f04f01027a0 2024-01-11T17:57:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.789+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04f01a1df0 con 0x7f04f01027a0 2024-01-11T17:57:25.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.789+0000 7f04df7fe700 1 -- 172.21.15.7:0/2460404647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04e409b4d0 con 0x7f04f01027a0 2024-01-11T17:57:25.790 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:57:25.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.791+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f04d80a1f70 msgr2=0x7f04d80a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.791+0000 7f04f4bff700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f04d80a1f70 0x7f04d80a4430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04f0102600 tx=0x7f04e000a3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.791+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 msgr2=0x7f04f0199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 0x7f04f0199c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003730 tx=0x7f04e400fa00 comp rx=0 tx=0).stop 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 shutdown_connections 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f04d80a1f70 0x7f04d80a4430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04f0103140 0x7f04f01a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04f01015b0 0x7f04f01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 --2- 172.21.15.7:0/2460404647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f01027a0 0x7f04f0199c20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 >> 172.21.15.7:0/2460404647 conn(0x7f04f00fcb00 msgr2=0x7f04f0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 shutdown_connections 2024-01-11T17:57:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:25.792+0000 7f04f4bff700 1 -- 172.21.15.7:0/2460404647 wait complete. 2024-01-11T17:57:25.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:57:25.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:25 smithi016 ceph-mon[43195]: pgmap v286: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:25.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:25 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3318779016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:25.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:25 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3501849240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:25.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 10]}]: dispatch 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 -- 172.21.15.7:0/4217866645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f01027a0 msgr2=0x7f38f0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 --2- 172.21.15.7:0/4217866645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f01027a0 0x7f38f0102c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009920 tx=0x7f38d8009c30 comp rx=0 tx=0).stop 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 -- 172.21.15.7:0/4217866645 shutdown_connections 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 --2- 172.21.15.7:0/4217866645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f01015b0 0x7f38f01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 --2- 172.21.15.7:0/4217866645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0103140 0x7f38f01099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 --2- 172.21.15.7:0/4217866645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f01027a0 0x7f38f0102c00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 -- 172.21.15.7:0/4217866645 >> 172.21.15.7:0/4217866645 conn(0x7f38f00fcb00 msgr2=0x7f38f00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 -- 172.21.15.7:0/4217866645 shutdown_connections 2024-01-11T17:57:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.018+0000 7f38f49ff700 1 -- 172.21.15.7:0/4217866645 wait complete. 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.019+0000 7f38f49ff700 1 Processor -- start 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.019+0000 7f38f49ff700 1 -- start start 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38f49ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 0x7f38f0197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38f49ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f01027a0 0x7f38f0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38f49ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f0103140 0x7f38f019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38f49ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f019f0a0 con 0x7f38f01015b0 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38f49ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f019f1e0 con 0x7f38f0103140 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38f49ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f019f370 con 0x7f38f01027a0 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 0x7f38f0197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 0x7f38f0197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46166/0 (socket says 172.21.15.7:46166) 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 -- 172.21.15.7:0/437324239 learned_addr learned my addr 172.21.15.7:0/437324239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38e7fff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f01027a0 0x7f38f0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38eed9d700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f0103140 0x7f38f019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 -- 172.21.15.7:0/437324239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f01027a0 msgr2=0x7f38f0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f01027a0 0x7f38f0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 -- 172.21.15.7:0/437324239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f0103140 msgr2=0x7f38f019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f0103140 0x7f38f019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38ee59c700 1 -- 172.21.15.7:0/437324239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009520 con 0x7f38f01015b0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.020+0000 7f38eed9d700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f0103140 0x7f38f019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.021+0000 7f38ee59c700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 0x7f38f0197520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f38e000e9f0 tx=0x7f38e000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.021+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0004220 con 0x7f38f01015b0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.021+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e0004380 con 0x7f38f01015b0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.021+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f019f650 con 0x7f38f01015b0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.021+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0003c00 con 0x7f38f01015b0 2024-01-11T17:57:26.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.021+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f019fbb0 con 0x7f38f01015b0 2024-01-11T17:57:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.022+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f01089e0 con 0x7f38f01015b0 2024-01-11T17:57:26.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.024+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f38e000c7c0 con 0x7f38f01015b0 2024-01-11T17:57:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.026+0000 7f38e77fe700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38dc0a1ff0 0x7f38dc0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.026+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f38e0014070 con 0x7f38f01015b0 2024-01-11T17:57:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.026+0000 7f38e7fff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38dc0a1ff0 0x7f38dc0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:25 smithi038 ceph-mon[43582]: pgmap v286: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:25 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3318779016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:25 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3501849240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 10]}]: dispatch 2024-01-11T17:57:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.026+0000 7f38e7fff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38dc0a1ff0 0x7f38dc0a44b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38d800fa40 tx=0x7f38d8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:26.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.029+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f38e0096da0 con 0x7f38f01015b0 2024-01-11T17:57:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:25 smithi007 ceph-mon[33361]: pgmap v286: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:25 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3318779016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:25 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3501849240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 10]}]: dispatch 2024-01-11T17:57:26.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.242+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f38f010cea0 con 0x7f38f01015b0 2024-01-11T17:57:26.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.242+0000 7f38e77fe700 1 -- 172.21.15.7:0/437324239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f38e009a3c0 con 0x7f38f01015b0 2024-01-11T17:57:26.243 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38dc0a1ff0 msgr2=0x7f38dc0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38dc0a1ff0 0x7f38dc0a44b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38d800fa40 tx=0x7f38d8019040 comp rx=0 tx=0).stop 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 msgr2=0x7f38f0197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 0x7f38f0197520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f38e000e9f0 tx=0x7f38e000ed00 comp rx=0 tx=0).stop 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 shutdown_connections 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38dc0a1ff0 0x7f38dc0a44b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f0103140 0x7f38f019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f01015b0 0x7f38f0197520 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 --2- 172.21.15.7:0/437324239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38f01027a0 0x7f38f0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 >> 172.21.15.7:0/437324239 conn(0x7f38f00fcb00 msgr2=0x7f38f0105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 shutdown_connections 2024-01-11T17:57:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.245+0000 7f38f49ff700 1 -- 172.21.15.7:0/437324239 wait complete. 2024-01-11T17:57:26.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:57:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/3037546543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4107370 msgr2=0x7fe5b4109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/3037546543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4107370 0x7fe5b4109760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00b0a0 tx=0x7fe5ac00b3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/3037546543 shutdown_connections 2024-01-11T17:57:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/3037546543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4102110 0x7fe5b4104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/3037546543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4107370 0x7fe5b4109760 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/3037546543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b4104a40 0x7fe5b4106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/3037546543 >> 172.21.15.7:0/3037546543 conn(0x7fe5b40fba60 msgr2=0x7fe5b40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/3037546543 shutdown_connections 2024-01-11T17:57:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.472+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/3037546543 wait complete. 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.473+0000 7fe5ba5ff700 1 Processor -- start 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.473+0000 7fe5ba5ff700 1 -- start start 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5ba5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b4102110 0x7fe5b41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5ba5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 0x7fe5b4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5ba5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4107370 0x7fe5b419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5ba5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419f120 con 0x7fe5b4104a40 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5ba5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419f260 con 0x7fe5b4102110 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5ba5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419f3f0 con 0x7fe5b4107370 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 0x7fe5b4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 0x7fe5b4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46182/0 (socket says 172.21.15.7:46182) 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 -- 172.21.15.7:0/2183318923 learned_addr learned my addr 172.21.15.7:0/2183318923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b3fff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b4102110 0x7fe5b41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b8b9c700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4107370 0x7fe5b419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 -- 172.21.15.7:0/2183318923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4107370 msgr2=0x7fe5b419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4107370 0x7fe5b419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 -- 172.21.15.7:0/2183318923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b4102110 msgr2=0x7fe5b41975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b4102110 0x7fe5b41975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.474+0000 7fe5b37fe700 1 -- 172.21.15.7:0/2183318923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac009d00 con 0x7fe5b4104a40 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.475+0000 7fe5b37fe700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 0x7fe5b4197ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e9a0 tx=0x7fe5a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.475+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8009790 con 0x7fe5b4104a40 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.475+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5a8010430 con 0x7fe5b4104a40 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.475+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b419f6d0 con 0x7fe5b4104a40 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.475+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8018680 con 0x7fe5b4104a40 2024-01-11T17:57:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.475+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b419fbe0 con 0x7fe5b4104a40 2024-01-11T17:57:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.476+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b40fd5e0 con 0x7fe5b4104a40 2024-01-11T17:57:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.478+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fe5a80105a0 con 0x7fe5b4104a40 2024-01-11T17:57:26.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.479+0000 7fe5b17fa700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe59c0a1f70 0x7fe59c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.480+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe5a8014070 con 0x7fe5b4104a40 2024-01-11T17:57:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.480+0000 7fe5b3fff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe59c0a1f70 0x7fe59c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.480+0000 7fe5b3fff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe59c0a1f70 0x7fe59c0a4430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009c60 tx=0x7fe5a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:26.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.483+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe5a8052380 con 0x7fe5b4104a40 2024-01-11T17:57:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.744+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5b410cdd0 con 0x7fe5b4104a40 2024-01-11T17:57:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.745+0000 7fe5b17fa700 1 -- 172.21.15.7:0/2183318923 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11309 (secure 0 0 0) 0x7fe5a8096c30 con 0x7fe5b4104a40 2024-01-11T17:57:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:26.745 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:34.314730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:57:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.747+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe59c0a1f70 msgr2=0x7fe59c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.747+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe59c0a1f70 0x7fe59c0a4430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009c60 tx=0x7fe5a4009260 comp rx=0 tx=0).stop 2024-01-11T17:57:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.747+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 msgr2=0x7fe5b4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.747+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 0x7fe5b4197ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e9a0 tx=0x7fe5a800ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 shutdown_connections 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe59c0a1f70 0x7fe59c0a4430 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5b4107370 0x7fe5b419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b4102110 0x7fe5b41975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 --2- 172.21.15.7:0/2183318923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5b4104a40 0x7fe5b4197ae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 >> 172.21.15.7:0/2183318923 conn(0x7fe5b40fba60 msgr2=0x7fe5b41029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 shutdown_connections 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.748+0000 7fe5ba5ff700 1 -- 172.21.15.7:0/2183318923 wait complete. 2024-01-11T17:57:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:57:26.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:26.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 -- 172.21.15.7:0/1930567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120103120 msgr2=0x7fb1201099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 --2- 172.21.15.7:0/1930567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120103120 0x7fb1201099d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb11800b0d0 tx=0x7fb11800b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 -- 172.21.15.7:0/1930567052 shutdown_connections 2024-01-11T17:57:26.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 --2- 172.21.15.7:0/1930567052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb120101590 0x7fb1201019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 --2- 172.21.15.7:0/1930567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120103120 0x7fb1201099d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 --2- 172.21.15.7:0/1930567052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb120102780 0x7fb120102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 -- 172.21.15.7:0/1930567052 >> 172.21.15.7:0/1930567052 conn(0x7fb1200fcb00 msgr2=0x7fb1200fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:26.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 -- 172.21.15.7:0/1930567052 shutdown_connections 2024-01-11T17:57:26.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.973+0000 7fb1257ff700 1 -- 172.21.15.7:0/1930567052 wait complete. 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 Processor -- start 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 -- start start 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 0x7fb1201975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb120102780 0x7fb120197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb120103120 0x7fb12019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12019f120 con 0x7fb120101590 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12019f260 con 0x7fb120102780 2024-01-11T17:57:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.974+0000 7fb1257ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12019f3f0 con 0x7fb120103120 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 0x7fb1201975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 0x7fb1201975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46204/0 (socket says 172.21.15.7:46204) 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 -- 172.21.15.7:0/2316618393 learned_addr learned my addr 172.21.15.7:0/2316618393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb117fff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb120102780 0x7fb120197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11f7fe700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb120103120 0x7fb12019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 -- 172.21.15.7:0/2316618393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb120103120 msgr2=0x7fb12019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb120103120 0x7fb12019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 -- 172.21.15.7:0/2316618393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb120102780 msgr2=0x7fb120197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb120102780 0x7fb120197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 -- 172.21.15.7:0/2316618393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb118009d00 con 0x7fb120101590 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.975+0000 7fb11effd700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 0x7fb1201975a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb11000e970 tx=0x7fb11000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.976+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110009790 con 0x7fb120101590 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.976+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12019f6d0 con 0x7fb120101590 2024-01-11T17:57:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.976+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb11000c7e0 con 0x7fb120101590 2024-01-11T17:57:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.976+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110003c10 con 0x7fb120101590 2024-01-11T17:57:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.976+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12019fbe0 con 0x7fb120101590 2024-01-11T17:57:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.977+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb120108940 con 0x7fb120101590 2024-01-11T17:57:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.979+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb11000cc50 con 0x7fb120101590 2024-01-11T17:57:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.980+0000 7fb11cff9700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb10c0a1f70 0x7fb10c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.980+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb110014070 con 0x7fb120101590 2024-01-11T17:57:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.980+0000 7fb117fff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb10c0a1f70 0x7fb10c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.981+0000 7fb117fff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb10c0a1f70 0x7fb10c0a4430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb108009c60 tx=0x7fb108009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:26.984+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb110096bd0 con 0x7fb120101590 2024-01-11T17:57:26.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:26 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2460404647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:26.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:26 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/437324239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:26.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 10]}]': finished 2024-01-11T17:57:26.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:26 smithi016 ceph-mon[43195]: osdmap e87: 12 total, 12 up, 12 in 2024-01-11T17:57:27.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:26 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2460404647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:27.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:26 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/437324239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:27.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 10]}]': finished 2024-01-11T17:57:27.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:26 smithi038 ceph-mon[43582]: osdmap e87: 12 total, 12 up, 12 in 2024-01-11T17:57:27.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:26 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2460404647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:27.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:26 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/437324239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:27.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 10]}]': finished 2024-01-11T17:57:27.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:26 smithi007 ceph-mon[33361]: osdmap e87: 12 total, 12 up, 12 in 2024-01-11T17:57:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.197+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1200683f0 con 0x7fb120101590 2024-01-11T17:57:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.199+0000 7fb11cff9700 1 -- 172.21.15.7:0/2316618393 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fb11009a1f0 con 0x7fb120101590 2024-01-11T17:57:27.200 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:26.341726+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb10c0a1f70 msgr2=0x7fb10c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb10c0a1f70 0x7fb10c0a4430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb108009c60 tx=0x7fb108009260 comp rx=0 tx=0).stop 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 msgr2=0x7fb1201975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 0x7fb1201975a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb11000e970 tx=0x7fb11000ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 shutdown_connections 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb10c0a1f70 0x7fb10c0a4430 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb120103120 0x7fb12019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb120101590 0x7fb1201975a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 --2- 172.21.15.7:0/2316618393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb120102780 0x7fb120197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 >> 172.21.15.7:0/2316618393 conn(0x7fb1200fcb00 msgr2=0x7fb120105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 shutdown_connections 2024-01-11T17:57:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.202+0000 7fb1257ff700 1 -- 172.21.15.7:0/2316618393 wait complete. 2024-01-11T17:57:27.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:57:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 -- 172.21.15.7:0/408724568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8102780 msgr2=0x7f89b8102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 --2- 172.21.15.7:0/408724568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8102780 0x7f89b8102be0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009990 tx=0x7f89ac009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 -- 172.21.15.7:0/408724568 shutdown_connections 2024-01-11T17:57:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 --2- 172.21.15.7:0/408724568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8103120 0x7f89b81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 --2- 172.21.15.7:0/408724568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8101590 0x7f89b81019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 --2- 172.21.15.7:0/408724568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8102780 0x7f89b8102be0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 -- 172.21.15.7:0/408724568 >> 172.21.15.7:0/408724568 conn(0x7f89b80fcb00 msgr2=0x7f89b80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 -- 172.21.15.7:0/408724568 shutdown_connections 2024-01-11T17:57:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.430+0000 7f89bcdff700 1 -- 172.21.15.7:0/408724568 wait complete. 2024-01-11T17:57:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 Processor -- start 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 -- start start 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8101590 0x7f89b8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8102780 0x7f89b8199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 0x7f89b81a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81a1290 con 0x7f89b8103120 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81a13d0 con 0x7f89b8102780 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.432+0000 7f89bcdff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81a1560 con 0x7f89b8101590 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 0x7f89b81a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 0x7f89b81a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46214/0 (socket says 172.21.15.7:46214) 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 -- 172.21.15.7:0/204875725 learned_addr learned my addr 172.21.15.7:0/204875725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b659c700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8101590 0x7f89b8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b5d9b700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8102780 0x7f89b8199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 -- 172.21.15.7:0/204875725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8101590 msgr2=0x7f89b8199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8101590 0x7f89b8199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 -- 172.21.15.7:0/204875725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8102780 msgr2=0x7f89b8199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8102780 0x7f89b8199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 -- 172.21.15.7:0/204875725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0095f0 con 0x7f89b8103120 2024-01-11T17:57:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89b6d9d700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 0x7f89b81a0cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007ee0 tx=0x7f89b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.433+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x7f89b8103120 2024-01-11T17:57:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.434+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b81a1840 con 0x7f89b8103120 2024-01-11T17:57:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.434+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b000f040 con 0x7f89b8103120 2024-01-11T17:57:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.434+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0005580 con 0x7f89b8103120 2024-01-11T17:57:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.434+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b81a1d00 con 0x7f89b8103120 2024-01-11T17:57:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.435+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b80683f0 con 0x7f89b8103120 2024-01-11T17:57:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.437+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f89b000db20 con 0x7f89b8103120 2024-01-11T17:57:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.438+0000 7f89a77fe700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89a00a1f70 0x7f89a00a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.438+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f89b00cd630 con 0x7f89b8103120 2024-01-11T17:57:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.438+0000 7f89b659c700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89a00a1f70 0x7f89a00a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.439+0000 7f89b659c700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89a00a1f70 0x7f89a00a4430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f89b81025e0 tx=0x7f89a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.442+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f89b00961e0 con 0x7f89b8103120 2024-01-11T17:57:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.706+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89b810ce00 con 0x7f89b8103120 2024-01-11T17:57:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.706+0000 7f89a77fe700 1 -- 172.21.15.7:0/204875725 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11309 (secure 0 0 0) 0x7f89b0099800 con 0x7f89b8103120 2024-01-11T17:57:27.707 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:27.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:34.314730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89a00a1f70 msgr2=0x7f89a00a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89a00a1f70 0x7f89a00a4430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f89b81025e0 tx=0x7f89a8009260 comp rx=0 tx=0).stop 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 msgr2=0x7f89b81a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 0x7f89b81a0cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007ee0 tx=0x7f89b000d3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 shutdown_connections 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89a00a1f70 0x7f89a00a4430 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89b8101590 0x7f89b8199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b8103120 0x7f89b81a0cc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 --2- 172.21.15.7:0/204875725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b8102780 0x7f89b8199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 >> 172.21.15.7:0/204875725 conn(0x7f89b80fcb00 msgr2=0x7f89b8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 shutdown_connections 2024-01-11T17:57:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.709+0000 7f89bcdff700 1 -- 172.21.15.7:0/204875725 wait complete. 2024-01-11T17:57:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:57:27.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-01-11T17:56:08.339093+0000', 'modified': '2024-01-11T17:56:34.314730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24602, 'mds_1': 24601, 'mds_2': 24577}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24547': {'gid': 24547, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/1587519060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 1587519060}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 1587519060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/2455039829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 2455039829}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 2455039829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24565': {'gid': 24565, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/2512926203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 2512926203}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2512926203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24577': {'gid': 24577, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.38:6839/443773776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 443773776}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 443773776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24601': {'gid': 24601, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.7:6841/898637958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6840', 'nonce': 898637958}, {'type': 'v1', 'addr': '172.21.15.7:6841', 'nonce': 898637958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/1892769619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 1892769619}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 1892769619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-11T17:57:27.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-01-11T17:57:27.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-01-11T17:57:27.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.936+0000 7f139bfff700 1 -- 172.21.15.7:0/106476737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941027a0 msgr2=0x7f1394102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.936+0000 7f139bfff700 1 --2- 172.21.15.7:0/106476737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941027a0 0x7f1394102c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1384009990 tx=0x7f1384009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 -- 172.21.15.7:0/106476737 shutdown_connections 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 --2- 172.21.15.7:0/106476737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1394103140 0x7f13941099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 --2- 172.21.15.7:0/106476737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941015b0 0x7f13941019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 --2- 172.21.15.7:0/106476737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941027a0 0x7f1394102c00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 -- 172.21.15.7:0/106476737 >> 172.21.15.7:0/106476737 conn(0x7f13940fcb00 msgr2=0x7f13940fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 -- 172.21.15.7:0/106476737 shutdown_connections 2024-01-11T17:57:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.937+0000 7f139bfff700 1 -- 172.21.15.7:0/106476737 wait complete. 2024-01-11T17:57:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 Processor -- start 2024-01-11T17:57:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 -- start start 2024-01-11T17:57:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941015b0 0x7f13940753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 0x7f1394073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1394103140 0x7f1394073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940746e0 con 0x7f13941027a0 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394074850 con 0x7f1394103140 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139bfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941a14d0 con 0x7f13941015b0 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 0x7f1394073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 0x7f1394073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46226/0 (socket says 172.21.15.7:46226) 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f139959a700 1 -- 172.21.15.7:0/2912666259 learned_addr learned my addr 172.21.15.7:0/2912666259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.938+0000 7f1399d9b700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941015b0 0x7f13940753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139a59c700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1394103140 0x7f1394073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139959a700 1 -- 172.21.15.7:0/2912666259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941015b0 msgr2=0x7f13940753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139959a700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941015b0 0x7f13940753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139959a700 1 -- 172.21.15.7:0/2912666259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1394103140 msgr2=0x7f1394073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139959a700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1394103140 0x7f1394073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139959a700 1 -- 172.21.15.7:0/2912666259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13840095f0 con 0x7f13941027a0 2024-01-11T17:57:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139959a700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 0x7f1394073a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f13840098e0 tx=0x7f1384005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f138affd700 1 -- 172.21.15.7:0/2912666259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138401d070 con 0x7f13941027a0 2024-01-11T17:57:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941a1750 con 0x7f13941027a0 2024-01-11T17:57:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f138affd700 1 -- 172.21.15.7:0/2912666259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1384022470 con 0x7f13941027a0 2024-01-11T17:57:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.939+0000 7f138affd700 1 -- 172.21.15.7:0/2912666259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384017650 con 0x7f13941027a0 2024-01-11T17:57:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.940+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a1b80 con 0x7f13941027a0 2024-01-11T17:57:27.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.942+0000 7f138affd700 1 -- 172.21.15.7:0/2912666259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f13840225e0 con 0x7f13941027a0 2024-01-11T17:57:27.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 Processor -- start 2024-01-11T17:57:27.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 -- start start 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f138affd700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f13800a1f70 0x7f13800a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1358a8040 con 0x55c1359b23b0 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13590bad0 con 0x55c1359b2d30 2024-01-11T17:57:27.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1359093f0 con 0x55c1359b1520 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f138affd700 1 -- 172.21.15.7:0/2912666259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f13840cf330 con 0x7f13941027a0 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.944+0000 7f139a59c700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f13800a1f70 0x7f13800a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46242/0 (socket says 172.21.15.7:46242) 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f1399d9b700 1 -- 172.21.15.7:0/1719582651 learned_addr learned my addr 172.21.15.7:0/1719582651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139a59c700 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 -- 172.21.15.7:0/1719582651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 msgr2=0x55c1359b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 -- 172.21.15.7:0/1719582651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 msgr2=0x55c1359b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 -- 172.21.15.7:0/1719582651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c135779360 con 0x55c1359b2d30 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f1399d9b700 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139959a700 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359b0f40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13840cc9e0 tx=0x7f1384005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5216cadb646650c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.945+0000 7f139a59c700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f13800a1f70 0x7f13800a4430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f138c009ec0 tx=0x7f138c009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f137ffff700 1 -- 172.21.15.7:0/1719582651 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840cf160 con 0x55c1359b2d30 2024-01-11T17:57:27.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f137ffff700 1 -- 172.21.15.7:0/1719582651 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f138409b920 con 0x55c1359b2d30 2024-01-11T17:57:27.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f137ffff700 1 -- 172.21.15.7:0/1719582651 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840a4170 con 0x55c1359b2d30 2024-01-11T17:57:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 -- 172.21.15.7:0/1719582651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 msgr2=0x55c1359b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359b0f40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13840cc9e0 tx=0x7f1384005fd0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 -- 172.21.15.7:0/1719582651 shutdown_connections 2024-01-11T17:57:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/1719582651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359b0f40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 -- 172.21.15.7:0/1719582651 >> 172.21.15.7:0/1719582651 conn(0x55c135926750 msgr2=0x55c1359abd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.946+0000 7f13afd5fb80 1 -- 172.21.15.7:0/1719582651 shutdown_connections 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.947+0000 7f13afd5fb80 1 -- 172.21.15.7:0/1719582651 wait complete. 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.947+0000 7f13afd5fb80 1 -- start start 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.947+0000 7f13afd5fb80 1 Processor -- start 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359bab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359bb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c135904ce0 con 0x55c1359b23b0 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c135873350 con 0x55c1359b2d30 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c135872ff0 con 0x55c1359b1520 2024-01-11T17:57:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f13afd5fb80 10 client.0 _ll_register_callbacks cb 0x55c1358a56d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359bab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:49758/0 (socket says 172.21.15.7:49758) 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f1399d9b700 1 -- 172.21.15.7:0/3684576312 learned_addr learned my addr 172.21.15.7:0/3684576312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139a59c700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139959a700 1 -- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 msgr2=0x55c1359bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139959a700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139959a700 1 -- 172.21.15.7:0/3684576312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 msgr2=0x55c1359c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139959a700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f139959a700 1 -- 172.21.15.7:0/3684576312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390009960 con 0x55c1359b23b0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.948+0000 7f1399d9b700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f139959a700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359bb060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f138409ba90 tx=0x7f1384005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b97dca46fe5c9e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f137d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:57:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f137d7fa700 1 -- 172.21.15.7:0/3684576312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138409b740 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1359a3fc0 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f137d7fa700 1 -- 172.21.15.7:0/3684576312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f138409c4e0 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f137d7fa700 1 -- 172.21.15.7:0/3684576312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f138409bde0 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f13afd5fb80 10 client.15180 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f13941a1b80 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f137d7fa700 1 -- 172.21.15.7:0/3684576312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1359a3fc0 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.949+0000 7f137d7fa700 1 -- 172.21.15.7:0/3684576312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13840afb30 con 0x55c1359b23b0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f13afd5fb80 10 client.15180 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f13afd5fb80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f13afd5fb80 20 client.15180 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c1359c6230 0x55c1359c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f13afd5fb80 4 client.15180 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1358fdb70 con 0x55c1359c6230 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f1399d9b700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c1359c6230 0x55c1359c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f1399d9b700 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c1359c6230 0x55c1359c5070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff12ad1249fa56a0 server_cookie=adf2e4cb134cc34c in_seq=0 out_seq=0 2024-01-11T17:57:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.950+0000 7f137d7fa700 10 client.15180 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.951+0000 7f137d7fa700 1 -- 172.21.15.7:0/3684576312 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1358fdb70 con 0x55c1359c6230 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.951+0000 7f137d7fa700 10 client.15180 handle_command_reply: tid=0 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.951+0000 7f13afd5fb80 1 client.15180 shutdown 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.951+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 msgr2=0x55c1359bb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359bb060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f138409ba90 tx=0x7f1384005ef0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 shutdown_connections 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c1359c6230 0x55c1359c5070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b1520 0x55c1359bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f137d7fa700 0 client.15180 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3684576312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b23b0 0x55c1359bb060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 >> 172.21.15.7:0/3684576312 conn(0x55c135926750 msgr2=0x55c1359afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 shutdown_connections 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3684576312 wait complete. 2024-01-11T17:57:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.952+0000 7f13afd5fb80 20 client.15180 trim_cache size 0 max 16384 2024-01-11T17:57:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 Processor -- start 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 -- start start 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b23b0 0x55c1359c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b2d30 0x55c1359b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13577b650 con 0x55c1359b1520 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1358a8040 con 0x55c1359b23b0 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1359ac360 con 0x55c1359b2d30 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46252/0 (socket says 172.21.15.7:46252) 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f1399d9b700 1 -- 172.21.15.7:0/3870780792 learned_addr learned my addr 172.21.15.7:0/3870780792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f139959a700 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b23b0 0x55c1359c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.971+0000 7f139a59c700 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b2d30 0x55c1359b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f1399d9b700 1 -- 172.21.15.7:0/3870780792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b2d30 msgr2=0x55c1359b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f1399d9b700 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b2d30 0x55c1359b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f1399d9b700 1 -- 172.21.15.7:0/3870780792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b23b0 msgr2=0x55c1359c2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f1399d9b700 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b23b0 0x55c1359c2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f1399d9b700 1 -- 172.21.15.7:0/3870780792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c135725b10 con 0x55c1359b1520 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f1399d9b700 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359b0ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55c1359c6f60 tx=0x7f1390009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d467697150dcf9ac server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f137f7fe700 1 -- 172.21.15.7:0/3870780792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390017070 con 0x55c1359b1520 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f137f7fe700 1 -- 172.21.15.7:0/3870780792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1390018040 con 0x55c1359b1520 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.972+0000 7f137f7fe700 1 -- 172.21.15.7:0/3870780792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390005280 con 0x55c1359b1520 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3870780792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 msgr2=0x55c1359b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359b0ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55c1359c6f60 tx=0x7f1390009260 comp rx=0 tx=0).stop 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3870780792 shutdown_connections 2024-01-11T17:57:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b2d30 0x55c1359b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359b0ed0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/3870780792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b23b0 0x55c1359c2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3870780792 >> 172.21.15.7:0/3870780792 conn(0x55c135926750 msgr2=0x55c1359ba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3870780792 shutdown_connections 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 -- 172.21.15.7:0/3870780792 wait complete. 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 -- start start 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.973+0000 7f13afd5fb80 1 Processor -- start 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 0x55c1359c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1359c1a80 con 0x55c1359b1520 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1359c1bf0 con 0x55c1359b2d30 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1359c1db0 con 0x55c1359b23b0 2024-01-11T17:57:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f13afd5fb80 10 client.0 _ll_register_callbacks cb 0x55c1358a56d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f139959a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 0x55c1359c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f1399d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46262/0 (socket says 172.21.15.7:46262) 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f139959a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 0x55c1359c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:49778/0 (socket says 172.21.15.7:49778) 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f1399d9b700 1 -- 172.21.15.7:0/2520817997 learned_addr learned my addr 172.21.15.7:0/2520817997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.974+0000 7f139a59c700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f139959a700 1 -- 172.21.15.7:0/2520817997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 msgr2=0x55c1359c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f139959a700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f139959a700 1 -- 172.21.15.7:0/2520817997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 msgr2=0x55c1359a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f139959a700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f139959a700 1 -- 172.21.15.7:0/2520817997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390012040 con 0x55c1359b23b0 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f1399d9b700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f139959a700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 0x55c1359c1000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f138400b2e0 tx=0x7f138409a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b9c0a25348976 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f137d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f137d7fa700 1 -- 172.21.15.7:0/2520817997 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138407add0 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f138409b920 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f137d7fa700 1 -- 172.21.15.7:0/2520817997 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f138409c4e0 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f137d7fa700 1 -- 172.21.15.7:0/2520817997 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f138409bbb0 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f13afd5fb80 10 client.25186 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.975+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c1358c61e0 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f137d7fa700 1 -- 172.21.15.7:0/2520817997 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f138409b920 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f137d7fa700 1 -- 172.21.15.7:0/2520817997 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1384097bd0 con 0x55c1359b23b0 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f13afd5fb80 10 client.25186 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f13afd5fb80 10 client.25186 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f13afd5fb80 20 client.25186 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c135a20860 0x55c135a1cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f13afd5fb80 4 client.25186 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:57:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c1358fd890 con 0x55c135a20860 2024-01-11T17:57:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.976+0000 7f1399d9b700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c135a20860 0x55c135a1cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.977+0000 7f1399d9b700 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c135a20860 0x55c135a1cdb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9de24c8670955fdf server_cookie=5807cf38f14d725f in_seq=0 out_seq=0 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.977+0000 7f137d7fa700 10 client.25186 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.977+0000 7f137d7fa700 1 -- 172.21.15.7:0/2520817997 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55c1358fd890 con 0x55c135a20860 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.977+0000 7f137d7fa700 10 client.25186 handle_command_reply: tid=0 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.977+0000 7f13afd5fb80 1 client.25186 shutdown 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 msgr2=0x55c1359c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 0x55c1359c1000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f138400b2e0 tx=0x7f138409a390 comp rx=0 tx=0).stop 2024-01-11T17:57:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 shutdown_connections 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x55c135a20860 0x55c135a1cdb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1359b2d30 0x55c1359c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1359b1520 0x55c1359a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 --2- 172.21.15.7:0/2520817997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c1359b23b0 0x55c1359c1000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f137d7fa700 0 client.25186 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 >> 172.21.15.7:0/2520817997 conn(0x55c135926750 msgr2=0x55c1359b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 shutdown_connections 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 1 -- 172.21.15.7:0/2520817997 wait complete. 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f13afd5fb80 20 client.25186 trim_cache size 0 max 16384 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:57:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24577, 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 54.592008876999998, 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 92.24538948 2024-01-11T17:57:27.984 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f13800a1f70 msgr2=0x7f13800a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.978+0000 7f139bfff700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f13800a1f70 0x7f13800a4430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f138c009ec0 tx=0x7f138c009920 comp rx=0 tx=0).stop 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 msgr2=0x7f1394073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 0x7f1394073a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f13840098e0 tx=0x7f1384005720 comp rx=0 tx=0).stop 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 shutdown_connections 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f13800a1f70 0x7f13800a4430 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13941015b0 0x7f13940753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1394103140 0x7f1394073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 --2- 172.21.15.7:0/2912666259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13941027a0 0x7f1394073a00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 >> 172.21.15.7:0/2912666259 conn(0x7f13940fcb00 msgr2=0x7f1394105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 shutdown_connections 2024-01-11T17:57:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:27.979+0000 7f139bfff700 1 -- 172.21.15.7:0/2912666259 wait complete. 2024-01-11T17:57:27.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:27 smithi007 ceph-mon[33361]: pgmap v288: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-11T17:57:27.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:27 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2183318923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:27.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:27 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2316618393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:27.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:27 smithi007 ceph-mon[33361]: osdmap e88: 12 total, 12 up, 12 in 2024-01-11T17:57:27.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:27 smithi016 ceph-mon[43195]: pgmap v288: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-11T17:57:27.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:27 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2183318923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:27.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:27 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2316618393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:27.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:27 smithi016 ceph-mon[43195]: osdmap e88: 12 total, 12 up, 12 in 2024-01-11T17:57:27.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-11T17:57:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:27 smithi038 ceph-mon[43582]: pgmap v288: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-11T17:57:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:27 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2183318923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:27 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2316618393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:27 smithi038 ceph-mon[43582]: osdmap e88: 12 total, 12 up, 12 in 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 -- 172.21.15.7:0/389743112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec109470 msgr2=0x7fc2ec10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/389743112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec109470 0x7fc2ec10b900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400b0a0 tx=0x7fc2e400b3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 -- 172.21.15.7:0/389743112 shutdown_connections 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/389743112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2ec075650 0x7fc2ec073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/389743112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec109470 0x7fc2ec10b900 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/389743112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ec074070 0x7fc2ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 -- 172.21.15.7:0/389743112 >> 172.21.15.7:0/389743112 conn(0x7fc2ec0fcb00 msgr2=0x7fc2ec0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 -- 172.21.15.7:0/389743112 shutdown_connections 2024-01-11T17:57:28.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.204+0000 7fc2f23ff700 1 -- 172.21.15.7:0/389743112 wait complete. 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.205+0000 7fc2f23ff700 1 Processor -- start 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.205+0000 7fc2f23ff700 1 -- start start 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f23ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 0x7fc2ec199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f23ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2ec075650 0x7fc2ec199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f23ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ec109470 0x7fc2ec1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f23ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec1a12e0 con 0x7fc2ec074070 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f23ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec1a1420 con 0x7fc2ec109470 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f23ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec1a15b0 con 0x7fc2ec075650 2024-01-11T17:57:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 0x7fc2ec199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 0x7fc2ec199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46268/0 (socket says 172.21.15.7:46268) 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1105308711 learned_addr learned my addr 172.21.15.7:0/1105308711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2ec075650 0x7fc2ec199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2f099c700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ec109470 0x7fc2ec1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1105308711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2ec075650 msgr2=0x7fc2ec199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2ec075650 0x7fc2ec199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1105308711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ec109470 msgr2=0x7fc2ec1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ec109470 0x7fc2ec1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.206+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1105308711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc009520 con 0x7fc2ec074070 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.207+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 0x7fc2ec199760 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00eb30 tx=0x7fc2dc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.207+0000 7fc2e97fa700 1 -- 172.21.15.7:0/1105308711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc009880 con 0x7fc2ec074070 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.207+0000 7fc2e97fa700 1 -- 172.21.15.7:0/1105308711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc0044e0 con 0x7fc2ec074070 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.207+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e4009d00 con 0x7fc2ec074070 2024-01-11T17:57:28.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.207+0000 7fc2e97fa700 1 -- 172.21.15.7:0/1105308711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc003e90 con 0x7fc2ec074070 2024-01-11T17:57:28.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.207+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec1a1c60 con 0x7fc2ec074070 2024-01-11T17:57:28.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.210+0000 7fc2e97fa700 1 -- 172.21.15.7:0/1105308711 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc2dc004000 con 0x7fc2ec074070 2024-01-11T17:57:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 Processor -- start 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc2e97fa700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2d40a1ff0 0x7fc2d40a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 -- start start 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc2e97fa700 1 -- 172.21.15.7:0/1105308711 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2dc014070 con 0x7fc2ec074070 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 0x558c762687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76266f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc2f099c700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2d40a1ff0 0x7fc2d40a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c762679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c7615e040 con 0x558c76267520 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c761c1ad0 con 0x558c762683b0 2024-01-11T17:57:28.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc306032b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c761bf3f0 con 0x558c76268d30 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 0x558c762687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.212+0000 7fc2f099c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c762679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 0x558c762687d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35882/0 (socket says 172.21.15.7:35882) 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 -- 172.21.15.7:0/768739643 learned_addr learned my addr 172.21.15.7:0/768739643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76266f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 -- 172.21.15.7:0/768739643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 msgr2=0x558c76266f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76266f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 -- 172.21.15.7:0/768739643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 msgr2=0x558c762679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c762679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 -- 172.21.15.7:0/768739643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c7602f360 con 0x558c762683b0 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2f099c700 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c762679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2f099c700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2d40a1ff0 0x7fc2d40a44b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40060b0 tx=0x7fc2e40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 0x558c762687d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc09aac0 tx=0x7fc2dc09aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d525e5528150ec79 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2d27fc700 1 -- 172.21.15.7:0/768739643 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc09b230 con 0x558c762683b0 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2d27fc700 1 -- 172.21.15.7:0/768739643 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc00cc90 con 0x558c762683b0 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.213+0000 7fc2d27fc700 1 -- 172.21.15.7:0/768739643 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0b3210 con 0x558c762683b0 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 -- 172.21.15.7:0/768739643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 msgr2=0x558c762687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 0x558c762687d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc09aac0 tx=0x7fc2dc09aa00 comp rx=0 tx=0).stop 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 -- 172.21.15.7:0/768739643 shutdown_connections 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c762679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c762683b0 0x558c762687d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 --2- 172.21.15.7:0/768739643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76266f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 -- 172.21.15.7:0/768739643 >> 172.21.15.7:0/768739643 conn(0x558c761dc750 msgr2=0x558c76261d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 -- 172.21.15.7:0/768739643 shutdown_connections 2024-01-11T17:57:28.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.214+0000 7fc306032b80 1 -- 172.21.15.7:0/768739643 wait complete. 2024-01-11T17:57:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.215+0000 7fc306032b80 1 -- start start 2024-01-11T17:57:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.215+0000 7fc306032b80 1 Processor -- start 2024-01-11T17:57:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c76270e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c762713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c762783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c761bace0 con 0x558c762683b0 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c76129350 con 0x558c76267520 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c76128ff0 con 0x558c76268d30 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc306032b80 10 client.0 _ll_register_callbacks cb 0x558c7615b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c762713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c762713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46282/0 (socket says 172.21.15.7:46282) 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/165477793 learned_addr learned my addr 172.21.15.7:0/165477793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2f099c700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c762783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c76270e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/165477793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 msgr2=0x558c762783e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c762783e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/165477793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 msgr2=0x558c76270e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c76270e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/165477793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec1a1c60 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2f099c700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c762783e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.216+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c76270e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c762713a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0011730 tx=0x7fc2e0011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d0888626245daf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc2cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/165477793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0016070 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c76259fc0 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/165477793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2e0004320 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/165477793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e00044e0 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc306032b80 10 client.15198 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558c75fdbb10 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.217+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/165477793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c76259fc0 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/165477793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2e000f800 con 0x558c762683b0 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc306032b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc306032b80 10 client.15198 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc306032b80 20 client.15198 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc306032b80 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c7627cad0 0x558c7627b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc306032b80 4 client.15198 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c761b3b70 con 0x558c7627cad0 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c7627cad0 0x558c7627b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.219+0000 7fc2cb7fe700 10 client.15198 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.218+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c7627cad0 0x558c7627b580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eba06398af579769 server_cookie=8d8abcbbe8b11f13 in_seq=0 out_seq=0 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.219+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/165477793 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c761b3b70 con 0x558c7627cad0 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.219+0000 7fc2cb7fe700 10 client.15198 handle_command_reply: tid=0 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.219+0000 7fc306032b80 1 client.15198 shutdown 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 msgr2=0x558c762713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c762713a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0011730 tx=0x7fc2e0011a40 comp rx=0 tx=0).stop 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 shutdown_connections 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c7627cad0 0x558c7627b580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c762783e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc2cb7fe700 0 client.15198 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c76270e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 --2- 172.21.15.7:0/165477793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c762713a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 >> 172.21.15.7:0/165477793 conn(0x558c761dc750 msgr2=0x558c76265e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 shutdown_connections 2024-01-11T17:57:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 1 -- 172.21.15.7:0/165477793 wait complete. 2024-01-11T17:57:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.220+0000 7fc306032b80 20 client.15198 trim_cache size 0 max 16384 2024-01-11T17:57:28.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 Processor -- start 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 -- start start 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c76266f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c762683b0 0x558c762783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76268d30 0x558c7626edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c76031650 con 0x558c76267520 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c7625e810 con 0x558c76268d30 2024-01-11T17:57:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.239+0000 7fc306032b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c761b3b70 con 0x558c762683b0 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c76266f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c76266f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46288/0 (socket says 172.21.15.7:46288) 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1159286403 learned_addr learned my addr 172.21.15.7:0/1159286403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2f099c700 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76268d30 0x558c7626edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c762683b0 0x558c762783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1159286403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c762683b0 msgr2=0x558c762783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c762683b0 0x558c762783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1159286403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76268d30 msgr2=0x558c7626edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76268d30 0x558c7626edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 -- 172.21.15.7:0/1159286403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c7617c1e0 con 0x558c76267520 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c762683b0 0x558c762783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c76266f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0975a0 tx=0x7fc2dc003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6e137981ca9479 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2d1ffb700 1 -- 172.21.15.7:0/1159286403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc09abb0 con 0x558c76267520 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2d1ffb700 1 -- 172.21.15.7:0/1159286403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc0a2ec0 con 0x558c76267520 2024-01-11T17:57:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.240+0000 7fc2d1ffb700 1 -- 172.21.15.7:0/1159286403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc09ad90 con 0x558c76267520 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 -- 172.21.15.7:0/1159286403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 msgr2=0x558c76266f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c76266f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0975a0 tx=0x7fc2dc003730 comp rx=0 tx=0).stop 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 -- 172.21.15.7:0/1159286403 shutdown_connections 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76268d30 0x558c7626edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c76267520 0x558c76266f00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 --2- 172.21.15.7:0/1159286403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c762683b0 0x558c762783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 -- 172.21.15.7:0/1159286403 >> 172.21.15.7:0/1159286403 conn(0x558c761dc750 msgr2=0x558c76261ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 -- 172.21.15.7:0/1159286403 shutdown_connections 2024-01-11T17:57:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.241+0000 7fc306032b80 1 -- 172.21.15.7:0/1159286403 wait complete. 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.242+0000 7fc306032b80 1 -- start start 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.242+0000 7fc306032b80 1 Processor -- start 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c7625b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c7625b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76277340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c7625bcb0 con 0x558c762683b0 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c7625be20 con 0x558c76267520 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c7625bfe0 con 0x558c76268d30 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc306032b80 10 client.0 _ll_register_callbacks cb 0x558c76278fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c7625b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c7625b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46292/0 (socket says 172.21.15.7:46292) 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/2800946012 learned_addr learned my addr 172.21.15.7:0/2800946012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2f099c700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76277340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c7625b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/2800946012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 msgr2=0x558c76277340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76277340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/2800946012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 msgr2=0x558c7625b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c7625b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.243+0000 7fc2eb7fe700 1 -- 172.21.15.7:0/2800946012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c7602f360 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc2eb7fe700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c7625b770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x558c76259d40 tx=0x7fc2e0005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3472d1deea96808f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc2cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/2800946012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0016070 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/2800946012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2e0000e40 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2dc00cc90 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/2800946012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e00055a0 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc306032b80 10 client.15204 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558c76033c80 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.244+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/2800946012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc00cc90 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/2800946012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2e0011760 con 0x558c762683b0 2024-01-11T17:57:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc306032b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc306032b80 10 client.15204 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc306032b80 20 client.15204 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc306032b80 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c762d6c60 0x558c762d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc306032b80 4 client.15204 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558c761b3890 con 0x558c762d6c60 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c762d6c60 0x558c762d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc2ebfff700 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c762d6c60 0x558c762d3100 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f9d82affd689b6d server_cookie=1942515f3ccfbfab in_seq=0 out_seq=0 2024-01-11T17:57:28.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.245+0000 7fc2cb7fe700 10 client.15204 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:28.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc2cb7fe700 1 -- 172.21.15.7:0/2800946012 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558c761b3890 con 0x558c762d6c60 2024-01-11T17:57:28.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc2cb7fe700 10 client.15204 handle_command_reply: tid=0 2024-01-11T17:57:28.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 client.15204 shutdown 2024-01-11T17:57:28.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 msgr2=0x558c7625b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c7625b770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x558c76259d40 tx=0x7fc2e0005b80 comp rx=0 tx=0).stop 2024-01-11T17:57:28.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 shutdown_connections 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x558c762d6c60 0x558c762d3100 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c76268d30 0x558c76277340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c76267520 0x558c7625b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 --2- 172.21.15.7:0/2800946012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c762683b0 0x558c7625b770 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 >> 172.21.15.7:0/2800946012 conn(0x558c761dc750 msgr2=0x558c76261ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc2cb7fe700 0 client.15204 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 shutdown_connections 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 1 -- 172.21.15.7:0/2800946012 wait complete. 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.246+0000 7fc306032b80 20 client.15204 trim_cache size 0 max 16384 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:57:28.250 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24601, 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 55.869481985999997, 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 86.940711887000006 2024-01-11T17:57:28.251 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2d40a1ff0 msgr2=0x7fc2d40a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2d40a1ff0 0x7fc2d40a44b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40060b0 tx=0x7fc2e40095c0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 msgr2=0x7fc2ec199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 0x7fc2ec199760 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00eb30 tx=0x7fc2dc00ee40 comp rx=0 tx=0).stop 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 shutdown_connections 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2d40a1ff0 0x7fc2d40a44b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ec109470 0x7fc2ec1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2ec074070 0x7fc2ec199760 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 --2- 172.21.15.7:0/1105308711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2ec075650 0x7fc2ec199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 >> 172.21.15.7:0/1105308711 conn(0x7fc2ec0fcb00 msgr2=0x7fc2ec105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 shutdown_connections 2024-01-11T17:57:28.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.247+0000 7fc2f23ff700 1 -- 172.21.15.7:0/1105308711 wait complete. 2024-01-11T17:57:28.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-11T17:57:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/362903496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 msgr2=0x7f5aa410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/362903496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 0x7f5aa410b900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b0a0 tx=0x7f5a9c00b3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/362903496 shutdown_connections 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/362903496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 0x7f5aa410b900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/362903496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4075650 0x7f5aa4073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/362903496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa4074070 0x7f5aa40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/362903496 >> 172.21.15.7:0/362903496 conn(0x7f5aa40fcb00 msgr2=0x7f5aa40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.470+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/362903496 shutdown_connections 2024-01-11T17:57:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.471+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/362903496 wait complete. 2024-01-11T17:57:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.471+0000 7f5aaa7ff700 1 Processor -- start 2024-01-11T17:57:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 -- start start 2024-01-11T17:57:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 0x7f5aa4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa4075650 0x7f5aa4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 0x7f5aa41a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a12e0 con 0x7f5aa4074070 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a1420 con 0x7f5aa4075650 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aaa7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a15b0 con 0x7f5aa4109470 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 0x7f5aa4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 0x7f5aa4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46318/0 (socket says 172.21.15.7:46318) 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2572502376 learned_addr learned my addr 172.21.15.7:0/2572502376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa4075650 0x7f5aa4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.472+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 0x7f5aa41a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2572502376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 msgr2=0x7f5aa41a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 0x7f5aa41a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2572502376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa4075650 msgr2=0x7f5aa4199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa4075650 0x7f5aa4199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2572502376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c009d00 con 0x7f5aa4074070 2024-01-11T17:57:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 0x7f5aa4199760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9a0 tx=0x7f5a9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa17fa700 1 -- 172.21.15.7:0/2572502376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94009790 con 0x7f5aa4074070 2024-01-11T17:57:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa17fa700 1 -- 172.21.15.7:0/2572502376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a94010430 con 0x7f5aa4074070 2024-01-11T17:57:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aa17fa700 1 -- 172.21.15.7:0/2572502376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94018680 con 0x7f5aa4074070 2024-01-11T17:57:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa41a1890 con 0x7f5aa4074070 2024-01-11T17:57:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.473+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa41a1d50 con 0x7f5aa4074070 2024-01-11T17:57:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.476+0000 7f5aa17fa700 1 -- 172.21.15.7:0/2572502376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f5a940105a0 con 0x7f5aa4074070 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.477+0000 7f5abe568b80 1 Processor -- start 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 -- start start 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a8d30 0x55b15b6a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b59e040 con 0x55b15b6a8d30 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b601ad0 con 0x55b15b6a7520 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5abe568b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b5ff3f0 con 0x55b15b6a83b0 2024-01-11T17:57:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa17fa700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a8c0a1ff0 0x7f5a8c0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa17fa700 1 -- 172.21.15.7:0/2572502376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a94014070 con 0x7f5aa4074070 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a8d30 0x55b15b6a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa8d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa8d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:49864/0 (socket says 172.21.15.7:49864) 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa8d9c700 1 -- 172.21.15.7:0/2942452431 learned_addr learned my addr 172.21.15.7:0/2942452431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.478+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a8c0a1ff0 0x7f5a8c0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 -- 172.21.15.7:0/2942452431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 msgr2=0x55b15b6a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 -- 172.21.15.7:0/2942452431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a8d30 msgr2=0x55b15b6a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a8d30 0x55b15b6a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 -- 172.21.15.7:0/2942452431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b15b46f360 con 0x55b15b6a83b0 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6a87d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b0a0 tx=0x7f5a9c008dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd21827a11ae796 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a8c0a1ff0 0x7f5a8c0a44b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c01a000 tx=0x7f5a9c026000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.479+0000 7f5a8a7fc700 1 -- 172.21.15.7:0/2942452431 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00e040 con 0x55b15b6a83b0 2024-01-11T17:57:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5a8a7fc700 1 -- 172.21.15.7:0/2942452431 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c00bcb0 con 0x55b15b6a83b0 2024-01-11T17:57:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5a8a7fc700 1 -- 172.21.15.7:0/2942452431 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c020950 con 0x55b15b6a83b0 2024-01-11T17:57:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 -- 172.21.15.7:0/2942452431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 msgr2=0x55b15b6a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6a87d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b0a0 tx=0x7f5a9c008dc0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 -- 172.21.15.7:0/2942452431 shutdown_connections 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6a87d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2942452431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a8d30 0x55b15b6a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 -- 172.21.15.7:0/2942452431 >> 172.21.15.7:0/2942452431 conn(0x55b15b61c750 msgr2=0x55b15b6a1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.480+0000 7f5abe568b80 1 -- 172.21.15.7:0/2942452431 shutdown_connections 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.481+0000 7f5abe568b80 1 -- 172.21.15.7:0/2942452431 wait complete. 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.481+0000 7f5abe568b80 1 -- start start 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.481+0000 7f5abe568b80 1 Processor -- start 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b5face0 con 0x55b15b6a83b0 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b569350 con 0x55b15b6a7520 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b568ff0 con 0x55b15b6a8d30 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5abe568b80 10 client.0 _ll_register_callbacks cb 0x55b15b59b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5aa37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35934/0 (socket says 172.21.15.7:35934) 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.482+0000 7f5aa3fff700 1 -- 172.21.15.7:0/4056774663 learned_addr learned my addr 172.21.15.7:0/4056774663 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa3fff700 1 -- 172.21.15.7:0/4056774663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 msgr2=0x55b15b6b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa3fff700 1 -- 172.21.15.7:0/4056774663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 msgr2=0x55b15b6b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa3fff700 1 -- 172.21.15.7:0/4056774663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a980098e0 con 0x55b15b6a7520 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b0e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a9408afd0 tx=0x7f5a9409a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0cd6b56d96b874 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5a837fe700 1 -- 172.21.15.7:0/4056774663 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94089c80 con 0x55b15b6a7520 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5a837fe700 1 -- 172.21.15.7:0/4056774663 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a9409af90 con 0x55b15b6a7520 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.483+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b15b699fc0 con 0x55b15b6a7520 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5a837fe700 1 -- 172.21.15.7:0/4056774663 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9409a890 con 0x55b15b6a7520 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 10 client.25199 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f5aa410c520 con 0x55b15b6a7520 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5a837fe700 1 -- 172.21.15.7:0/4056774663 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a940b2080 con 0x55b15b6a7520 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5a837fe700 1 -- 172.21.15.7:0/4056774663 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a94099d80 con 0x55b15b6a7520 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 10 client.25199 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 10 client.25199 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 20 client.25199 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b6bc9f0 0x55b15b6bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 4 client.25199 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.484+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b15b5f3b70 con 0x55b15b6bc9f0 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.485+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b6bc9f0 0x55b15b6bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.485+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b6bc9f0 0x55b15b6bb4d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b419c78b409e60f server_cookie=9ef34330edb5edc0 in_seq=0 out_seq=0 2024-01-11T17:57:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.485+0000 7f5a837fe700 10 client.25199 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:28.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5a837fe700 1 -- 172.21.15.7:0/4056774663 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b15b5f3b70 con 0x55b15b6bc9f0 2024-01-11T17:57:28.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5a837fe700 10 client.25199 handle_command_reply: tid=0 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 client.25199 shutdown 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 msgr2=0x55b15b6b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b0e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a9408afd0 tx=0x7f5a9409a3d0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 shutdown_connections 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b6bc9f0 0x55b15b6bb4d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b0e00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 --2- 172.21.15.7:0/4056774663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5a837fe700 0 client.25199 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 >> 172.21.15.7:0/4056774663 conn(0x55b15b61c750 msgr2=0x55b15b6a5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 shutdown_connections 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 1 -- 172.21.15.7:0/4056774663 wait complete. 2024-01-11T17:57:28.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.486+0000 7f5abe568b80 20 client.25199 trim_cache size 0 max 16384 2024-01-11T17:57:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.505+0000 7f5abe568b80 1 Processor -- start 2024-01-11T17:57:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.505+0000 7f5abe568b80 1 -- start start 2024-01-11T17:57:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.505+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 0x55b15b6aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.505+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a8d30 0x55b15b6b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5abe568b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b471650 con 0x55b15b6a7520 2024-01-11T17:57:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5abe568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b5f3ba0 con 0x55b15b6a8d30 2024-01-11T17:57:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5abe568b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b5f6dd0 con 0x55b15b6a83b0 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 0x55b15b6aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 0x55b15b6aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46340/0 (socket says 172.21.15.7:46340) 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2762396384 learned_addr learned my addr 172.21.15.7:0/2762396384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a8d30 0x55b15b6b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2762396384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 msgr2=0x55b15b6b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6b8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2762396384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a8d30 msgr2=0x55b15b6b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a8d30 0x55b15b6b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 -- 172.21.15.7:0/2762396384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b15b41bb10 con 0x55b15b6a7520 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a8d30 0x55b15b6b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.506+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 0x55b15b6aee30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e970 tx=0x7f5a94098e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f181b779308234d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5a89ffb700 1 -- 172.21.15.7:0/2762396384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940cb580 con 0x55b15b6a7520 2024-01-11T17:57:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5a89ffb700 1 -- 172.21.15.7:0/2762396384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b15b699fc0 con 0x55b15b6a7520 2024-01-11T17:57:28.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5a89ffb700 1 -- 172.21.15.7:0/2762396384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9409ea70 con 0x55b15b6a7520 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 -- 172.21.15.7:0/2762396384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 msgr2=0x55b15b6aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 0x55b15b6aee30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e970 tx=0x7f5a94098e40 comp rx=0 tx=0).stop 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 -- 172.21.15.7:0/2762396384 shutdown_connections 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a8d30 0x55b15b6b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a7520 0x55b15b6aee30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 --2- 172.21.15.7:0/2762396384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a83b0 0x55b15b6b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 -- 172.21.15.7:0/2762396384 >> 172.21.15.7:0/2762396384 conn(0x55b15b61c750 msgr2=0x55b15b6a1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 -- 172.21.15.7:0/2762396384 shutdown_connections 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.507+0000 7f5abe568b80 1 -- 172.21.15.7:0/2762396384 wait complete. 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 -- start start 2024-01-11T17:57:28.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 Processor -- start 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b6b7d50 con 0x55b15b6a83b0 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b6b7ec0 con 0x55b15b6a7520 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15b6b8080 con 0x55b15b6a8d30 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.508+0000 7f5abe568b80 10 client.0 _ll_register_callbacks cb 0x55b15b59b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46350/0 (socket says 172.21.15.7:46350) 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 -- 172.21.15.7:0/602597156 learned_addr learned my addr 172.21.15.7:0/602597156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa8d9c700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 -- 172.21.15.7:0/602597156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 msgr2=0x55b15b6b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 -- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 msgr2=0x55b15b6b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 -- 172.21.15.7:0/602597156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a980040f0 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5aa37fe700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b4d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55b15b6bd980 tx=0x7f5a98003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712ec41ba5e1857 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.509+0000 7f5a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5a837fe700 1 -- 172.21.15.7:0/602597156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98016070 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a9c00bcb0 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5a837fe700 1 -- 172.21.15.7:0/602597156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a98003e40 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5a837fe700 1 -- 172.21.15.7:0/602597156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9801e810 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5abe568b80 10 client.15222 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f5aa410c520 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5a837fe700 1 -- 172.21.15.7:0/602597156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c00bcb0 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5a837fe700 1 -- 172.21.15.7:0/602597156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a98024020 con 0x55b15b6a83b0 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.510+0000 7f5abe568b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5abe568b80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5abe568b80 20 client.15222 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5abe568b80 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b7175e0 0x55b15b713a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5abe568b80 4 client.15222 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b15b5f3890 con 0x55b15b7175e0 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b7175e0 0x55b15b713a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5aa3fff700 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b7175e0 0x55b15b713a60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627f8d873e4cab4d server_cookie=1ffd46565a34faf8 in_seq=0 out_seq=0 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.511+0000 7f5a837fe700 10 client.15222 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5a837fe700 1 -- 172.21.15.7:0/602597156 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b15b5f3890 con 0x55b15b7175e0 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5a837fe700 10 client.15222 handle_command_reply: tid=0 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 client.15222 shutdown 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 msgr2=0x55b15b6b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b4d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55b15b6bd980 tx=0x7f5a98003960 comp rx=0 tx=0).stop 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 shutdown_connections 2024-01-11T17:57:28.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15b7175e0 0x55b15b713a60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15b6a8d30 0x55b15b6b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15b6a7520 0x55b15b6b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 --2- 172.21.15.7:0/602597156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15b6a83b0 0x55b15b6b4d20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5a837fe700 0 client.15222 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 >> 172.21.15.7:0/602597156 conn(0x55b15b61c750 msgr2=0x55b15b6b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 shutdown_connections 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 1 -- 172.21.15.7:0/602597156 wait complete. 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.512+0000 7f5abe568b80 20 client.15222 trim_cache size 0 max 16384 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:57:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24602, 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 79.276006393000003, 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 85.248540020999997 2024-01-11T17:57:28.518 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a8c0a1ff0 msgr2=0x7f5a8c0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a8c0a1ff0 0x7f5a8c0a44b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c01a000 tx=0x7f5a9c026000 comp rx=0 tx=0).stop 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 msgr2=0x7f5aa4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 0x7f5aa4199760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9a0 tx=0x7f5a9400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 shutdown_connections 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a8c0a1ff0 0x7f5a8c0a44b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aa4109470 0x7f5aa41a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa4074070 0x7f5aa4199760 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 --2- 172.21.15.7:0/2572502376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa4075650 0x7f5aa4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 >> 172.21.15.7:0/2572502376 conn(0x7f5aa40fcb00 msgr2=0x7f5aa4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 shutdown_connections 2024-01-11T17:57:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:28.513+0000 7f5aaa7ff700 1 -- 172.21.15.7:0/2572502376 wait complete. 2024-01-11T17:57:28.532 INFO:tasks.cephadm:Setup complete, yielding 2024-01-11T17:57:28.532 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-01-11T17:57:28.541 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:57:28.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph orch status' 2024-01-11T17:57:28.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:28 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/204875725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:28.972 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:28.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:28 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/204875725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:29.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:28 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/204875725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:29.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:29 smithi007 ceph-mon[33361]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:29.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:29 smithi016 ceph-mon[43195]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:30.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:29 smithi038 ceph-mon[43582]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 -- 172.21.15.7:0/3508856169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901020d0 msgr2=0x7fa5901044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3508856169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901020d0 0x7fa5901044c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa584009990 tx=0x7fa584009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 -- 172.21.15.7:0/3508856169 shutdown_connections 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3508856169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa590107330 0x7fa5901097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3508856169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5901020d0 0x7fa5901044c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3508856169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa590104a00 0x7fa590106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 -- 172.21.15.7:0/3508856169 >> 172.21.15.7:0/3508856169 conn(0x7fa5900fbae0 msgr2=0x7fa5900fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 -- 172.21.15.7:0/3508856169 shutdown_connections 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.185+0000 7fa596e5d700 1 -- 172.21.15.7:0/3508856169 wait complete. 2024-01-11T17:57:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.187+0000 7fa596e5d700 1 Processor -- start 2024-01-11T17:57:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.187+0000 7fa596e5d700 1 -- start start 2024-01-11T17:57:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.187+0000 7fa596e5d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 0x7fa590101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa596e5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa590104a00 0x7fa590100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa596e5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590107330 0x7fa5901005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa596e5d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590100d40 con 0x7fa590107330 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa596e5d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590100eb0 con 0x7fa590104a00 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa596e5d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901a15f0 con 0x7fa5901020d0 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa594bf9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 0x7fa590101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa5953fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590107330 0x7fa5901005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa594bf9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 0x7fa590101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:49902/0 (socket says 172.21.15.7:49902) 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa594bf9700 1 -- 172.21.15.7:0/3868941804 learned_addr learned my addr 172.21.15.7:0/3868941804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa58ffff700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa590104a00 0x7fa590100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa5953fa700 1 -- 172.21.15.7:0/3868941804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 msgr2=0x7fa590101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa5953fa700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 0x7fa590101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa5953fa700 1 -- 172.21.15.7:0/3868941804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa590104a00 msgr2=0x7fa590100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa5953fa700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa590104a00 0x7fa590100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa5953fa700 1 -- 172.21.15.7:0/3868941804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840095f0 con 0x7fa590107330 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.188+0000 7fa594bf9700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 0x7fa590101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.189+0000 7fa5953fa700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590107330 0x7fa5901005c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa58800ef90 tx=0x7fa58800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.189+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588018070 con 0x7fa590107330 2024-01-11T17:57:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.189+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58800f040 con 0x7fa590107330 2024-01-11T17:57:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.189+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800cc30 con 0x7fa590107330 2024-01-11T17:57:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.189+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5901a1870 con 0x7fa590107330 2024-01-11T17:57:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.189+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5901a1d20 con 0x7fa590107330 2024-01-11T17:57:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.190+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5900fd660 con 0x7fa590107330 2024-01-11T17:57:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.192+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fa588003a00 con 0x7fa590107330 2024-01-11T17:57:30.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.194+0000 7fa58dffb700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5800a1ff0 0x7fa5800a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:30.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.194+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa5880cd9e0 con 0x7fa590107330 2024-01-11T17:57:30.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.195+0000 7fa594bf9700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5800a1ff0 0x7fa5800a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:30.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.195+0000 7fa594bf9700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5800a1ff0 0x7fa5800a44b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b2e0 tx=0x7fa58400b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.199+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa588096620 con 0x7fa590107330 2024-01-11T17:57:30.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.426+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa59002d3c0 con 0x7fa5800a1ff0 2024-01-11T17:57:30.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.427+0000 7fa58dffb700 1 -- 172.21.15.7:0/3868941804 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa59002d3c0 con 0x7fa5800a1ff0 2024-01-11T17:57:30.427 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-01-11T17:57:30.428 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-01-11T17:57:30.428 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5800a1ff0 msgr2=0x7fa5800a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5800a1ff0 0x7fa5800a44b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b2e0 tx=0x7fa58400b770 comp rx=0 tx=0).stop 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590107330 msgr2=0x7fa5901005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590107330 0x7fa5901005c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa58800ef90 tx=0x7fa58800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 shutdown_connections 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5800a1ff0 0x7fa5800a44b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5901020d0 0x7fa590101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa590107330 0x7fa5901005c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 --2- 172.21.15.7:0/3868941804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa590104a00 0x7fa590100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 >> 172.21.15.7:0/3868941804 conn(0x7fa5900fbae0 msgr2=0x7fa590102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.429+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 shutdown_connections 2024-01-11T17:57:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:30.430+0000 7fa596e5d700 1 -- 172.21.15.7:0/3868941804 wait complete. 2024-01-11T17:57:30.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph orch ps' 2024-01-11T17:57:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:31.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:31 smithi007 ceph-mon[33361]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:31.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:31 smithi007 ceph-mon[33361]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-01-11T17:57:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:31 smithi016 ceph-mon[43195]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:31 smithi016 ceph-mon[43195]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-01-11T17:57:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:31 smithi038 ceph-mon[43582]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:31 smithi038 ceph-mon[43582]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-01-11T17:57:32.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 -- 172.21.15.7:0/2414182971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914104a00 msgr2=0x7f3914106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 --2- 172.21.15.7:0/2414182971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914104a00 0x7f3914106df0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3904009990 tx=0x7f3904009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 -- 172.21.15.7:0/2414182971 shutdown_connections 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 --2- 172.21.15.7:0/2414182971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3914107330 0x7f39141097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 --2- 172.21.15.7:0/2414182971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39141020d0 0x7f39141044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 --2- 172.21.15.7:0/2414182971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914104a00 0x7f3914106df0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 -- 172.21.15.7:0/2414182971 >> 172.21.15.7:0/2414182971 conn(0x7f39140fbae0 msgr2=0x7f39140fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 -- 172.21.15.7:0/2414182971 shutdown_connections 2024-01-11T17:57:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.434+0000 7f391b283700 1 -- 172.21.15.7:0/2414182971 wait complete. 2024-01-11T17:57:32.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.435+0000 7f391b283700 1 Processor -- start 2024-01-11T17:57:32.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 -- start start 2024-01-11T17:57:32.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39141020d0 0x7f3914197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 0x7f3914197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914107330 0x7f391419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419f110 con 0x7f3914107330 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419f250 con 0x7f3914104a00 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391b283700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391419f3e0 con 0x7f39141020d0 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391881e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 0x7f3914197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391901f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39141020d0 0x7f3914197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391881e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 0x7f3914197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50182/0 (socket says 172.21.15.7:50182) 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f391881e700 1 -- 172.21.15.7:0/3154377168 learned_addr learned my addr 172.21.15.7:0/3154377168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.436+0000 7f3919820700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914107330 0x7f391419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391881e700 1 -- 172.21.15.7:0/3154377168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39141020d0 msgr2=0x7f3914197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391881e700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39141020d0 0x7f3914197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391881e700 1 -- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914107330 msgr2=0x7f391419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391881e700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914107330 0x7f391419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391881e700 1 -- 172.21.15.7:0/3154377168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39040095f0 con 0x7f3914104a00 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391901f700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39141020d0 0x7f3914197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391881e700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 0x7f3914197ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3904000c00 tx=0x7f3904009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390401d070 con 0x7f3914104a00 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3904021410 con 0x7f3914104a00 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391419f660 con 0x7f3914104a00 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.437+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904017650 con 0x7f3914104a00 2024-01-11T17:57:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.438+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391419fa70 con 0x7f3914104a00 2024-01-11T17:57:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.439+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140fd660 con 0x7f3914104a00 2024-01-11T17:57:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.441+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f3904021580 con 0x7f3914104a00 2024-01-11T17:57:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.442+0000 7f390a7fc700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39000a1f70 0x7f39000a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.442+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f39040cf8e0 con 0x7f3914104a00 2024-01-11T17:57:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.442+0000 7f391901f700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39000a1f70 0x7f39000a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.443+0000 7f391901f700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39000a1f70 0x7f39000a4430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3910010420 tx=0x7f39100073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.445+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3904098580 con 0x7f3914104a00 2024-01-11T17:57:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.664+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f391402d500 con 0x7f39000a1f70 2024-01-11T17:57:32.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.678+0000 7f390a7fc700 1 -- 172.21.15.7:0/3154377168 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f391402d500 con 0x7f39000a1f70 2024-01-11T17:57:32.679 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-01-11T17:57:32.679 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (91s) 17s ago 91s 17.9M - 19.0.0-446-gc27531ad a1bd98dcc13b fea8f7d6a088 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi016 running (89s) 22s ago 89s 30.2M - 19.0.0-446-gc27531ad a1bd98dcc13b 0468188a2c19 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi038 running (101s) 22s ago 100s 20.2M - 19.0.0-446-gc27531ad a1bd98dcc13b 44b513081536 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (107s) 17s ago 106s 13.9M - 19.0.0-446-gc27531ad a1bd98dcc13b 97079eb5cee6 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi016 running (105s) 22s ago 104s 17.6M - 19.0.0-446-gc27531ad a1bd98dcc13b 0c4e69b9f9b8 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi038 running (103s) 22s ago 102s 16.8M - 19.0.0-446-gc27531ad a1bd98dcc13b 69a1c96d2a89 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (94s) 17s ago 94s 13.8M - 19.0.0-446-gc27531ad a1bd98dcc13b e104acea60b0 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi016 running (99s) 22s ago 98s 18.7M - 19.0.0-446-gc27531ad a1bd98dcc13b 13df28825d8e 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi038 running (97s) 22s ago 96s 21.1M - 19.0.0-446-gc27531ad a1bd98dcc13b dbd488d7373e 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (109s) 17s ago 109s 13.9M - 19.0.0-446-gc27531ad a1bd98dcc13b 7a159006276d 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi016 running (112s) 22s ago 112s 15.0M - 19.0.0-446-gc27531ad a1bd98dcc13b 3d25785caeeb 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi038 running (114s) 22s ago 113s 16.6M - 19.0.0-446-gc27531ad a1bd98dcc13b 908f17be23f4 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (8m) 17s ago 8m 530M - 19.0.0-446-gc27531ad a1bd98dcc13b ee005514df34 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi016 *:8443,8765 running (6m) 22s ago 6m 448M - 19.0.0-446-gc27531ad a1bd98dcc13b 35ddcb941cb8 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi038 *:8443,8765 running (6m) 22s ago 6m 451M - 19.0.0-446-gc27531ad a1bd98dcc13b 673ca8eaafcc 2024-01-11T17:57:32.680 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (8m) 17s ago 8m 92.3M 2048M 19.0.0-446-gc27531ad a1bd98dcc13b a03a168ceed1 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi016 running (6m) 22s ago 6m 62.9M 2048M 19.0.0-446-gc27531ad a1bd98dcc13b ca083f7c5ed7 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi038 running (6m) 22s ago 6m 62.5M 2048M 19.0.0-446-gc27531ad a1bd98dcc13b 4de4cd043e18 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 17s ago 5m 70.3M 4319M 19.0.0-446-gc27531ad a1bd98dcc13b 92210e86fd3e 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi016 running (5m) 22s ago 5m 149M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b 609b0cb34a7a 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi038 running (5m) 22s ago 5m 144M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b 735b61a8f2b3 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (4m) 17s ago 4m 139M 4319M 19.0.0-446-gc27531ad a1bd98dcc13b e7f3f8962815 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi016 running (4m) 22s ago 4m 137M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b c2d4e4fe80dd 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi038 running (4m) 22s ago 4m 138M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b 3bd0bc7c700d 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (3m) 17s ago 3m 132M 4319M 19.0.0-446-gc27531ad a1bd98dcc13b 72347846feb0 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi016 running (3m) 22s ago 3m 66.9M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b 8d8611d9f2a3 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi038 running (3m) 22s ago 3m 61.7M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b 043a6eedf046 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (2m) 17s ago 2m 59.5M 4319M 19.0.0-446-gc27531ad a1bd98dcc13b 70c974059a64 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi016 running (2m) 22s ago 2m 139M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b 40f410b11fc1 2024-01-11T17:57:32.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi038 running (2m) 22s ago 2m 125M 4279M 19.0.0-446-gc27531ad a1bd98dcc13b d1e6eee43bed 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39000a1f70 msgr2=0x7f39000a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39000a1f70 0x7f39000a4430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3910010420 tx=0x7f39100073d0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 msgr2=0x7f3914197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 0x7f3914197ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3904000c00 tx=0x7f3904009580 comp rx=0 tx=0).stop 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 shutdown_connections 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39000a1f70 0x7f39000a4430 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39141020d0 0x7f3914197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3914107330 0x7f391419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 --2- 172.21.15.7:0/3154377168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3914104a00 0x7f3914197ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 >> 172.21.15.7:0/3154377168 conn(0x7f39140fbae0 msgr2=0x7f3914102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 shutdown_connections 2024-01-11T17:57:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:32.681+0000 7f391b283700 1 -- 172.21.15.7:0/3154377168 wait complete. 2024-01-11T17:57:33.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph orch ls' 2024-01-11T17:57:33.496 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:33.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:33 smithi007 ceph-mon[33361]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-01-11T17:57:33.970 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:33 smithi007 ceph-mon[33361]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:33 smithi016 ceph-mon[43195]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-01-11T17:57:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:33 smithi016 ceph-mon[43195]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:33 smithi038 ceph-mon[43582]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-01-11T17:57:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:33 smithi038 ceph-mon[43582]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 -- 172.21.15.7:0/877201751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314102850 msgr2=0x7f9314102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 --2- 172.21.15.7:0/877201751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314102850 0x7f9314102cb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93080099c0 tx=0x7f9308009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 -- 172.21.15.7:0/877201751 shutdown_connections 2024-01-11T17:57:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 --2- 172.21.15.7:0/877201751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9314101650 0x7f9314101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 --2- 172.21.15.7:0/877201751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93141031f0 0x7f9314109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 --2- 172.21.15.7:0/877201751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9314102850 0x7f9314102cb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 -- 172.21.15.7:0/877201751 >> 172.21.15.7:0/877201751 conn(0x7f93140fcbb0 msgr2=0x7f93140ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 -- 172.21.15.7:0/877201751 shutdown_connections 2024-01-11T17:57:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.675+0000 7f93191ca700 1 -- 172.21.15.7:0/877201751 wait complete. 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.676+0000 7f93191ca700 1 Processor -- start 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.676+0000 7f93191ca700 1 -- start start 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f93191ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314101650 0x7f9314197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f93191ca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9314102850 0x7f9314197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f93191ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 0x7f931419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f93191ca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419f0a0 con 0x7f93141031f0 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f93191ca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419f1e0 con 0x7f9314101650 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f93191ca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419f370 con 0x7f9314102850 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 0x7f931419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 0x7f931419ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48638/0 (socket says 172.21.15.7:48638) 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 -- 172.21.15.7:0/4014830988 learned_addr learned my addr 172.21.15.7:0/4014830988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931259c700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9314102850 0x7f9314197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f9312d9d700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314101650 0x7f9314197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 -- 172.21.15.7:0/4014830988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9314102850 msgr2=0x7f9314197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9314102850 0x7f9314197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 -- 172.21.15.7:0/4014830988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314101650 msgr2=0x7f9314197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314101650 0x7f9314197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.677+0000 7f931359e700 1 -- 172.21.15.7:0/4014830988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080095f0 con 0x7f93141031f0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.678+0000 7f931359e700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 0x7f931419ead0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f930c00ef60 tx=0x7f930c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.678+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c018070 con 0x7f93141031f0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.678+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f930c00f040 con 0x7f93141031f0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.678+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931419f5f0 con 0x7f93141031f0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.678+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00ccc0 con 0x7f93141031f0 2024-01-11T17:57:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.678+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931419faa0 con 0x7f93141031f0 2024-01-11T17:57:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.679+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9314108a90 con 0x7f93141031f0 2024-01-11T17:57:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.681+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f930c003a00 con 0x7f93141031f0 2024-01-11T17:57:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.682+0000 7f92fbfff700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f92fc0a1f70 0x7f92fc0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.683+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f930c0cd980 con 0x7f93141031f0 2024-01-11T17:57:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.683+0000 7f9312d9d700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f92fc0a1f70 0x7f92fc0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.683+0000 7f9312d9d700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f92fc0a1f70 0x7f92fc0a4430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f93141026b0 tx=0x7f9304009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.687+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f930c0965c0 con 0x7f93141031f0 2024-01-11T17:57:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.906+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f931402d460 con 0x7f92fc0a1f70 2024-01-11T17:57:34.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.913+0000 7f92fbfff700 1 -- 172.21.15.7:0/4014830988 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f931402d460 con 0x7f92fc0a1f70 2024-01-11T17:57:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-01-11T17:57:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 24s ago 118s smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi038=c;smithi038=f;smithi038=i;smithi038=l;count:12 2024-01-11T17:57:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 24s ago 6m smithi007=x;smithi016=y;smithi038=z;count:3 2024-01-11T17:57:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 24s ago 6m smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi038:172.21.15.38=c;count:3 2024-01-11T17:57:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 24s ago - 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f92fc0a1f70 msgr2=0x7f92fc0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f92fc0a1f70 0x7f92fc0a4430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f93141026b0 tx=0x7f9304009260 comp rx=0 tx=0).stop 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 msgr2=0x7f931419ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 0x7f931419ead0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f930c00ef60 tx=0x7f930c00ef90 comp rx=0 tx=0).stop 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 shutdown_connections 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f92fc0a1f70 0x7f92fc0a4430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141031f0 0x7f931419ead0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314101650 0x7f9314197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 --2- 172.21.15.7:0/4014830988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9314102850 0x7f9314197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:34.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 >> 172.21.15.7:0/4014830988 conn(0x7f93140fcbb0 msgr2=0x7f9314105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:34.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.915+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 shutdown_connections 2024-01-11T17:57:34.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:34.916+0000 7f93191ca700 1 -- 172.21.15.7:0/4014830988 wait complete. 2024-01-11T17:57:35.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph orch host ls' 2024-01-11T17:57:35.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:35 smithi007 ceph-mon[33361]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s 2024-01-11T17:57:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:35 smithi016 ceph-mon[43195]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s 2024-01-11T17:57:35.748 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:35.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:35 smithi038 ceph-mon[43582]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s 2024-01-11T17:57:36.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:36 smithi007 ceph-mon[33361]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:36.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:36 smithi016 ceph-mon[43195]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:36.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:36 smithi038 ceph-mon[43582]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 -- 172.21.15.7:0/2845956477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714102850 msgr2=0x7fc714102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/2845956477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714102850 0x7fc714102cb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc7040099c0 tx=0x7fc704009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 -- 172.21.15.7:0/2845956477 shutdown_connections 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/2845956477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7141031f0 0x7fc714109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/2845956477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc714101650 0x7fc714101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/2845956477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714102850 0x7fc714102cb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 -- 172.21.15.7:0/2845956477 >> 172.21.15.7:0/2845956477 conn(0x7fc7140fcbb0 msgr2=0x7fc7140ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 -- 172.21.15.7:0/2845956477 shutdown_connections 2024-01-11T17:57:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.912+0000 7fc71b7fc700 1 -- 172.21.15.7:0/2845956477 wait complete. 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.913+0000 7fc71b7fc700 1 Processor -- start 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.913+0000 7fc71b7fc700 1 -- start start 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc71b7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 0x7fc7140753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc71b7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc714102850 0x7fc714073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc71b7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7141031f0 0x7fc714073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc71b7fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140746c0 con 0x7fc714101650 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc71b7fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714074830 con 0x7fc7141031f0 2024-01-11T17:57:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc71b7fc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141a15b0 con 0x7fc714102850 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 0x7fc7140753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc718d97700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc714102850 0x7fc714073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 0x7fc7140753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48652/0 (socket says 172.21.15.7:48652) 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 -- 172.21.15.7:0/955561785 learned_addr learned my addr 172.21.15.7:0/955561785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719d99700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7141031f0 0x7fc714073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 -- 172.21.15.7:0/955561785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc714102850 msgr2=0x7fc714073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc714102850 0x7fc714073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 -- 172.21.15.7:0/955561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7141031f0 msgr2=0x7fc714073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7141031f0 0x7fc714073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.914+0000 7fc719598700 1 -- 172.21.15.7:0/955561785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040095f0 con 0x7fc714101650 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.915+0000 7fc719598700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 0x7fc7140753b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc71000e970 tx=0x7fc71000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.915+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710009790 con 0x7fc714101650 2024-01-11T17:57:36.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.915+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc71000c7e0 con 0x7fc714101650 2024-01-11T17:57:36.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.915+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710003c70 con 0x7fc714101650 2024-01-11T17:57:36.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.915+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7141a1890 con 0x7fc714101650 2024-01-11T17:57:36.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.915+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141a1d40 con 0x7fc714101650 2024-01-11T17:57:36.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.916+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc714108ae0 con 0x7fc714101650 2024-01-11T17:57:36.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.918+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc71000cc50 con 0x7fc714101650 2024-01-11T17:57:36.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.920+0000 7fc70a7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7000a1ff0 0x7fc7000a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:36.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.920+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc710014070 con 0x7fc714101650 2024-01-11T17:57:36.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.921+0000 7fc718d97700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7000a1ff0 0x7fc7000a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:36.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.921+0000 7fc718d97700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7000a1ff0 0x7fc7000a44b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc70400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:36.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:36.924+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc710096bc0 con 0x7fc714101650 2024-01-11T17:57:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.142+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc71402d4d0 con 0x7fc7000a1ff0 2024-01-11T17:57:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.143+0000 7fc70a7fc700 1 -- 172.21.15.7:0/955561785 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7fc71402d4d0 con 0x7fc7000a1ff0 2024-01-11T17:57:37.144 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-01-11T17:57:37.144 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-01-11T17:57:37.144 INFO:teuthology.orchestra.run.smithi007.stdout:smithi016 172.21.15.16 2024-01-11T17:57:37.144 INFO:teuthology.orchestra.run.smithi007.stdout:smithi038 172.21.15.38 2024-01-11T17:57:37.144 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7000a1ff0 msgr2=0x7fc7000a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7000a1ff0 0x7fc7000a44b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc70400b540 comp rx=0 tx=0).stop 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 msgr2=0x7fc7140753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 0x7fc7140753b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc71000e970 tx=0x7fc71000ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 shutdown_connections 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7000a1ff0 0x7fc7000a44b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7141031f0 0x7fc714073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714101650 0x7fc7140753b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 --2- 172.21.15.7:0/955561785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc714102850 0x7fc714073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 >> 172.21.15.7:0/955561785 conn(0x7fc7140fcbb0 msgr2=0x7fc714105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 shutdown_connections 2024-01-11T17:57:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:37.146+0000 7fc71b7fc700 1 -- 172.21.15.7:0/955561785 wait complete. 2024-01-11T17:57:37.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:37 smithi007 ceph-mon[33361]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-11T17:57:37.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph orch device ls' 2024-01-11T17:57:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:37 smithi016 ceph-mon[43195]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-11T17:57:37.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:37 smithi038 ceph-mon[43582]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-11T17:57:37.969 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:38.452 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:38 smithi007 ceph-mon[33361]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:38.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:38 smithi016 ceph-mon[43195]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:38.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:38 smithi038 ceph-mon[43582]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 -- 172.21.15.7:0/2053969557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e0101650 msgr2=0x7fb6e0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 --2- 172.21.15.7:0/2053969557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e0101650 0x7fb6e0101a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009990 tx=0x7fb6dc009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 -- 172.21.15.7:0/2053969557 shutdown_connections 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 --2- 172.21.15.7:0/2053969557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e0101650 0x7fb6e0101a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 --2- 172.21.15.7:0/2053969557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e01031f0 0x7fb6e0109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 --2- 172.21.15.7:0/2053969557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0102850 0x7fb6e0102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 -- 172.21.15.7:0/2053969557 >> 172.21.15.7:0/2053969557 conn(0x7fb6e00fcbb0 msgr2=0x7fb6e00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 -- 172.21.15.7:0/2053969557 shutdown_connections 2024-01-11T17:57:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.168+0000 7fb6e8553700 1 -- 172.21.15.7:0/2053969557 wait complete. 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.169+0000 7fb6e8553700 1 Processor -- start 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.169+0000 7fb6e8553700 1 -- start start 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e8553700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 0x7fb6e0197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e8553700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e0102850 0x7fb6e0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e8553700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e01031f0 0x7fb6e019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e8553700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019f090 con 0x7fb6e0101650 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e8553700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019f1d0 con 0x7fb6e0102850 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e8553700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019f360 con 0x7fb6e01031f0 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 0x7fb6e0197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 0x7fb6e0197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48674/0 (socket says 172.21.15.7:48674) 2024-01-11T17:57:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 -- 172.21.15.7:0/3760560470 learned_addr learned my addr 172.21.15.7:0/3760560470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e6af0700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e01031f0 0x7fb6e019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e5aee700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e0102850 0x7fb6e0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 -- 172.21.15.7:0/3760560470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e01031f0 msgr2=0x7fb6e019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e01031f0 0x7fb6e019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 -- 172.21.15.7:0/3760560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e0102850 msgr2=0x7fb6e0197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e0102850 0x7fb6e0197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e62ef700 1 -- 172.21.15.7:0/3760560470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0009520 con 0x7fb6e0101650 2024-01-11T17:57:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.170+0000 7fb6e6af0700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e01031f0 0x7fb6e019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.171+0000 7fb6e62ef700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 0x7fb6e0197550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005b40 tx=0x7fb6dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.171+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc01d070 con 0x7fb6e0101650 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.171+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc021410 con 0x7fb6e0101650 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.171+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc0095f0 con 0x7fb6e0101650 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.171+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc017650 con 0x7fb6e0101650 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.171+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e0073b20 con 0x7fb6e0101650 2024-01-11T17:57:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.172+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e0108b00 con 0x7fb6e0101650 2024-01-11T17:57:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.174+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb6dc021580 con 0x7fb6e0101650 2024-01-11T17:57:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.176+0000 7fb6d77fe700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb6cc0a1ff0 0x7fb6cc0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.177+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb6dc0cf180 con 0x7fb6e0101650 2024-01-11T17:57:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.177+0000 7fb6e5aee700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb6cc0a1ff0 0x7fb6cc0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.177+0000 7fb6e5aee700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb6cc0a1ff0 0x7fb6cc0a44b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009df0 tx=0x7fb6d0012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.180+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb6dc097dc0 con 0x7fb6e0101650 2024-01-11T17:57:39.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.402+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e002d4c0 con 0x7fb6cc0a1ff0 2024-01-11T17:57:39.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.404+0000 7fb6d77fe700 1 -- 172.21.15.7:0/3760560470 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fb6e002d4c0 con 0x7fb6cc0a1ff0 2024-01-11T17:57:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-01-11T17:57:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-01-11T17:57:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-01-11T17:57:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 2m ago Has a FileSystem, LVM detected 2024-01-11T17:57:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb6cc0a1ff0 msgr2=0x7fb6cc0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb6cc0a1ff0 0x7fb6cc0a44b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009df0 tx=0x7fb6d0012040 comp rx=0 tx=0).stop 2024-01-11T17:57:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 msgr2=0x7fb6e0197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 0x7fb6e0197550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc005b40 tx=0x7fb6dc004730 comp rx=0 tx=0).stop 2024-01-11T17:57:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 shutdown_connections 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb6cc0a1ff0 0x7fb6cc0a44b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.406+0000 7fb6e8553700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6e01031f0 0x7fb6e019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.407+0000 7fb6e8553700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6e0101650 0x7fb6e0197550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.407+0000 7fb6e8553700 1 --2- 172.21.15.7:0/3760560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e0102850 0x7fb6e0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.407+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 >> 172.21.15.7:0/3760560470 conn(0x7fb6e00fcbb0 msgr2=0x7fb6e0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.407+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 shutdown_connections 2024-01-11T17:57:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:39.407+0000 7fb6e8553700 1 -- 172.21.15.7:0/3760560470 wait complete. 2024-01-11T17:57:39.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:39 smithi007 ceph-mon[33361]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-01-11T17:57:39.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:39 smithi016 ceph-mon[43195]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-01-11T17:57:39.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:39 smithi038 ceph-mon[43582]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-01-11T17:57:39.803 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-01-11T17:57:39.812 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:57:39.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph fs dump' 2024-01-11T17:57:40.239 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:40.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:40 smithi016 ceph-mon[43195]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:40.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:40 smithi007 ceph-mon[33361]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:40.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:40 smithi038 ceph-mon[43582]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-11T17:57:41.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3982512653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 msgr2=0x7fdafc102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:41.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3982512653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc102cb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0099c0 tx=0x7fdaec009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3982512653 shutdown_connections 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3982512653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdafc1031f0 0x7fdafc109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3982512653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 0x7fdafc101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3982512653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc102cb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3982512653 >> 172.21.15.7:0/3982512653 conn(0x7fdafc0fcbb0 msgr2=0x7fdafc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3982512653 shutdown_connections 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.476+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3982512653 wait complete. 2024-01-11T17:57:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.478+0000 7fdb03dbf700 1 Processor -- start 2024-01-11T17:57:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.478+0000 7fdb03dbf700 1 -- start start 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.478+0000 7fdb03dbf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 0x7fdafc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.478+0000 7fdb03dbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb03dbf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdafc1031f0 0x7fdafc1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb03dbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1a12c0 con 0x7fdafc102850 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb03dbf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1a1400 con 0x7fdafc101650 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb03dbf700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1a1590 con 0x7fdafc1031f0 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb01b5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 0x7fdafc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48686/0 (socket says 172.21.15.7:48686) 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 -- 172.21.15.7:0/3875458339 learned_addr learned my addr 172.21.15.7:0/3875458339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb01b5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 0x7fdafc199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50258/0 (socket says 172.21.15.7:50258) 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0235c700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdafc1031f0 0x7fdafc1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 -- 172.21.15.7:0/3875458339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdafc1031f0 msgr2=0x7fdafc1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdafc1031f0 0x7fdafc1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 -- 172.21.15.7:0/3875458339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 msgr2=0x7fdafc199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 0x7fdafc199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.479+0000 7fdb0135a700 1 -- 172.21.15.7:0/3875458339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec0095f0 con 0x7fdafc102850 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.480+0000 7fdb0135a700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc199c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009990 tx=0x7fdaec00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.480+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec01d070 con 0x7fdafc102850 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.480+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaec00bda0 con 0x7fdafc102850 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.480+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc1a1810 con 0x7fdafc102850 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.480+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0173f0 con 0x7fdafc102850 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.480+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc1a1c20 con 0x7fdafc102850 2024-01-11T17:57:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.481+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc0682d0 con 0x7fdafc102850 2024-01-11T17:57:41.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.483+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fdaec022470 con 0x7fdafc102850 2024-01-11T17:57:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.484+0000 7fdaf2ffd700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdae80a1f70 0x7fdae80a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.485+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdaec0cf0c0 con 0x7fdafc102850 2024-01-11T17:57:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.485+0000 7fdb01b5b700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdae80a1f70 0x7fdae80a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.485+0000 7fdb01b5b700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdae80a1f70 0x7fdae80a4430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdafc1026b0 tx=0x7fdaf8007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:41.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.487+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdaec097d00 con 0x7fdafc102850 2024-01-11T17:57:41.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:41 smithi007 ceph-mon[33361]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:41.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.741+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdafc10d060 con 0x7fdafc102850 2024-01-11T17:57:41.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.742+0000 7fdaf2ffd700 1 -- 172.21.15.7:0/3875458339 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2920 (secure 0 0 0) 0x7fdaec031b80 con 0x7fdafc102850 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:e22 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 22 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-01-11T17:56:08.339093+0000 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-01-11T17:56:34.314730+0000 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-01-11T17:57:41.743 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24602,1=24601,2=24577} 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-01-11T17:57:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3,4] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{0:24602} state up:active seq 3 addr [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{0:24565} state up:standby-replay seq 1 addr [v2:172.21.15.16:6836/2512926203,v1:172.21.15.16:6837/2512926203] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{1:24601} state up:active seq 9 addr [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{1:24559} state up:standby-replay seq 1 addr [v2:172.21.15.38:6836/2455039829,v1:172.21.15.38:6837/2455039829] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{2:24577} state up:active seq 11 addr [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{2:24547} state up:standby-replay seq 1 addr [v2:172.21.15.38:6834/1587519060,v1:172.21.15.38:6835/1587519060] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-01-11T17:57:41.745 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:41.746 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.7:6834/2806336658,v1:172.21.15.7:6835/2806336658] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.746 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.7:6836/1543402898,v1:172.21.15.7:6837/1543402898] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.746 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{-1:14589} state up:standby seq 1 addr [v2:172.21.15.7:6838/727242421,v1:172.21.15.7:6839/727242421] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.746 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{-1:24487} state up:standby seq 1 addr [v2:172.21.15.38:6832/3639639157,v1:172.21.15.38:6833/3639639157] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.746 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24488} state up:standby seq 1 addr [v2:172.21.15.16:6832/189478188,v1:172.21.15.16:6833/189478188] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.746 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{-1:24524} state up:standby seq 1 addr [v2:172.21.15.16:6834/4278122554,v1:172.21.15.16:6835/4278122554] compat {c=[1],r=[1],i=[fff]}] 2024-01-11T17:57:41.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:41 smithi016 ceph-mon[43195]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdae80a1f70 msgr2=0x7fdae80a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdae80a1f70 0x7fdae80a4430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdafc1026b0 tx=0x7fdaf8007400 comp rx=0 tx=0).stop 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 msgr2=0x7fdafc199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc199c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009990 tx=0x7fdaec00fa00 comp rx=0 tx=0).stop 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 shutdown_connections 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdae80a1f70 0x7fdae80a4430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdafc1031f0 0x7fdafc1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc101650 0x7fdafc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 --2- 172.21.15.7:0/3875458339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdafc102850 0x7fdafc199c70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 >> 172.21.15.7:0/3875458339 conn(0x7fdafc0fcbb0 msgr2=0x7fdafc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 shutdown_connections 2024-01-11T17:57:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:41.744+0000 7fdb03dbf700 1 -- 172.21.15.7:0/3875458339 wait complete. 2024-01-11T17:57:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:57:41.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:41 smithi038 ceph-mon[43582]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:42.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- bash -c 'ceph osd dump' 2024-01-11T17:57:42.568 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:42.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:42 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3875458339' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-11T17:57:42.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:42 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3875458339' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-11T17:57:42.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:42 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3875458339' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-11T17:57:43.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:43 smithi016 ceph-mon[43195]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 -- 172.21.15.7:0/3564882548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 msgr2=0x7f2468102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 --2- 172.21.15.7:0/3564882548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468102cb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24580099c0 tx=0x7f2458009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 -- 172.21.15.7:0/3564882548 shutdown_connections 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 --2- 172.21.15.7:0/3564882548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 0x7f2468101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 --2- 172.21.15.7:0/3564882548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 0x7f2468109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 --2- 172.21.15.7:0/3564882548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468102cb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 -- 172.21.15.7:0/3564882548 >> 172.21.15.7:0/3564882548 conn(0x7f24680fcbb0 msgr2=0x7f24680ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 -- 172.21.15.7:0/3564882548 shutdown_connections 2024-01-11T17:57:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.767+0000 7f246fdb8700 1 -- 172.21.15.7:0/3564882548 wait complete. 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.768+0000 7f246fdb8700 1 Processor -- start 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.768+0000 7f246fdb8700 1 -- start start 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.768+0000 7f246fdb8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 0x7f2468199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.768+0000 7f246fdb8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246fdb8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 0x7f24681a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246fdb8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24681a1270 con 0x7f2468102850 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246fdb8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24681a13b0 con 0x7f24681031f0 2024-01-11T17:57:43.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246fdb8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24681a1540 con 0x7f2468101650 2024-01-11T17:57:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246d353700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246db54700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 0x7f2468199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246db54700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 0x7f2468199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43430/0 (socket says 172.21.15.7:43430) 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246db54700 1 -- 172.21.15.7:0/481355082 learned_addr learned my addr 172.21.15.7:0/481355082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246e355700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 0x7f24681a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246d353700 1 -- 172.21.15.7:0/481355082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 msgr2=0x7f2468199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246d353700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 0x7f2468199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246d353700 1 -- 172.21.15.7:0/481355082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 msgr2=0x7f24681a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246d353700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 0x7f24681a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246d353700 1 -- 172.21.15.7:0/481355082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24580095f0 con 0x7f2468102850 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.769+0000 7f246e355700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 0x7f24681a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.770+0000 7f246d353700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468199c70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f24580058e0 tx=0x7f245800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.770+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245801d070 con 0x7f2468102850 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.770+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24681a17c0 con 0x7f2468102850 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.770+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2458005050 con 0x7f2468102850 2024-01-11T17:57:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.770+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24580049b0 con 0x7f2468102850 2024-01-11T17:57:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.770+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24681a1c60 con 0x7f2468102850 2024-01-11T17:57:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.771+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24680682d0 con 0x7f2468102850 2024-01-11T17:57:43.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.773+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f2458005430 con 0x7f2468102850 2024-01-11T17:57:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.774+0000 7f245effd700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f24540a1f70 0x7f24540a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.775+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f24580ceac0 con 0x7f2468102850 2024-01-11T17:57:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.775+0000 7f246db54700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f24540a1f70 0x7f24540a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.775+0000 7f246db54700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f24540a1f70 0x7f24540a4430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2464005950 tx=0x7f2464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:43.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:43 smithi038 ceph-mon[43582]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.778+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2458097700 con 0x7f2468102850 2024-01-11T17:57:43.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:43 smithi007 ceph-mon[33361]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:43.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.992+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f246810ce50 con 0x7f2468102850 2024-01-11T17:57:43.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.993+0000 7f245effd700 1 -- 172.21.15.7:0/481355082 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5571 (secure 0 0 0) 0x7f245809ad20 con 0x7f2468102850 2024-01-11T17:57:43.993 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 88 2024-01-11T17:57:43.993 INFO:teuthology.orchestra.run.smithi007.stdout:fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T17:57:43.993 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-01-11T17:49:24.248673+0000 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-01-11T17:57:27.343922+0000 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 27 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-01-11T17:57:43.994 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/953692014,v1:172.21.15.7:6803/953692014] [v2:172.21.15.7:6804/953692014,v1:172.21.15.7:6805/953692014] exists,up e34d29ca-d1a5-4771-a752-5acca1c7d041 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6800/2714939694,v1:172.21.15.16:6801/2714939694] [v2:172.21.15.16:6802/2714939694,v1:172.21.15.16:6803/2714939694] exists,up 7b918edd-6f35-4186-83f6-8dea129fa4d8 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6800/2733698254,v1:172.21.15.38:6801/2733698254] [v2:172.21.15.38:6802/2733698254,v1:172.21.15.38:6803/2733698254] exists,up 85ec6460-ec16-4142-abff-60b2b9d83550 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/1132146736,v1:172.21.15.7:6811/1132146736] [v2:172.21.15.7:6812/1132146736,v1:172.21.15.7:6813/1132146736] exists,up ad9f9282-a8c3-43df-954d-8bc2a5854cf1 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 32 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6808/1933820349,v1:172.21.15.16:6809/1933820349] [v2:172.21.15.16:6810/1933820349,v1:172.21.15.16:6811/1933820349] exists,up 3740f4c2-4680-4dce-b64a-65e8405a98e8 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6808/3973290350,v1:172.21.15.38:6809/3973290350] [v2:172.21.15.38:6810/3973290350,v1:172.21.15.38:6811/3973290350] exists,up e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 44 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/1282115079,v1:172.21.15.7:6819/1282115079] [v2:172.21.15.7:6820/1282115079,v1:172.21.15.7:6821/1282115079] exists,up 731711fe-aa80-4c77-a022-b1a16f24306b 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6816/760747298,v1:172.21.15.16:6817/760747298] [v2:172.21.15.16:6818/760747298,v1:172.21.15.16:6819/760747298] exists,up e1e933c6-92f2-4974-83cf-633e3530103e 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 56 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6816/659578485,v1:172.21.15.38:6817/659578485] [v2:172.21.15.38:6818/659578485,v1:172.21.15.38:6819/659578485] exists,up e3bb6d44-a03b-4e41-a286-3f6e2104397d 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/1883138205,v1:172.21.15.7:6827/1883138205] [v2:172.21.15.7:6828/1883138205,v1:172.21.15.7:6829/1883138205] exists,up fb7e8378-c57e-4c86-9184-fe68617c8169 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 68 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6824/2304237438,v1:172.21.15.16:6825/2304237438] [v2:172.21.15.16:6826/2304237438,v1:172.21.15.16:6827/2304237438] exists,up bf5a2cb2-377f-4650-9971-178ba7b1b176 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 75 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6824/3838329833,v1:172.21.15.38:6825/3838329833] [v2:172.21.15.38:6826/3838329833,v1:172.21.15.38:6827/3838329833] exists,up 5dc65b78-877e-4fc7-8aef-6d5d8af09353 2024-01-11T17:57:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.a [2,11] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.1c [8,11] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.21 [5,9] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.2c [4,9] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.31 [4,6] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.3a [4,8] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.3b [3,9] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.b [9,8] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.e [9,11] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.11 [5,10] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.17 [9,8] 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3570192633 expires 2024-01-12T17:50:24.497330+0000 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/867047662 expires 2024-01-12T17:50:24.497330+0000 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1912965293 expires 2024-01-12T17:50:24.497330+0000 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3811163237 expires 2024-01-12T17:49:57.975302+0000 2024-01-11T17:57:43.996 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3086228555 expires 2024-01-12T17:49:57.975302+0000 2024-01-11T17:57:43.997 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/867047662 expires 2024-01-12T17:50:24.497330+0000 2024-01-11T17:57:43.997 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/4071850608 expires 2024-01-12T17:50:24.497330+0000 2024-01-11T17:57:43.997 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/1981932676 expires 2024-01-12T17:49:57.975302+0000 2024-01-11T17:57:43.997 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/1981932676 expires 2024-01-12T17:49:57.975302+0000 2024-01-11T17:57:43.997 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/113093279 expires 2024-01-12T17:49:57.975302+0000 2024-01-11T17:57:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f24540a1f70 msgr2=0x7f24540a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f24540a1f70 0x7f24540a4430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2464005950 tx=0x7f2464009260 comp rx=0 tx=0).stop 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 msgr2=0x7f2468199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468199c70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f24580058e0 tx=0x7f245800f690 comp rx=0 tx=0).stop 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 shutdown_connections 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f24540a1f70 0x7f24540a4430 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2468101650 0x7f2468199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24681031f0 0x7f24681a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 --2- 172.21.15.7:0/481355082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2468102850 0x7f2468199c70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 >> 172.21.15.7:0/481355082 conn(0x7f24680fcbb0 msgr2=0x7f2468073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 shutdown_connections 2024-01-11T17:57:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:43.995+0000 7f246fdb8700 1 -- 172.21.15.7:0/481355082 wait complete. 2024-01-11T17:57:44.424 INFO:teuthology.run_tasks:Running task fs.ready... 2024-01-11T17:57:44.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:44.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 -- 172.21.15.7:0/1041945182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380102790 msgr2=0x7fa380102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:44.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 --2- 172.21.15.7:0/1041945182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380102790 0x7fa380102bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3740099c0 tx=0x7fa374009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 -- 172.21.15.7:0/1041945182 shutdown_connections 2024-01-11T17:57:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 --2- 172.21.15.7:0/1041945182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3801015a0 0x7fa3801019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 --2- 172.21.15.7:0/1041945182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa380103130 0x7fa3801099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 --2- 172.21.15.7:0/1041945182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380102790 0x7fa380102bf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 -- 172.21.15.7:0/1041945182 >> 172.21.15.7:0/1041945182 conn(0x7fa3800fcb30 msgr2=0x7fa3800fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 -- 172.21.15.7:0/1041945182 shutdown_connections 2024-01-11T17:57:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.644+0000 7fa385dff700 1 -- 172.21.15.7:0/1041945182 wait complete. 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.645+0000 7fa385dff700 1 Processor -- start 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.645+0000 7fa385dff700 1 -- start start 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa385dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3801015a0 0x7fa380197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa385dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa380102790 0x7fa3801979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa385dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 0x7fa38019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa385dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019f000 con 0x7fa380103130 2024-01-11T17:57:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa385dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019f140 con 0x7fa3801015a0 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa385dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019f2d0 con 0x7fa380102790 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 0x7fa38019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 0x7fa38019ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44582/0 (socket says 172.21.15.7:44582) 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 -- 172.21.15.7:0/3576495036 learned_addr learned my addr 172.21.15.7:0/3576495036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37effd700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa380102790 0x7fa3801979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37f7fe700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3801015a0 0x7fa380197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 -- 172.21.15.7:0/3576495036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa380102790 msgr2=0x7fa3801979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa380102790 0x7fa3801979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 -- 172.21.15.7:0/3576495036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3801015a0 msgr2=0x7fa380197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3801015a0 0x7fa380197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.646+0000 7fa37ffff700 1 -- 172.21.15.7:0/3576495036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740095f0 con 0x7fa380103130 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.647+0000 7fa37ffff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 0x7fa38019ea30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa37800ef30 tx=0x7fa37800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.647+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378018070 con 0x7fa380103130 2024-01-11T17:57:44.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.647+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38019f550 con 0x7fa380103130 2024-01-11T17:57:44.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.647+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37800f040 con 0x7fa380103130 2024-01-11T17:57:44.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.647+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800cd00 con 0x7fa380103130 2024-01-11T17:57:44.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.647+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38019fa10 con 0x7fa380103130 2024-01-11T17:57:44.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.648+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa380108950 con 0x7fa380103130 2024-01-11T17:57:44.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.650+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fa378003a00 con 0x7fa380103130 2024-01-11T17:57:44.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.652+0000 7fa37cff9700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3680a1f70 0x7fa3680a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:44.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.652+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa3780cd9e0 con 0x7fa380103130 2024-01-11T17:57:44.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.652+0000 7fa37f7fe700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3680a1f70 0x7fa3680a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.653+0000 7fa37f7fe700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3680a1f70 0x7fa3680a4430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa3801025f0 tx=0x7fa370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.656+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa378096680 con 0x7fa380103130 2024-01-11T17:57:44.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:44 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/481355082' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-11T17:57:44.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:44 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/481355082' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-11T17:57:44.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:44 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/481355082' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-11T17:57:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.875+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa38010cfa0 con 0x7fa380103130 2024-01-11T17:57:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.877+0000 7fa37cff9700 1 -- 172.21.15.7:0/3576495036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fa378099ca0 con 0x7fa380103130 2024-01-11T17:57:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:44.878 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3680a1f70 msgr2=0x7fa3680a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3680a1f70 0x7fa3680a4430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa3801025f0 tx=0x7fa370009260 comp rx=0 tx=0).stop 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 msgr2=0x7fa38019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 0x7fa38019ea30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa37800ef30 tx=0x7fa37800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 shutdown_connections 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3680a1f70 0x7fa3680a4430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa380103130 0x7fa38019ea30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3801015a0 0x7fa380197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 --2- 172.21.15.7:0/3576495036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa380102790 0x7fa3801979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 >> 172.21.15.7:0/3576495036 conn(0x7fa3800fcb30 msgr2=0x7fa380105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 shutdown_connections 2024-01-11T17:57:44.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:44.880+0000 7fa385dff700 1 -- 172.21.15.7:0/3576495036 wait complete. 2024-01-11T17:57:44.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:57:44.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:57:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 -- 172.21.15.7:0/1693483513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe184103140 msgr2=0x7fe1841099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 --2- 172.21.15.7:0/1693483513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe184103140 0x7fe1841099f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00b0d0 tx=0x7fe17c00b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 -- 172.21.15.7:0/1693483513 shutdown_connections 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 --2- 172.21.15.7:0/1693483513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1841015b0 0x7fe1841019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 --2- 172.21.15.7:0/1693483513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe184103140 0x7fe1841099f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 --2- 172.21.15.7:0/1693483513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1841027a0 0x7fe184102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.108+0000 7fe1895ff700 1 -- 172.21.15.7:0/1693483513 >> 172.21.15.7:0/1693483513 conn(0x7fe1840fcb00 msgr2=0x7fe1840fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.109+0000 7fe1895ff700 1 -- 172.21.15.7:0/1693483513 shutdown_connections 2024-01-11T17:57:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.109+0000 7fe1895ff700 1 -- 172.21.15.7:0/1693483513 wait complete. 2024-01-11T17:57:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.110+0000 7fe1895ff700 1 Processor -- start 2024-01-11T17:57:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.110+0000 7fe1895ff700 1 -- start start 2024-01-11T17:57:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1895ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 0x7fe1840753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1895ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1841027a0 0x7fe184073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1895ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe184103140 0x7fe184073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1895ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1840746e0 con 0x7fe1841015b0 2024-01-11T17:57:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1895ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184074850 con 0x7fe1841027a0 2024-01-11T17:57:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1895ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1841a1650 con 0x7fe184103140 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 0x7fe1840753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 0x7fe1840753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44594/0 (socket says 172.21.15.7:44594) 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 -- 172.21.15.7:0/4211496970 learned_addr learned my addr 172.21.15.7:0/4211496970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1827fc700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1841027a0 0x7fe184073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1837fe700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe184103140 0x7fe184073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 -- 172.21.15.7:0/4211496970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe184103140 msgr2=0x7fe184073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe184103140 0x7fe184073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 -- 172.21.15.7:0/4211496970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1841027a0 msgr2=0x7fe184073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1841027a0 0x7fe184073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe182ffd700 1 -- 172.21.15.7:0/4211496970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c009d00 con 0x7fe1841015b0 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.111+0000 7fe1837fe700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe184103140 0x7fe184073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.112+0000 7fe182ffd700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 0x7fe1840753c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe17400e970 tx=0x7fe17400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.112+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174009790 con 0x7fe1841015b0 2024-01-11T17:57:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.112+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe17400eec0 con 0x7fe1841015b0 2024-01-11T17:57:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.112+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1841a1930 con 0x7fe1841015b0 2024-01-11T17:57:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.112+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174003cf0 con 0x7fe1841015b0 2024-01-11T17:57:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.112+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1841a1e40 con 0x7fe1841015b0 2024-01-11T17:57:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.113+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe184108900 con 0x7fe1841015b0 2024-01-11T17:57:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.115+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fe17400ccd0 con 0x7fe1841015b0 2024-01-11T17:57:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.116+0000 7fe16bfff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe16c0a1f70 0x7fe16c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.116+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe174014070 con 0x7fe1841015b0 2024-01-11T17:57:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.117+0000 7fe1827fc700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe16c0a1f70 0x7fe16c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.117+0000 7fe1827fc700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe16c0a1f70 0x7fe16c0a4430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe178009c60 tx=0x7fe178009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.120+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fe174096cf0 con 0x7fe1841015b0 2024-01-11T17:57:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.337+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe18410cec0 con 0x7fe1841015b0 2024-01-11T17:57:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.338+0000 7fe16bfff700 1 -- 172.21.15.7:0/4211496970 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe17409a310 con 0x7fe1841015b0 2024-01-11T17:57:45.339 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:57:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe16c0a1f70 msgr2=0x7fe16c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe16c0a1f70 0x7fe16c0a4430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe178009c60 tx=0x7fe178009260 comp rx=0 tx=0).stop 2024-01-11T17:57:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 msgr2=0x7fe1840753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 0x7fe1840753c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe17400e970 tx=0x7fe17400ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 shutdown_connections 2024-01-11T17:57:45.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe16c0a1f70 0x7fe16c0a4430 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe184103140 0x7fe184073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1841015b0 0x7fe1840753c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 --2- 172.21.15.7:0/4211496970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1841027a0 0x7fe184073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.340+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 >> 172.21.15.7:0/4211496970 conn(0x7fe1840fcb00 msgr2=0x7fe184105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.341+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 shutdown_connections 2024-01-11T17:57:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.341+0000 7fe1895ff700 1 -- 172.21.15.7:0/4211496970 wait complete. 2024-01-11T17:57:45.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:57:45.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 -- 172.21.15.7:0/1120603471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 msgr2=0x7f3260102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 --2- 172.21.15.7:0/1120603471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260102c00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f32540099c0 tx=0x7f3254009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 -- 172.21.15.7:0/1120603471 shutdown_connections 2024-01-11T17:57:45.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 --2- 172.21.15.7:0/1120603471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f32601019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 --2- 172.21.15.7:0/1120603471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3260103140 0x7f32601099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 --2- 172.21.15.7:0/1120603471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260102c00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 -- 172.21.15.7:0/1120603471 >> 172.21.15.7:0/1120603471 conn(0x7f32600fcb00 msgr2=0x7f32600fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:45.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 -- 172.21.15.7:0/1120603471 shutdown_connections 2024-01-11T17:57:45.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.568+0000 7f32665ff700 1 -- 172.21.15.7:0/1120603471 wait complete. 2024-01-11T17:57:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.569+0000 7f32665ff700 1 Processor -- start 2024-01-11T17:57:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.569+0000 7f32665ff700 1 -- start start 2024-01-11T17:57:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f32665ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f3260199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f32665ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f32665ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3260103140 0x7f32601a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f32665ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32601a1240 con 0x7f32601027a0 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f32665ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32601a1380 con 0x7f3260103140 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f32665ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32601a1510 con 0x7f32601015b0 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f325ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f3260199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f325f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f325ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f3260199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43466/0 (socket says 172.21.15.7:43466) 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f325f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44606/0 (socket says 172.21.15.7:44606) 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f325ffff700 1 -- 172.21.15.7:0/2532641426 learned_addr learned my addr 172.21.15.7:0/2532641426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.570+0000 7f3264b9c700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3260103140 0x7f32601a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325f7fe700 1 -- 172.21.15.7:0/2532641426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 msgr2=0x7f3260199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325f7fe700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f3260199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325f7fe700 1 -- 172.21.15.7:0/2532641426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3260103140 msgr2=0x7f32601a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325f7fe700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3260103140 0x7f32601a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325f7fe700 1 -- 172.21.15.7:0/2532641426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540095f0 con 0x7f32601027a0 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325ffff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f3260199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:45.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325f7fe700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260199c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f325400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325401d070 con 0x7f32601027a0 2024-01-11T17:57:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325400bda0 con 0x7f32601027a0 2024-01-11T17:57:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254004b00 con 0x7f32601027a0 2024-01-11T17:57:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32601a1790 con 0x7f32601027a0 2024-01-11T17:57:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.571+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32601a1bb0 con 0x7f32601027a0 2024-01-11T17:57:45.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.572+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32600683f0 con 0x7f32601027a0 2024-01-11T17:57:45.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.574+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f3254004c60 con 0x7f32601027a0 2024-01-11T17:57:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.576+0000 7f325d7fa700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f32480a1f70 0x7f32480a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.576+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f32540cebd0 con 0x7f32601027a0 2024-01-11T17:57:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.576+0000 7f325ffff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f32480a1f70 0x7f32480a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:45.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.577+0000 7f325ffff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f32480a1f70 0x7f32480a4430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3250005950 tx=0x7f3250009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.580+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3254097870 con 0x7f32601027a0 2024-01-11T17:57:45.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:45 smithi016 ceph-mon[43195]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:45.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:45 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3576495036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:45.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:45 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4211496970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:45.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:45 smithi038 ceph-mon[43582]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:45.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:45 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3576495036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:45.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:45 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4211496970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:45.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.792+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32601a1da0 con 0x7f32601027a0 2024-01-11T17:57:45.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.792+0000 7f325d7fa700 1 -- 172.21.15.7:0/2532641426 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f325409ae90 con 0x7f32601027a0 2024-01-11T17:57:45.793 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f32480a1f70 msgr2=0x7f32480a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f32480a1f70 0x7f32480a4430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3250005950 tx=0x7f3250009260 comp rx=0 tx=0).stop 2024-01-11T17:57:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 msgr2=0x7f3260199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:45.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260199c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f325400fa00 comp rx=0 tx=0).stop 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 shutdown_connections 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f32480a1f70 0x7f32480a4430 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32601015b0 0x7f3260199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3260103140 0x7f32601a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 --2- 172.21.15.7:0/2532641426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32601027a0 0x7f3260199c50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.794+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 >> 172.21.15.7:0/2532641426 conn(0x7f32600fcb00 msgr2=0x7f3260073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.795+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 shutdown_connections 2024-01-11T17:57:45.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:45.795+0000 7f32665ff700 1 -- 172.21.15.7:0/2532641426 wait complete. 2024-01-11T17:57:45.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:45 smithi007 ceph-mon[33361]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:45.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:45 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3576495036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:45.812 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:45 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4211496970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:45.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 -- 172.21.15.7:0/2078191161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694075650 msgr2=0x7fc694073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/2078191161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694075650 0x7fc694073aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc684009990 tx=0x7fc684009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 -- 172.21.15.7:0/2078191161 shutdown_connections 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/2078191161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694075650 0x7fc694073aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/2078191161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc694109470 0x7fc69410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/2078191161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694074070 0x7fc6940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 -- 172.21.15.7:0/2078191161 >> 172.21.15.7:0/2078191161 conn(0x7fc6940fcb00 msgr2=0x7fc6940fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 -- 172.21.15.7:0/2078191161 shutdown_connections 2024-01-11T17:57:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.022+0000 7fc69a7ff700 1 -- 172.21.15.7:0/2078191161 wait complete. 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.023+0000 7fc69a7ff700 1 Processor -- start 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.023+0000 7fc69a7ff700 1 -- start start 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc69a7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc694074070 0x7fc694199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc69a7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 0x7fc694199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc69a7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694109470 0x7fc6941a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc69a7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941a1260 con 0x7fc694075650 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc69a7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941a13a0 con 0x7fc694074070 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc69a7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941a1530 con 0x7fc694109470 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 0x7fc694199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc693fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc694074070 0x7fc694199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 0x7fc694199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44622/0 (socket says 172.21.15.7:44622) 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc6937fe700 1 -- 172.21.15.7:0/3865296571 learned_addr learned my addr 172.21.15.7:0/3865296571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.024+0000 7fc698d9c700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694109470 0x7fc6941a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6937fe700 1 -- 172.21.15.7:0/3865296571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694109470 msgr2=0x7fc6941a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6937fe700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694109470 0x7fc6941a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6937fe700 1 -- 172.21.15.7:0/3865296571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc694074070 msgr2=0x7fc694199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6937fe700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc694074070 0x7fc694199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6937fe700 1 -- 172.21.15.7:0/3865296571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840095f0 con 0x7fc694075650 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6937fe700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 0x7fc694199c70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc68800e970 tx=0x7fc68800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688009790 con 0x7fc694075650 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6941a1810 con 0x7fc694075650 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc68800c7e0 con 0x7fc694075650 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688003c70 con 0x7fc694075650 2024-01-11T17:57:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.025+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6941a1d70 con 0x7fc694075650 2024-01-11T17:57:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.026+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc694108230 con 0x7fc694075650 2024-01-11T17:57:46.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.028+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fc68800cc50 con 0x7fc694075650 2024-01-11T17:57:46.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.030+0000 7fc6917fa700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc67c0a1ff0 0x7fc67c0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.030+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc688014070 con 0x7fc694075650 2024-01-11T17:57:46.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.030+0000 7fc693fff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc67c0a1ff0 0x7fc67c0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.031+0000 7fc693fff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc67c0a1ff0 0x7fc67c0a44b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc68400b2e0 tx=0x7fc68400b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:46.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.034+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc688096bb0 con 0x7fc694075650 2024-01-11T17:57:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.243+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc69410f190 con 0x7fc694075650 2024-01-11T17:57:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.244+0000 7fc6917fa700 1 -- 172.21.15.7:0/3865296571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc688003dd0 con 0x7fc694075650 2024-01-11T17:57:46.245 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:57:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc67c0a1ff0 msgr2=0x7fc67c0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc67c0a1ff0 0x7fc67c0a44b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc68400b2e0 tx=0x7fc68400b750 comp rx=0 tx=0).stop 2024-01-11T17:57:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 msgr2=0x7fc694199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 0x7fc694199c70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc68800e970 tx=0x7fc68800ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 shutdown_connections 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc67c0a1ff0 0x7fc67c0a44b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc694109470 0x7fc6941a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc694074070 0x7fc694199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 --2- 172.21.15.7:0/3865296571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc694075650 0x7fc694199c70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 >> 172.21.15.7:0/3865296571 conn(0x7fc6940fcb00 msgr2=0x7fc694105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 shutdown_connections 2024-01-11T17:57:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.246+0000 7fc69a7ff700 1 -- 172.21.15.7:0/3865296571 wait complete. 2024-01-11T17:57:46.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 -- 172.21.15.7:0/10623010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1015b0 msgr2=0x7f934c1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 --2- 172.21.15.7:0/10623010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1015b0 0x7f934c1019d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f933c0099c0 tx=0x7f933c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 -- 172.21.15.7:0/10623010 shutdown_connections 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 --2- 172.21.15.7:0/10623010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934c103140 0x7f934c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 --2- 172.21.15.7:0/10623010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1015b0 0x7f934c1019d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 --2- 172.21.15.7:0/10623010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c1027a0 0x7f934c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 -- 172.21.15.7:0/10623010 >> 172.21.15.7:0/10623010 conn(0x7f934c0fcb00 msgr2=0x7f934c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 -- 172.21.15.7:0/10623010 shutdown_connections 2024-01-11T17:57:46.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.475+0000 7f93513ff700 1 -- 172.21.15.7:0/10623010 wait complete. 2024-01-11T17:57:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 Processor -- start 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 -- start start 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934c1015b0 0x7f934c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 0x7f934c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c103140 0x7f934c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a1260 con 0x7f934c1027a0 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a13a0 con 0x7f934c1015b0 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.476+0000 7f93513ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a1530 con 0x7f934c103140 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 0x7f934c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 0x7f934c199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44626/0 (socket says 172.21.15.7:44626) 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 -- 172.21.15.7:0/1249991210 learned_addr learned my addr 172.21.15.7:0/1249991210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f934b7fe700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c103140 0x7f934c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f934affd700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934c1015b0 0x7f934c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 -- 172.21.15.7:0/1249991210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c103140 msgr2=0x7f934c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c103140 0x7f934c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 -- 172.21.15.7:0/1249991210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934c1015b0 msgr2=0x7f934c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934c1015b0 0x7f934c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 -- 172.21.15.7:0/1249991210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0095f0 con 0x7f934c1027a0 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.477+0000 7f9343fff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 0x7f934c199c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f933400e9a0 tx=0x7f933400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.478+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334009790 con 0x7f934c1027a0 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.478+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c1a1810 con 0x7f934c1027a0 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.478+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9334010430 con 0x7f934c1027a0 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.478+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334018680 con 0x7f934c1027a0 2024-01-11T17:57:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.478+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c1a1d70 con 0x7f934c1027a0 2024-01-11T17:57:46.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.479+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c0683f0 con 0x7f934c1027a0 2024-01-11T17:57:46.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.481+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f93340105a0 con 0x7f934c1027a0 2024-01-11T17:57:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.483+0000 7f9348ff9700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f93380a1f70 0x7f93380a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.484+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9334014070 con 0x7f934c1027a0 2024-01-11T17:57:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.484+0000 7f934affd700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f93380a1f70 0x7f93380a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.484+0000 7f934affd700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f93380a1f70 0x7f93380a4430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f933c009990 tx=0x7f933c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:46.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.485+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f9334096e80 con 0x7f934c1027a0 2024-01-11T17:57:46.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:46 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2532641426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:46.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:46 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3865296571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:46.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.700+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f934c10d070 con 0x7f934c1027a0 2024-01-11T17:57:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.700+0000 7f9348ff9700 1 -- 172.21.15.7:0/1249991210 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9334026020 con 0x7f934c1027a0 2024-01-11T17:57:46.701 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.702+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f93380a1f70 msgr2=0x7f93380a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.702+0000 7f93513ff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f93380a1f70 0x7f93380a4430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f933c009990 tx=0x7f933c000bc0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.702+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 msgr2=0x7f934c199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.702+0000 7f93513ff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 0x7f934c199c20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f933400e9a0 tx=0x7f933400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.702+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 shutdown_connections 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.702+0000 7f93513ff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f93380a1f70 0x7f93380a4430 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.703+0000 7f93513ff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f934c103140 0x7f934c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.703+0000 7f93513ff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934c1015b0 0x7f934c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.703+0000 7f93513ff700 1 --2- 172.21.15.7:0/1249991210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f934c1027a0 0x7f934c199c20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.703+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 >> 172.21.15.7:0/1249991210 conn(0x7f934c0fcb00 msgr2=0x7f934c107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.703+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 shutdown_connections 2024-01-11T17:57:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.703+0000 7f93513ff700 1 -- 172.21.15.7:0/1249991210 wait complete. 2024-01-11T17:57:46.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:57:46.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:46 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2532641426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:46.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:46 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3865296571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:46 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2532641426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:46.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:46 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3865296571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 -- 172.21.15.7:0/2371745539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 msgr2=0x7f48c81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 --2- 172.21.15.7:0/2371745539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81019b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009990 tx=0x7f48b8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 -- 172.21.15.7:0/2371745539 shutdown_connections 2024-01-11T17:57:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 --2- 172.21.15.7:0/2371745539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81019b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 --2- 172.21.15.7:0/2371745539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8103120 0x7f48c81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 --2- 172.21.15.7:0/2371745539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8102780 0x7f48c8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 -- 172.21.15.7:0/2371745539 >> 172.21.15.7:0/2371745539 conn(0x7f48c80fcb00 msgr2=0x7f48c80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 -- 172.21.15.7:0/2371745539 shutdown_connections 2024-01-11T17:57:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.926+0000 7f48ccfff700 1 -- 172.21.15.7:0/2371745539 wait complete. 2024-01-11T17:57:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.927+0000 7f48ccfff700 1 Processor -- start 2024-01-11T17:57:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.927+0000 7f48ccfff700 1 -- start start 2024-01-11T17:57:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48ccfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48ccfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8102780 0x7f48c8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48ccfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48ccfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81a1220 con 0x7f48c8103120 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48ccfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81a1360 con 0x7f48c8102780 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48ccfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81a14f0 con 0x7f48c8101590 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44640/0 (socket says 172.21.15.7:44640) 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 -- 172.21.15.7:0/1346678015 learned_addr learned my addr 172.21.15.7:0/1346678015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48bffff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8102780 0x7f48c8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 -- 172.21.15.7:0/1346678015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 msgr2=0x7f48c81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 -- 172.21.15.7:0/1346678015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8102780 msgr2=0x7f48c8199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8102780 0x7f48c8199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.928+0000 7f48c6d9d700 1 -- 172.21.15.7:0/1346678015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80095f0 con 0x7f48c8103120 2024-01-11T17:57:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.929+0000 7f48c6d9d700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81a0c50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f48c000ef30 tx=0x7f48c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.929+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0018070 con 0x7f48c8103120 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.929+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48c000f040 con 0x7f48c8103120 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.929+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c81a1770 con 0x7f48c8103120 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.929+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000cd00 con 0x7f48c8103120 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.929+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c81a1c80 con 0x7f48c8103120 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.930+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c80683f0 con 0x7f48c8103120 2024-01-11T17:57:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.932+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f48c0003a00 con 0x7f48c8103120 2024-01-11T17:57:46.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.933+0000 7f48bf7fe700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b40a1ff0 0x7f48b40a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:46.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.933+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f48c00cd9f0 con 0x7f48c8103120 2024-01-11T17:57:46.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.934+0000 7f48c659c700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b40a1ff0 0x7f48b40a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:46.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.934+0000 7f48c659c700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b40a1ff0 0x7f48b40a44b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b680 tx=0x7f48b800b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:46.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:46.937+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f48c0096630 con 0x7f48c8103120 2024-01-11T17:57:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.200+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48c810cfd0 con 0x7f48c8103120 2024-01-11T17:57:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.200+0000 7f48bf7fe700 1 -- 172.21.15.7:0/1346678015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11309 (secure 0 0 0) 0x7f48c0099c50 con 0x7f48c8103120 2024-01-11T17:57:47.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:47.202 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:34.314730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:57:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b40a1ff0 msgr2=0x7f48b40a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b40a1ff0 0x7f48b40a44b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b680 tx=0x7f48b800b750 comp rx=0 tx=0).stop 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 msgr2=0x7f48c81a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81a0c50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f48c000ef30 tx=0x7f48c000c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 shutdown_connections 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b40a1ff0 0x7f48b40a44b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81a0c50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 --2- 172.21.15.7:0/1346678015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8102780 0x7f48c8199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 >> 172.21.15.7:0/1346678015 conn(0x7f48c80fcb00 msgr2=0x7f48c8107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 shutdown_connections 2024-01-11T17:57:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.203+0000 7f48ccfff700 1 -- 172.21.15.7:0/1346678015 wait complete. 2024-01-11T17:57:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:57:47.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 -- 172.21.15.7:0/3703014257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 msgr2=0x7fa73c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 --2- 172.21.15.7:0/3703014257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c1019b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0099c0 tx=0x7fa72c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 -- 172.21.15.7:0/3703014257 shutdown_connections 2024-01-11T17:57:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 --2- 172.21.15.7:0/3703014257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c103120 0x7fa73c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 --2- 172.21.15.7:0/3703014257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c1019b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 --2- 172.21.15.7:0/3703014257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c102780 0x7fa73c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 -- 172.21.15.7:0/3703014257 >> 172.21.15.7:0/3703014257 conn(0x7fa73c0fcb00 msgr2=0x7fa73c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 -- 172.21.15.7:0/3703014257 shutdown_connections 2024-01-11T17:57:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.429+0000 7fa741fff700 1 -- 172.21.15.7:0/3703014257 wait complete. 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 Processor -- start 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 -- start start 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c102780 0x7fa73c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c103120 0x7fa73c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c1a12a0 con 0x7fa73c101590 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c1a13e0 con 0x7fa73c102780 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa741fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c1a1570 con 0x7fa73c103120 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44664/0 (socket says 172.21.15.7:44664) 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa73b7fe700 1 -- 172.21.15.7:0/893786234 learned_addr learned my addr 172.21.15.7:0/893786234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.431+0000 7fa73affd700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c102780 0x7fa73c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73bfff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c103120 0x7fa73c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73b7fe700 1 -- 172.21.15.7:0/893786234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c103120 msgr2=0x7fa73c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73b7fe700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c103120 0x7fa73c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73b7fe700 1 -- 172.21.15.7:0/893786234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c102780 msgr2=0x7fa73c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73b7fe700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c102780 0x7fa73c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73b7fe700 1 -- 172.21.15.7:0/893786234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0095f0 con 0x7fa73c101590 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73affd700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c102780 0x7fa73c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa73b7fe700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c199720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa72c003730 tx=0x7fa72c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c01d070 con 0x7fa73c101590 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72c00fe70 con 0x7fa73c101590 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0173f0 con 0x7fa73c101590 2024-01-11T17:57:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.432+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c1a17f0 con 0x7fa73c101590 2024-01-11T17:57:47.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.433+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c1a1bc0 con 0x7fa73c101590 2024-01-11T17:57:47.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.434+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c068430 con 0x7fa73c101590 2024-01-11T17:57:47.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.435+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fa72c022470 con 0x7fa73c101590 2024-01-11T17:57:47.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.437+0000 7fa738ff9700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa7240a1ff0 0x7fa7240a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.437+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa72c0cefe0 con 0x7fa73c101590 2024-01-11T17:57:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.437+0000 7fa73affd700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa7240a1ff0 0x7fa7240a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.438+0000 7fa73affd700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa7240a1ff0 0x7fa7240a44b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa730011490 tx=0x7fa730009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.441+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa72c097c80 con 0x7fa73c101590 2024-01-11T17:57:47.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:47 smithi007 ceph-mon[33361]: pgmap v299: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-11T17:57:47.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:47 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1249991210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:47.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:47 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1346678015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.655+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa73c10d090 con 0x7fa73c101590 2024-01-11T17:57:47.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.656+0000 7fa738ff9700 1 -- 172.21.15.7:0/893786234 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fa72c09b2a0 con 0x7fa73c101590 2024-01-11T17:57:47.657 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:47.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa7240a1ff0 msgr2=0x7fa7240a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa7240a1ff0 0x7fa7240a44b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa730011490 tx=0x7fa730009260 comp rx=0 tx=0).stop 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 msgr2=0x7fa73c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c199720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa72c003730 tx=0x7fa72c00fa00 comp rx=0 tx=0).stop 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 shutdown_connections 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa7240a1ff0 0x7fa7240a44b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c103120 0x7fa73c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c101590 0x7fa73c199720 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 --2- 172.21.15.7:0/893786234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73c102780 0x7fa73c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 >> 172.21.15.7:0/893786234 conn(0x7fa73c0fcb00 msgr2=0x7fa73c107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 shutdown_connections 2024-01-11T17:57:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.659+0000 7fa741fff700 1 -- 172.21.15.7:0/893786234 wait complete. 2024-01-11T17:57:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:57:47.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:57:47.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:47 smithi016 ceph-mon[43195]: pgmap v299: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-11T17:57:47.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:47 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1249991210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:47.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:47 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1346678015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:47.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:47 smithi038 ceph-mon[43582]: pgmap v299: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-11T17:57:47.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:47 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1249991210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:47.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:47 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1346678015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 -- 172.21.15.7:0/1021720272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 msgr2=0x7ff2081019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 --2- 172.21.15.7:0/1021720272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 0x7ff2081019d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff204009990 tx=0x7ff204009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 -- 172.21.15.7:0/1021720272 shutdown_connections 2024-01-11T17:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 --2- 172.21.15.7:0/1021720272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 0x7ff2081019d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 --2- 172.21.15.7:0/1021720272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff208103140 0x7ff2081099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 --2- 172.21.15.7:0/1021720272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2081027a0 0x7ff208102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 -- 172.21.15.7:0/1021720272 >> 172.21.15.7:0/1021720272 conn(0x7ff2080fcb00 msgr2=0x7ff2080fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 -- 172.21.15.7:0/1021720272 shutdown_connections 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.888+0000 7ff20fbff700 1 -- 172.21.15.7:0/1021720272 wait complete. 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.889+0000 7ff20fbff700 1 Processor -- start 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 -- start start 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 0x7ff2080753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2081027a0 0x7ff208073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 0x7ff208073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080746e0 con 0x7ff208103140 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff208074850 con 0x7ff2081027a0 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20fbff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2081a1530 con 0x7ff2081015b0 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20e19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 0x7ff208073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20d19a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2081027a0 0x7ff208073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20e19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 0x7ff208073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44688/0 (socket says 172.21.15.7:44688) 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20e19c700 1 -- 172.21.15.7:0/2256878659 learned_addr learned my addr 172.21.15.7:0/2256878659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.890+0000 7ff20d99b700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 0x7ff2080753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20d19a700 1 -- 172.21.15.7:0/2256878659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 msgr2=0x7ff2080753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20d19a700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 0x7ff2080753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20d19a700 1 -- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 msgr2=0x7ff208073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20d19a700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 0x7ff208073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20d19a700 1 -- 172.21.15.7:0/2256878659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040095f0 con 0x7ff2081027a0 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20e19c700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 0x7ff208073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20d19a700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2081027a0 0x7ff208073a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800e9f0 tx=0x7ff1f800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800c9c0 con 0x7ff2081027a0 2024-01-11T17:57:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2081a1810 con 0x7ff2081027a0 2024-01-11T17:57:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1f8010430 con 0x7ff2081027a0 2024-01-11T17:57:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8018680 con 0x7ff2081027a0 2024-01-11T17:57:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.891+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2081a1d70 con 0x7ff2081027a0 2024-01-11T17:57:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.892+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff208108a30 con 0x7ff2081027a0 2024-01-11T17:57:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.895+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7ff1f8010860 con 0x7ff2081027a0 2024-01-11T17:57:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.896+0000 7ff1feffd700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff1f40a1f70 0x7ff1f40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:47.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.896+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff1f8014070 con 0x7ff2081027a0 2024-01-11T17:57:47.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.896+0000 7ff20d99b700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff1f40a1f70 0x7ff1f40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:47.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.897+0000 7ff20d99b700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff1f40a1f70 0x7ff1f40a4430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff204000c00 tx=0x7ff204005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:47.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:47.898+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7ff1f8096b50 con 0x7ff2081027a0 2024-01-11T17:57:48.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.108+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff20810ce20 con 0x7ff2081027a0 2024-01-11T17:57:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.109+0000 7ff1feffd700 1 -- 172.21.15.7:0/2256878659 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1f809a170 con 0x7ff2081027a0 2024-01-11T17:57:48.110 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:57:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff1f40a1f70 msgr2=0x7ff1f40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff1f40a1f70 0x7ff1f40a4430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff204000c00 tx=0x7ff204005fb0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2081027a0 msgr2=0x7ff208073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2081027a0 0x7ff208073a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800e9f0 tx=0x7ff1f800ed00 comp rx=0 tx=0).stop 2024-01-11T17:57:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 shutdown_connections 2024-01-11T17:57:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff1f40a1f70 0x7ff1f40a4430 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2081015b0 0x7ff2080753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff208103140 0x7ff208073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 --2- 172.21.15.7:0/2256878659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2081027a0 0x7ff208073a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 >> 172.21.15.7:0/2256878659 conn(0x7ff2080fcb00 msgr2=0x7ff208105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 shutdown_connections 2024-01-11T17:57:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.111+0000 7ff20fbff700 1 -- 172.21.15.7:0/2256878659 wait complete. 2024-01-11T17:57:48.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:57:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 -- 172.21.15.7:0/88856108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc103140 msgr2=0x7f57fc1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 --2- 172.21.15.7:0/88856108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc103140 0x7f57fc1099f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b0d0 tx=0x7f57f400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 -- 172.21.15.7:0/88856108 shutdown_connections 2024-01-11T17:57:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 --2- 172.21.15.7:0/88856108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1015b0 0x7f57fc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 --2- 172.21.15.7:0/88856108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc103140 0x7f57fc1099f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 --2- 172.21.15.7:0/88856108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57fc1027a0 0x7f57fc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 -- 172.21.15.7:0/88856108 >> 172.21.15.7:0/88856108 conn(0x7f57fc0fcb00 msgr2=0x7f57fc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 -- 172.21.15.7:0/88856108 shutdown_connections 2024-01-11T17:57:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.338+0000 7f5803fff700 1 -- 172.21.15.7:0/88856108 wait complete. 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.339+0000 7f5803fff700 1 Processor -- start 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.339+0000 7f5803fff700 1 -- start start 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5803fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1015b0 0x7f57fc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5803fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 0x7f57fc199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5803fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57fc103140 0x7f57fc1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5803fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1a13c0 con 0x7f57fc1027a0 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5803fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1a1500 con 0x7f57fc103140 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5803fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1a1690 con 0x7f57fc1015b0 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 0x7f57fc199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 0x7f57fc199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44702/0 (socket says 172.21.15.7:44702) 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 -- 172.21.15.7:0/3967520508 learned_addr learned my addr 172.21.15.7:0/3967520508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f5801d9b700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1015b0 0x7f57fc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580259c700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57fc103140 0x7f57fc1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 -- 172.21.15.7:0/3967520508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1015b0 msgr2=0x7f57fc199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1015b0 0x7f57fc199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 -- 172.21.15.7:0/3967520508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57fc103140 msgr2=0x7f57fc1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57fc103140 0x7f57fc1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.340+0000 7f580159a700 1 -- 172.21.15.7:0/3967520508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009520 con 0x7f57fc1027a0 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.341+0000 7f580159a700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 0x7f57fc199dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9a0 tx=0x7f57ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.341+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec009790 con 0x7f57fc1027a0 2024-01-11T17:57:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.341+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57ec00cbd0 con 0x7f57fc1027a0 2024-01-11T17:57:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.341+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f4009d00 con 0x7f57fc1027a0 2024-01-11T17:57:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.341+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec018680 con 0x7f57fc1027a0 2024-01-11T17:57:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.341+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc1a1cf0 con 0x7f57fc1027a0 2024-01-11T17:57:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.342+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc0683f0 con 0x7f57fc1027a0 2024-01-11T17:57:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.344+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f57ec00cd40 con 0x7f57fc1027a0 2024-01-11T17:57:48.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.346+0000 7f57f2ffd700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57e80a1f70 0x7f57e80a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.346+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f57ec014070 con 0x7f57fc1027a0 2024-01-11T17:57:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.346+0000 7f5801d9b700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57e80a1f70 0x7f57e80a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.347+0000 7f5801d9b700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57e80a1f70 0x7f57e80a4430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57fc102600 tx=0x7f57f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:48.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.348+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f57ec096f60 con 0x7f57fc1027a0 2024-01-11T17:57:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.559+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57fc10cfb0 con 0x7f57fc1027a0 2024-01-11T17:57:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.560+0000 7f57f2ffd700 1 -- 172.21.15.7:0/3967520508 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57ec09a580 con 0x7f57fc1027a0 2024-01-11T17:57:48.561 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57e80a1f70 msgr2=0x7f57e80a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57e80a1f70 0x7f57e80a4430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57fc102600 tx=0x7f57f8009260 comp rx=0 tx=0).stop 2024-01-11T17:57:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 msgr2=0x7f57fc199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 0x7f57fc199dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9a0 tx=0x7f57ec00ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 shutdown_connections 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57e80a1f70 0x7f57e80a4430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1015b0 0x7f57fc199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57fc103140 0x7f57fc1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 --2- 172.21.15.7:0/3967520508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc1027a0 0x7f57fc199dd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 >> 172.21.15.7:0/3967520508 conn(0x7f57fc0fcb00 msgr2=0x7f57fc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 shutdown_connections 2024-01-11T17:57:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.562+0000 7f5803fff700 1 -- 172.21.15.7:0/3967520508 wait complete. 2024-01-11T17:57:48.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:57:48.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:48 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/893786234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:48.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:48 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2256878659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:48.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:48 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/893786234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:48.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:48 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2256878659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:48 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/893786234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:48.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:48 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2256878659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 -- 172.21.15.7:0/138853229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040075650 msgr2=0x7fb040073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 --2- 172.21.15.7:0/138853229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040075650 0x7fb040073aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009990 tx=0x7fb03c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 -- 172.21.15.7:0/138853229 shutdown_connections 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 --2- 172.21.15.7:0/138853229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040075650 0x7fb040073aa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 --2- 172.21.15.7:0/138853229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040109470 0x7fb04010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 --2- 172.21.15.7:0/138853229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040074070 0x7fb0400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 -- 172.21.15.7:0/138853229 >> 172.21.15.7:0/138853229 conn(0x7fb0400fcb00 msgr2=0x7fb0400fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 -- 172.21.15.7:0/138853229 shutdown_connections 2024-01-11T17:57:48.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.791+0000 7fb047fff700 1 -- 172.21.15.7:0/138853229 wait complete. 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 Processor -- start 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 -- start start 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 0x7fb0401996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040075650 0x7fb040199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040109470 0x7fb0401a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401a1210 con 0x7fb040074070 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401a1350 con 0x7fb040075650 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.792+0000 7fb047fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401a14e0 con 0x7fb040109470 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb045d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 0x7fb0401996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb045d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 0x7fb0401996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2024-01-11T17:57:48.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb045d9b700 1 -- 172.21.15.7:0/553817192 learned_addr learned my addr 172.21.15.7:0/553817192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040075650 0x7fb040199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04659c700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040109470 0x7fb0401a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 -- 172.21.15.7:0/553817192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040109470 msgr2=0x7fb0401a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040109470 0x7fb0401a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 -- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 msgr2=0x7fb0401996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 0x7fb0401996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 -- 172.21.15.7:0/553817192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0095f0 con 0x7fb040075650 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb045d9b700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 0x7fb0401996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.793+0000 7fb04559a700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040075650 0x7fb040199c20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9c0 tx=0x7fb03000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.794+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030004220 con 0x7fb040075650 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.794+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0401a17c0 con 0x7fb040075650 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.794+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb030004380 con 0x7fb040075650 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.794+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030003be0 con 0x7fb040075650 2024-01-11T17:57:48.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.794+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0401a1d20 con 0x7fb040075650 2024-01-11T17:57:48.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.795+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb040108230 con 0x7fb040075650 2024-01-11T17:57:48.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.797+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fb030003d40 con 0x7fb040075650 2024-01-11T17:57:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.798+0000 7fb036ffd700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb02c0a1f70 0x7fb02c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.798+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb030014070 con 0x7fb040075650 2024-01-11T17:57:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.798+0000 7fb045d9b700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb02c0a1f70 0x7fb02c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.799+0000 7fb045d9b700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb02c0a1f70 0x7fb02c0a4430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00b2e0 tx=0x7fb03c00b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:48.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:48.802+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb030096d30 con 0x7fb040075650 2024-01-11T17:57:49.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.018+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb04010f190 con 0x7fb040075650 2024-01-11T17:57:49.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.018+0000 7fb036ffd700 1 -- 172.21.15.7:0/553817192 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb03009a350 con 0x7fb040075650 2024-01-11T17:57:49.019 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb02c0a1f70 msgr2=0x7fb02c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb02c0a1f70 0x7fb02c0a4430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00b2e0 tx=0x7fb03c00b750 comp rx=0 tx=0).stop 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040075650 msgr2=0x7fb040199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040075650 0x7fb040199c20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9c0 tx=0x7fb03000ecd0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 shutdown_connections 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb02c0a1f70 0x7fb02c0a4430 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb040109470 0x7fb0401a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb040074070 0x7fb0401996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 --2- 172.21.15.7:0/553817192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb040075650 0x7fb040199c20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 >> 172.21.15.7:0/553817192 conn(0x7fb0400fcb00 msgr2=0x7fb040105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:49.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 shutdown_connections 2024-01-11T17:57:49.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.021+0000 7fb047fff700 1 -- 172.21.15.7:0/553817192 wait complete. 2024-01-11T17:57:49.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 -- 172.21.15.7:0/2817331823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc103140 msgr2=0x7f57cc1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 --2- 172.21.15.7:0/2817331823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc103140 0x7f57cc1099f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b0d0 tx=0x7f57c400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 -- 172.21.15.7:0/2817331823 shutdown_connections 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 --2- 172.21.15.7:0/2817331823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 --2- 172.21.15.7:0/2817331823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc103140 0x7f57cc1099f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 --2- 172.21.15.7:0/2817331823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57cc1027a0 0x7f57cc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 -- 172.21.15.7:0/2817331823 >> 172.21.15.7:0/2817331823 conn(0x7f57cc0fcb00 msgr2=0x7f57cc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 -- 172.21.15.7:0/2817331823 shutdown_connections 2024-01-11T17:57:49.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.247+0000 7f57d41ff700 1 -- 172.21.15.7:0/2817331823 wait complete. 2024-01-11T17:57:49.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.248+0000 7f57d41ff700 1 Processor -- start 2024-01-11T17:57:49.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.248+0000 7f57d41ff700 1 -- start start 2024-01-11T17:57:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d41ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d41ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc1027a0 0x7f57cc199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d41ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57cc103140 0x7f57cc1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d41ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a1320 con 0x7f57cc1027a0 2024-01-11T17:57:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d41ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a1460 con 0x7f57cc103140 2024-01-11T17:57:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d41ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a15f0 con 0x7f57cc1015b0 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d179a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc1027a0 0x7f57cc199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d1f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d1f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43574/0 (socket says 172.21.15.7:43574) 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d1f9b700 1 -- 172.21.15.7:0/1725272799 learned_addr learned my addr 172.21.15.7:0/1725272799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.249+0000 7f57d279c700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57cc103140 0x7f57cc1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d1f9b700 1 -- 172.21.15.7:0/1725272799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57cc103140 msgr2=0x7f57cc1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d1f9b700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57cc103140 0x7f57cc1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d1f9b700 1 -- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc1027a0 msgr2=0x7f57cc199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d1f9b700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc1027a0 0x7f57cc199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d1f9b700 1 -- 172.21.15.7:0/1725272799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009d00 con 0x7f57cc1015b0 2024-01-11T17:57:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d1f9b700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1997a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f57c800e970 tx=0x7f57c800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:49.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8009790 con 0x7f57cc1015b0 2024-01-11T17:57:49.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c800c7e0 con 0x7f57cc1015b0 2024-01-11T17:57:49.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc1a18d0 con 0x7f57cc1015b0 2024-01-11T17:57:49.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8003c70 con 0x7f57cc1015b0 2024-01-11T17:57:49.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.250+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc1a1de0 con 0x7f57cc1015b0 2024-01-11T17:57:49.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.251+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc0683f0 con 0x7f57cc1015b0 2024-01-11T17:57:49.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.253+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f57c800cc50 con 0x7f57cc1015b0 2024-01-11T17:57:49.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.255+0000 7f57c2ffd700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57b80a1f70 0x7f57b80a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.255+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f57c8014070 con 0x7f57cc1015b0 2024-01-11T17:57:49.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.255+0000 7f57d179a700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57b80a1f70 0x7f57b80a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.255+0000 7f57d179a700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57b80a1f70 0x7f57b80a4430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009c60 tx=0x7f57bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:49.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.257+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f57c8096d80 con 0x7f57cc1015b0 2024-01-11T17:57:49.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.472+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f57cc1a2000 con 0x7f57cc1015b0 2024-01-11T17:57:49.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.473+0000 7f57c2ffd700 1 -- 172.21.15.7:0/1725272799 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f57c8003dd0 con 0x7f57cc1015b0 2024-01-11T17:57:49.474 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:49.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.475+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57b80a1f70 msgr2=0x7f57b80a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.475+0000 7f57d41ff700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57b80a1f70 0x7f57b80a4430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009c60 tx=0x7f57bc009260 comp rx=0 tx=0).stop 2024-01-11T17:57:49.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.475+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 msgr2=0x7f57cc1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.475+0000 7f57d41ff700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1997a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f57c800e970 tx=0x7f57c800ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 shutdown_connections 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f57b80a1f70 0x7f57b80a4430 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57cc1015b0 0x7f57cc1997a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57cc103140 0x7f57cc1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 --2- 172.21.15.7:0/1725272799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57cc1027a0 0x7f57cc199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 >> 172.21.15.7:0/1725272799 conn(0x7f57cc0fcb00 msgr2=0x7f57cc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:49.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 shutdown_connections 2024-01-11T17:57:49.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.476+0000 7f57d41ff700 1 -- 172.21.15.7:0/1725272799 wait complete. 2024-01-11T17:57:49.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 -- 172.21.15.7:0/2693094717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280101590 msgr2=0x7fa2801019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 --2- 172.21.15.7:0/2693094717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280101590 0x7fa2801019b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa2700099c0 tx=0x7fa270009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 -- 172.21.15.7:0/2693094717 shutdown_connections 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 --2- 172.21.15.7:0/2693094717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa280103120 0x7fa2801099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 --2- 172.21.15.7:0/2693094717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280101590 0x7fa2801019b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 --2- 172.21.15.7:0/2693094717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa280102780 0x7fa280102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 -- 172.21.15.7:0/2693094717 >> 172.21.15.7:0/2693094717 conn(0x7fa2800fcb00 msgr2=0x7fa2800fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 -- 172.21.15.7:0/2693094717 shutdown_connections 2024-01-11T17:57:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.701+0000 7fa284bff700 1 -- 172.21.15.7:0/2693094717 wait complete. 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 Processor -- start 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 -- start start 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa280101590 0x7fa2801996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 0x7fa280199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa280103120 0x7fa2801a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2801a1270 con 0x7fa280102780 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2801a13b0 con 0x7fa280103120 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.702+0000 7fa284bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2801a1540 con 0x7fa280101590 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 0x7fa280199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 0x7fa280199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44766/0 (socket says 172.21.15.7:44766) 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 -- 172.21.15.7:0/1732793606 learned_addr learned my addr 172.21.15.7:0/1732793606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27e59c700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa280101590 0x7fa2801996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27ed9d700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa280103120 0x7fa2801a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 -- 172.21.15.7:0/1732793606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa280101590 msgr2=0x7fa2801996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa280101590 0x7fa2801996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 -- 172.21.15.7:0/1732793606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa280103120 msgr2=0x7fa2801a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa280103120 0x7fa2801a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 -- 172.21.15.7:0/1732793606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700095f0 con 0x7fa280102780 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.703+0000 7fa27dd9b700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 0x7fa280199c30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa27400e9a0 tx=0x7fa27400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.704+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274009790 con 0x7fa280102780 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.704+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2801a1820 con 0x7fa280102780 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.704+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa274010430 con 0x7fa280102780 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.704+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274018680 con 0x7fa280102780 2024-01-11T17:57:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.704+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2801a1d30 con 0x7fa280102780 2024-01-11T17:57:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.705+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2800683f0 con 0x7fa280102780 2024-01-11T17:57:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.707+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fa2740105a0 con 0x7fa280102780 2024-01-11T17:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.709+0000 7fa26f7fe700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2680a1f70 0x7fa2680a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.709+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa274014070 con 0x7fa280102780 2024-01-11T17:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.709+0000 7fa27e59c700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2680a1f70 0x7fa2680a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.710+0000 7fa27e59c700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2680a1f70 0x7fa2680a4430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa270003730 tx=0x7fa270000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.713+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa274052380 con 0x7fa280102780 2024-01-11T17:57:49.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:49 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3967520508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:49.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:49 smithi016 ceph-mon[43195]: pgmap v300: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:49.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:49 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/553817192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:49 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3967520508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:49 smithi038 ceph-mon[43582]: pgmap v300: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:49.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:49 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/553817192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:49.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:49 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3967520508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:49.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:49 smithi007 ceph-mon[33361]: pgmap v300: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-11T17:57:49.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:49 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/553817192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:49.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.976+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa28010d030 con 0x7fa280102780 2024-01-11T17:57:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.976+0000 7fa26f7fe700 1 -- 172.21.15.7:0/1732793606 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11309 (secure 0 0 0) 0x7fa274096c30 con 0x7fa280102780 2024-01-11T17:57:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:49.977 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:56:34.314730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:57:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2680a1f70 msgr2=0x7fa2680a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2680a1f70 0x7fa2680a4430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa270003730 tx=0x7fa270000bc0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 msgr2=0x7fa280199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 0x7fa280199c30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa27400e9a0 tx=0x7fa27400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 shutdown_connections 2024-01-11T17:57:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2680a1f70 0x7fa2680a4430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa280101590 0x7fa2801996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa280103120 0x7fa2801a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 --2- 172.21.15.7:0/1732793606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa280102780 0x7fa280199c30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 >> 172.21.15.7:0/1732793606 conn(0x7fa2800fcb00 msgr2=0x7fa280073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 shutdown_connections 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:49.979+0000 7fa284bff700 1 -- 172.21.15.7:0/1732793606 wait complete. 2024-01-11T17:57:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-01-11T17:57:49.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 -- 172.21.15.7:0/3241285461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c103120 msgr2=0x7f2e1c1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 --2- 172.21.15.7:0/3241285461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c103120 0x7f2e1c1099d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400b0d0 tx=0x7f2e1400b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 -- 172.21.15.7:0/3241285461 shutdown_connections 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 --2- 172.21.15.7:0/3241285461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1c101590 0x7f2e1c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 --2- 172.21.15.7:0/3241285461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c103120 0x7f2e1c1099d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 --2- 172.21.15.7:0/3241285461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c102780 0x7f2e1c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.202+0000 7f2e217ff700 1 -- 172.21.15.7:0/3241285461 >> 172.21.15.7:0/3241285461 conn(0x7f2e1c0fcb00 msgr2=0x7f2e1c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.203+0000 7f2e217ff700 1 -- 172.21.15.7:0/3241285461 shutdown_connections 2024-01-11T17:57:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.203+0000 7f2e217ff700 1 -- 172.21.15.7:0/3241285461 wait complete. 2024-01-11T17:57:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 Processor -- start 2024-01-11T17:57:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 -- start start 2024-01-11T17:57:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1c101590 0x7f2e1c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c102780 0x7f2e1c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 0x7f2e1c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c19f0f0 con 0x7f2e1c102780 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c19f230 con 0x7f2e1c103120 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e217ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c19f3c0 con 0x7f2e1c101590 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 0x7f2e1c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e1a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c102780 0x7f2e1c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 0x7f2e1c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46370/0 (socket says 172.21.15.7:46370) 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e1b7fe700 1 -- 172.21.15.7:0/86198345 learned_addr learned my addr 172.21.15.7:0/86198345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.204+0000 7f2e1affd700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1c101590 0x7f2e1c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e1b7fe700 1 -- 172.21.15.7:0/86198345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1c101590 msgr2=0x7f2e1c197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e1b7fe700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1c101590 0x7f2e1c197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e1b7fe700 1 -- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c102780 msgr2=0x7f2e1c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e1b7fe700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c102780 0x7f2e1c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e1b7fe700 1 -- 172.21.15.7:0/86198345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009520 con 0x7f2e1c103120 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e1b7fe700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 0x7f2e1c19eb20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e14007b60 tx=0x7f2e14008ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400e040 con 0x7f2e1c103120 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14009d00 con 0x7f2e1c103120 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e1401d930 con 0x7f2e1c103120 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1401ba00 con 0x7f2e1c103120 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.205+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c19f980 con 0x7f2e1c103120 2024-01-11T17:57:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.206+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c108990 con 0x7f2e1c103120 2024-01-11T17:57:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.209+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f2e1401daa0 con 0x7f2e1c103120 2024-01-11T17:57:50.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.210+0000 7f2e03fff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1f70 0x7f2e040a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.210+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2e140cf6d0 con 0x7f2e1c103120 2024-01-11T17:57:50.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.211+0000 7f2e1affd700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1f70 0x7f2e040a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.211+0000 7f2e1affd700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1f70 0x7f2e040a4430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c1025e0 tx=0x7f2e0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.212+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2e14098310 con 0x7f2e1c103120 2024-01-11T17:57:50.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.430+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e1c0683f0 con 0x7f2e1c103120 2024-01-11T17:57:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.432+0000 7f2e03fff700 1 -- 172.21.15.7:0/86198345 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7f2e140a0e60 con 0x7f2e1c103120 2024-01-11T17:57:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:50.433 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1f70 msgr2=0x7f2e040a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1f70 0x7f2e040a4430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c1025e0 tx=0x7f2e0c009260 comp rx=0 tx=0).stop 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 msgr2=0x7f2e1c19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 0x7f2e1c19eb20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e14007b60 tx=0x7f2e14008ec0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 shutdown_connections 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2e040a1f70 0x7f2e040a4430 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1c101590 0x7f2e1c197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e1c103120 0x7f2e1c19eb20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 --2- 172.21.15.7:0/86198345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e1c102780 0x7f2e1c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 >> 172.21.15.7:0/86198345 conn(0x7f2e1c0fcb00 msgr2=0x7f2e1c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 shutdown_connections 2024-01-11T17:57:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.434+0000 7f2e217ff700 1 -- 172.21.15.7:0/86198345 wait complete. 2024-01-11T17:57:50.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-01-11T17:56:08.339093+0000', 'modified': '2024-01-11T17:56:34.314730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24602, 'mds_1': 24601, 'mds_2': 24577}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24547': {'gid': 24547, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6835/1587519060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 1587519060}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 1587519060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/2455039829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 2455039829}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 2455039829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24565': {'gid': 24565, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/2512926203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 2512926203}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 2512926203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24577': {'gid': 24577, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.38:6839/443773776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 443773776}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 443773776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24601': {'gid': 24601, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.7:6841/898637958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6840', 'nonce': 898637958}, {'type': 'v1', 'addr': '172.21.15.7:6841', 'nonce': 898637958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/1892769619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 1892769619}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 1892769619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-11T17:57:50.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-01-11T17:57:50.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 -- 172.21.15.7:0/2217206146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 msgr2=0x7f3cd0109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/2217206146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd0109a00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b0d0 tx=0x7f3cc800b3e0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 -- 172.21.15.7:0/2217206146 shutdown_connections 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/2217206146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd01015c0 0x7f3cd01019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/2217206146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd0109a00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/2217206146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd01027b0 0x7f3cd0102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 -- 172.21.15.7:0/2217206146 >> 172.21.15.7:0/2217206146 conn(0x7f3cd00fcb30 msgr2=0x7f3cd00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 -- 172.21.15.7:0/2217206146 shutdown_connections 2024-01-11T17:57:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.659+0000 7f3cd75ff700 1 -- 172.21.15.7:0/2217206146 wait complete. 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 Processor -- start 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 -- start start 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd01015c0 0x7f3cd0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd01027b0 0x7f3cd0199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd01a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01a13e0 con 0x7f3cd0103150 2024-01-11T17:57:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01a1520 con 0x7f3cd01015c0 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.660+0000 7f3cd75ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01a16b0 con 0x7f3cd01027b0 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd01a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd01a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44800/0 (socket says 172.21.15.7:44800) 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/1643006810 learned_addr learned my addr 172.21.15.7:0/1643006810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd01027b0 0x7f3cd0199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd539b700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd01015c0 0x7f3cd0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/1643006810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd01027b0 msgr2=0x7f3cd0199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd01027b0 0x7f3cd0199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/1643006810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd01015c0 msgr2=0x7f3cd0199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd01015c0 0x7f3cd0199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/1643006810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc8009d00 con 0x7f3cd0103150 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.661+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd01a0e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8007b60 tx=0x7f3cc8008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.662+0000 7f3cc67fc700 1 -- 172.21.15.7:0/1643006810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800e040 con 0x7f3cd0103150 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.662+0000 7f3cc67fc700 1 -- 172.21.15.7:0/1643006810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc801d930 con 0x7f3cd0103150 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.662+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd01a1930 con 0x7f3cd0103150 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.662+0000 7f3cc67fc700 1 -- 172.21.15.7:0/1643006810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc801b9f0 con 0x7f3cd0103150 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.662+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd01a1e00 con 0x7f3cd0103150 2024-01-11T17:57:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.665+0000 7f3cc67fc700 1 -- 172.21.15.7:0/1643006810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f3cc801daa0 con 0x7f3cd0103150 2024-01-11T17:57:50.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3ceb3c0b80 1 Processor -- start 2024-01-11T17:57:50.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3cc67fc700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3cbc0a1ff0 0x7f3cbc0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3ceb3c0b80 1 -- start start 2024-01-11T17:57:50.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3cc67fc700 1 -- 172.21.15.7:0/1643006810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3cc80cf1d0 con 0x7f3cd0103150 2024-01-11T17:57:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x55824097d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097dd30 0x55824097bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.666+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3cbc0a1ff0 0x7f3cbc0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 0x55824097c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558240873040 con 0x55824097dd30 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582408d6ad0 con 0x55824097d3b0 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582408d43f0 con 0x55824097c520 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 0x55824097c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd4b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097dd30 0x55824097bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 0x55824097c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43644/0 (socket says 172.21.15.7:43644) 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/959293329 learned_addr learned my addr 172.21.15.7:0/959293329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd539b700 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x55824097d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/959293329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 msgr2=0x55824097d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x55824097d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/959293329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097dd30 msgr2=0x55824097bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097dd30 0x55824097bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd5b9c700 1 -- 172.21.15.7:0/959293329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558240744360 con 0x55824097c520 2024-01-11T17:57:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.667+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3cbc0a1ff0 0x7f3cbc0a44b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00134b0 tx=0x7f3cc0013870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.668+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 0x55824097c9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8098030 tx=0x7f3cc8097fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f2971e5d7445229 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.668+0000 7f3cbb7fe700 1 -- 172.21.15.7:0/959293329 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc809bb30 con 0x55824097c520 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.668+0000 7f3cbb7fe700 1 -- 172.21.15.7:0/959293329 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc809fc80 con 0x55824097c520 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.668+0000 7f3cbb7fe700 1 -- 172.21.15.7:0/959293329 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc809b020 con 0x55824097c520 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.668+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/959293329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 msgr2=0x55824097c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.668+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 0x55824097c9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8098030 tx=0x7f3cc8097fa0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/959293329 shutdown_connections 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097c520 0x55824097c9a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x55824097d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/959293329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097dd30 0x55824097bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/959293329 >> 172.21.15.7:0/959293329 conn(0x5582408f1750 msgr2=0x558240976d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/959293329 shutdown_connections 2024-01-11T17:57:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/959293329 wait complete. 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 -- start start 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.669+0000 7f3ceb3c0b80 1 Processor -- start 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x55824098e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 0x55824098e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097dd30 0x558240995960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582408cfce0 con 0x55824097c520 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824083e350 con 0x55824097dd30 2024-01-11T17:57:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824083dff0 con 0x55824097d3b0 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3ceb3c0b80 10 client.0 _ll_register_callbacks cb 0x5582408706d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x55824098e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd5b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097dd30 0x558240995960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd4b9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 0x55824098e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x55824098e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44820/0 (socket says 172.21.15.7:44820) 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd539b700 1 -- 172.21.15.7:0/2440092681 learned_addr learned my addr 172.21.15.7:0/2440092681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd4b9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 0x55824098e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43646/0 (socket says 172.21.15.7:43646) 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.670+0000 7f3cd5b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097dd30 0x558240995960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46406/0 (socket says 172.21.15.7:46406) 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/2440092681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097dd30 msgr2=0x558240995960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097dd30 0x558240995960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/2440092681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 msgr2=0x55824098e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x55824098e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/2440092681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd01a1e00 con 0x55824097d3b0 2024-01-11T17:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 0x55824098e920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0013c70 tx=0x7f3cc001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf2cc827cd84158 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/2440092681 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0018040 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55824096efc0 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/2440092681 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3cc0004830 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/2440092681 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc00049f0 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.671+0000 7f3ceb3c0b80 10 client.25354 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5582406f0b10 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/2440092681 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55824096efc0 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/2440092681 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3cc0026070 con 0x55824097d3b0 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 10 client.25354 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 10 client.25354 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 20 client.25354 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x558240999ff0 0x558240998aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 4 client.25354 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.672+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582408c8b70 con 0x558240999ff0 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.673+0000 7f3cd539b700 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x558240999ff0 0x558240998aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.673+0000 7f3cd539b700 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x558240999ff0 0x558240998aa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12a8e7ce537201fc server_cookie=a38b7a74a5fdb81f in_seq=0 out_seq=0 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.673+0000 7f3cb8ff9700 10 client.25354 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/2440092681 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582408c8b70 con 0x558240999ff0 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3cb8ff9700 10 client.25354 handle_command_reply: tid=0 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 client.25354 shutdown 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 msgr2=0x55824098e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 0x55824098e920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0013c70 tx=0x7f3cc001b000 comp rx=0 tx=0).stop 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 shutdown_connections 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x558240999ff0 0x558240998aa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097dd30 0x558240995960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x55824098e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3cb8ff9700 0 client.25354 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2440092681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097d3b0 0x55824098e920 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 >> 172.21.15.7:0/2440092681 conn(0x5582408f1750 msgr2=0x55824097ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 shutdown_connections 2024-01-11T17:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2440092681 wait complete. 2024-01-11T17:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.674+0000 7f3ceb3c0b80 20 client.25354 trim_cache size 0 max 16384 2024-01-11T17:57:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.693+0000 7f3ceb3c0b80 1 Processor -- start 2024-01-11T17:57:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.693+0000 7f3ceb3c0b80 1 -- start start 2024-01-11T17:57:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x558240983e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x558240995920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x55824097be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558240746650 con 0x55824097c520 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824098ce40 con 0x55824097d3b0 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582408c8b70 con 0x55824097dd30 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x558240983e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x558240983e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44834/0 (socket says 172.21.15.7:44834) 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 -- 172.21.15.7:0/2551061852 learned_addr learned my addr 172.21.15.7:0/2551061852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x55824097be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x558240995920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 -- 172.21.15.7:0/2551061852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 msgr2=0x55824097be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x55824097be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 -- 172.21.15.7:0/2551061852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 msgr2=0x558240995920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x558240995920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.694+0000 7f3cd539b700 1 -- 172.21.15.7:0/2551061852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582408911e0 con 0x55824097c520 2024-01-11T17:57:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.695+0000 7f3cd539b700 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x558240983e20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55824099af80 tx=0x7f3ccc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804b5238a0dafcf9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.695+0000 7f3cbaffd700 1 -- 172.21.15.7:0/2551061852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc017040 con 0x55824097c520 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.695+0000 7f3cbaffd700 1 -- 172.21.15.7:0/2551061852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ccc018040 con 0x55824097c520 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.695+0000 7f3cbaffd700 1 -- 172.21.15.7:0/2551061852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc01f050 con 0x55824097c520 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.695+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2551061852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 msgr2=0x558240983e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.695+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x558240983e20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55824099af80 tx=0x7f3ccc009260 comp rx=0 tx=0).stop 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2551061852 shutdown_connections 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x55824097be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097c520 0x558240983e20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/2551061852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097d3b0 0x558240995920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2551061852 >> 172.21.15.7:0/2551061852 conn(0x5582408f1750 msgr2=0x558240976ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2551061852 shutdown_connections 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/2551061852 wait complete. 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 -- start start 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.696+0000 7f3ceb3c0b80 1 Processor -- start 2024-01-11T17:57:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097c520 0x558240991e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 0x5582409923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x558240994ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582409953e0 con 0x55824097d3b0 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558240995550 con 0x55824097c520 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558240995710 con 0x55824097dd30 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3ceb3c0b80 10 client.0 _ll_register_callbacks cb 0x5582408706d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 0x5582409923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 0x5582409923c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44844/0 (socket says 172.21.15.7:44844) 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/185298287 learned_addr learned my addr 172.21.15.7:0/185298287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd539b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097c520 0x558240991e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd5b9c700 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x558240994ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 msgr2=0x558240994ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x558240994ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/185298287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097c520 msgr2=0x558240991e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097c520 0x558240991e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.697+0000 7f3cd4b9a700 1 -- 172.21.15.7:0/185298287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc011440 con 0x55824097d3b0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3cd4b9a700 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 0x5582409923c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0004f00 tx=0x7f3cc0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ad39e10ab50ba server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3cb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/185298287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0018070 con 0x55824097d3b0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cc809fc80 con 0x55824097d3b0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/185298287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3cc00105a0 con 0x55824097d3b0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/185298287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc0010ad0 con 0x55824097d3b0 2024-01-11T17:57:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3ceb3c0b80 10 client.15360 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558240748c80 con 0x55824097d3b0 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.698+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/185298287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc809fc80 con 0x55824097d3b0 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/185298287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3cc00036f0 con 0x55824097d3b0 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3ceb3c0b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3ceb3c0b80 10 client.15360 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3ceb3c0b80 20 client.15360 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x5582409f4b70 0x5582409f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3ceb3c0b80 4 client.15360 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5582408c8890 con 0x5582409f4b70 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.699+0000 7f3cd539b700 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x5582409f4b70 0x5582409f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3cb8ff9700 10 client.15360 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3cd539b700 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x5582409f4b70 0x5582409f0fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f85eafa63f44ca88 server_cookie=4f3a903edebf4714 in_seq=0 out_seq=0 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3cb8ff9700 1 -- 172.21.15.7:0/185298287 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5582408c8890 con 0x5582409f4b70 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3cb8ff9700 10 client.15360 handle_command_reply: tid=0 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 client.15360 shutdown 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 msgr2=0x5582409923c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 0x5582409923c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0004f00 tx=0x7f3cc0003f90 comp rx=0 tx=0).stop 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 shutdown_connections 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x5582409f4b70 0x5582409f0fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55824097dd30 0x558240994ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3cb8ff9700 0 client.15360 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824097c520 0x558240991e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 --2- 172.21.15.7:0/185298287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824097d3b0 0x5582409923c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 >> 172.21.15.7:0/185298287 conn(0x5582408f1750 msgr2=0x5582409a2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 shutdown_connections 2024-01-11T17:57:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.700+0000 7f3ceb3c0b80 1 -- 172.21.15.7:0/185298287 wait complete. 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3ceb3c0b80 20 client.15360 trim_cache size 0 max 16384 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24577, 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:57:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:57:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 77.314517576, 2024-01-11T17:57:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-01-11T17:57:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-01-11T17:57:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-01-11T17:57:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 114.967898053 2024-01-11T17:57:50.709 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3cbc0a1ff0 msgr2=0x7f3cbc0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3cbc0a1ff0 0x7f3cbc0a44b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00134b0 tx=0x7f3cc0013870 comp rx=0 tx=0).stop 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 msgr2=0x7f3cd01a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd01a0e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8007b60 tx=0x7f3cc8008e40 comp rx=0 tx=0).stop 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 shutdown_connections 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3cbc0a1ff0 0x7f3cbc0a44b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd0103150 0x7f3cd01a0e10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd01015c0 0x7f3cd0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 --2- 172.21.15.7:0/1643006810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd01027b0 0x7f3cd0199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 >> 172.21.15.7:0/1643006810 conn(0x7f3cd00fcb30 msgr2=0x7f3cd0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 shutdown_connections 2024-01-11T17:57:50.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.701+0000 7f3cd75ff700 1 -- 172.21.15.7:0/1643006810 wait complete. 2024-01-11T17:57:50.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:50 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1725272799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:50.718 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:50 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1732793606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:50.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-11T17:57:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:50 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1725272799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:50 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1732793606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:50 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1725272799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:57:50.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:50 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1732793606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:57:50.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 -- 172.21.15.7:0/373410341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 msgr2=0x7fadd4102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 --2- 172.21.15.7:0/373410341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd4102c00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fadc80099c0 tx=0x7fadc8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 -- 172.21.15.7:0/373410341 shutdown_connections 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 --2- 172.21.15.7:0/373410341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd4103140 0x7fadd41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 --2- 172.21.15.7:0/373410341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd41015b0 0x7fadd41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 --2- 172.21.15.7:0/373410341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd4102c00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 -- 172.21.15.7:0/373410341 >> 172.21.15.7:0/373410341 conn(0x7fadd40fcb00 msgr2=0x7fadd40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.926+0000 7fadd8fff700 1 -- 172.21.15.7:0/373410341 shutdown_connections 2024-01-11T17:57:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.927+0000 7fadd8fff700 1 -- 172.21.15.7:0/373410341 wait complete. 2024-01-11T17:57:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.927+0000 7fadd8fff700 1 Processor -- start 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 -- start start 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd41015b0 0x7fadd4197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4103140 0x7fadd419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd419f010 con 0x7fadd41027a0 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd419f150 con 0x7fadd4103140 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd8fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd419f2e0 con 0x7fadd41015b0 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd41015b0 0x7fadd4197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd41015b0 0x7fadd4197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43668/0 (socket says 172.21.15.7:43668) 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd259c700 1 -- 172.21.15.7:0/3148659874 learned_addr learned my addr 172.21.15.7:0/3148659874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.928+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4103140 0x7fadd419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd1d9b700 1 -- 172.21.15.7:0/3148659874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd41015b0 msgr2=0x7fadd4197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd41015b0 0x7fadd4197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd1d9b700 1 -- 172.21.15.7:0/3148659874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4103140 msgr2=0x7fadd419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4103140 0x7fadd419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd1d9b700 1 -- 172.21.15.7:0/3148659874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc009c40 con 0x7fadd41027a0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd41979d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fadc8003730 tx=0x7fadc800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadc37fe700 1 -- 172.21.15.7:0/3148659874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc801d070 con 0x7fadd41027a0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadc37fe700 1 -- 172.21.15.7:0/3148659874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc80059a0 con 0x7fadd41027a0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc80095f0 con 0x7fadd41027a0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadc37fe700 1 -- 172.21.15.7:0/3148659874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8004970 con 0x7fadd41027a0 2024-01-11T17:57:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.929+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd419f8d0 con 0x7fadd41027a0 2024-01-11T17:57:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.932+0000 7fadc37fe700 1 -- 172.21.15.7:0/3148659874 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fadc8004ad0 con 0x7fadd41027a0 2024-01-11T17:57:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.933+0000 7fadecdfab80 1 Processor -- start 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 -- start start 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadc37fe700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fadbc0a1ff0 0x7fadbc0a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadc37fe700 1 -- 172.21.15.7:0/3148659874 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fadc80ce970 con 0x7fadd41027a0 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c099857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09985d30 0x555c09983f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c099849a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0987b040 con 0x555c09985d30 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c098dead0 con 0x555c099853b0 2024-01-11T17:57:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadecdfab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c098dc3f0 con 0x555c09984520 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fadbc0a1ff0 0x7fadbc0a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadd259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c099857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.934+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09985d30 0x555c09983f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c099857d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46434/0 (socket says 172.21.15.7:46434) 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd259c700 1 -- 172.21.15.7:0/1837485789 learned_addr learned my addr 172.21.15.7:0/1837485789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c099849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd1d9b700 1 -- 172.21.15.7:0/1837485789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 msgr2=0x555c099849a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c099849a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fadbc0a1ff0 0x7fadbc0a44b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fd0 tx=0x7fadcc009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd1d9b700 1 -- 172.21.15.7:0/1837485789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 msgr2=0x555c099857d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c099857d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd1d9b700 1 -- 172.21.15.7:0/1837485789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0974c360 con 0x555c09985d30 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd259c700 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c099857d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09985d30 0x555c09983f40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fadc80cbf90 tx=0x7fadc809b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7115bda2d08da2b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadc0ff9700 1 -- 172.21.15.7:0/1837485789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc809acf0 con 0x555c09985d30 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadc0ff9700 1 -- 172.21.15.7:0/1837485789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc809ba90 con 0x555c09985d30 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.935+0000 7fadc0ff9700 1 -- 172.21.15.7:0/1837485789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80cc1e0 con 0x555c09985d30 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 -- 172.21.15.7:0/1837485789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09985d30 msgr2=0x555c09983f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09985d30 0x555c09983f40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fadc80cbf90 tx=0x7fadc809b1e0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 -- 172.21.15.7:0/1837485789 shutdown_connections 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c099849a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c099857d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 --2- 172.21.15.7:0/1837485789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09985d30 0x555c09983f40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 -- 172.21.15.7:0/1837485789 >> 172.21.15.7:0/1837485789 conn(0x555c098f9750 msgr2=0x555c0997ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 -- 172.21.15.7:0/1837485789 shutdown_connections 2024-01-11T17:57:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.936+0000 7fadecdfab80 1 -- 172.21.15.7:0/1837485789 wait complete. 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.937+0000 7fadecdfab80 1 -- start start 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.937+0000 7fadecdfab80 1 Processor -- start 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.937+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c0998d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.937+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c0998df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c09994f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadecdfab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c098d7ce0 con 0x555c09984520 2024-01-11T17:57:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadecdfab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c09846350 con 0x555c099853b0 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadecdfab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c09845ff0 con 0x555c09985d30 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadecdfab80 10 client.0 _ll_register_callbacks cb 0x555c098786d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c0998d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c09994f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c0998d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44874/0 (socket says 172.21.15.7:44874) 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd259c700 1 -- 172.21.15.7:0/281124411 learned_addr learned my addr 172.21.15.7:0/281124411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c0998df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd2d9d700 1 -- 172.21.15.7:0/281124411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 msgr2=0x555c0998df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c0998df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd2d9d700 1 -- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 msgr2=0x555c0998d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c0998d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd2d9d700 1 -- 172.21.15.7:0/281124411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc4009a50 con 0x555c09985d30 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.938+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c0998df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c09994f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fadcc019f20 tx=0x7fadcc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d209b2c2ad4a9ba8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadb9ffb700 1 -- 172.21.15.7:0/281124411 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00e040 con 0x555c09985d30 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c09976fc0 con 0x555c09985d30 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadb9ffb700 1 -- 172.21.15.7:0/281124411 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fadcc017dc0 con 0x555c09985d30 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadb9ffb700 1 -- 172.21.15.7:0/281124411 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadcc023480 con 0x555c09985d30 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadecdfab80 10 client.25372 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7fadd419f8d0 con 0x555c09985d30 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.939+0000 7fadb9ffb700 1 -- 172.21.15.7:0/281124411 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c09976fc0 con 0x555c09985d30 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadb9ffb700 1 -- 172.21.15.7:0/281124411 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadcc0305d0 con 0x555c09985d30 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadecdfab80 10 client.25372 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadecdfab80 10 client.25372 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadecdfab80 20 client.25372 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadecdfab80 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c09999630 0x555c099980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadecdfab80 4 client.25372 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c098d0b70 con 0x555c09999630 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.940+0000 7fadd259c700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c09999630 0x555c099980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.941+0000 7fadd259c700 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c09999630 0x555c099980b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b9a89f92f46fd00 server_cookie=76997650cfe96850 in_seq=0 out_seq=0 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.941+0000 7fadb9ffb700 10 client.25372 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.941+0000 7fadb9ffb700 1 -- 172.21.15.7:0/281124411 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c098d0b70 con 0x555c09999630 2024-01-11T17:57:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.941+0000 7fadb9ffb700 10 client.25372 handle_command_reply: tid=0 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.941+0000 7fadecdfab80 1 client.25372 shutdown 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 msgr2=0x555c09994f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c09994f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fadcc019f20 tx=0x7fadcc004b40 comp rx=0 tx=0).stop 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 shutdown_connections 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c09999630 0x555c099980b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c09994f70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c0998d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadb9ffb700 0 client.25372 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 --2- 172.21.15.7:0/281124411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c0998df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 >> 172.21.15.7:0/281124411 conn(0x555c098f9750 msgr2=0x555c09982e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 shutdown_connections 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 1 -- 172.21.15.7:0/281124411 wait complete. 2024-01-11T17:57:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.942+0000 7fadecdfab80 20 client.25372 trim_cache size 0 max 16384 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 Processor -- start 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 -- start start 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c09983e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c09994f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c0998d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0974e650 con 0x555c09984520 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0997f360 con 0x555c099853b0 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadecdfab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0997b7c0 con 0x555c09985d30 2024-01-11T17:57:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadd2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c0998d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadd259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c09983e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadd2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c0998d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43682/0 (socket says 172.21.15.7:43682) 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadd2d9d700 1 -- 172.21.15.7:0/2995607884 learned_addr learned my addr 172.21.15.7:0/2995607884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.961+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c09994f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadd2d9d700 1 -- 172.21.15.7:0/2995607884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 msgr2=0x555c09994f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c09994f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadd2d9d700 1 -- 172.21.15.7:0/2995607884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 msgr2=0x555c09983e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c09983e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadd2d9d700 1 -- 172.21.15.7:0/2995607884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c096f8b10 con 0x555c09985d30 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c0998d7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009f10 tx=0x7fadcc00f1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf43f4b9afff863 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadbbfff700 1 -- 172.21.15.7:0/2995607884 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00e070 con 0x555c09985d30 2024-01-11T17:57:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadbbfff700 1 -- 172.21.15.7:0/2995607884 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadcc00f040 con 0x555c09985d30 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.962+0000 7fadbbfff700 1 -- 172.21.15.7:0/2995607884 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc023ba0 con 0x555c09985d30 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 -- 172.21.15.7:0/2995607884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 msgr2=0x555c0998d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c0998d7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009f10 tx=0x7fadcc00f1b0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 -- 172.21.15.7:0/2995607884 shutdown_connections 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09985d30 0x555c0998d7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c09984520 0x555c09983e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 --2- 172.21.15.7:0/2995607884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c099853b0 0x555c09994f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 -- 172.21.15.7:0/2995607884 >> 172.21.15.7:0/2995607884 conn(0x555c098f9750 msgr2=0x555c099889c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 -- 172.21.15.7:0/2995607884 shutdown_connections 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 -- 172.21.15.7:0/2995607884 wait complete. 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 -- start start 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.963+0000 7fadecdfab80 1 Processor -- start 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c09993ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c099853b0 0x555c09994410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c09985d30 0x555c09994950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0998d570 con 0x555c099853b0 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c09994e90 con 0x555c09985d30 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c09995050 con 0x555c09984520 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadecdfab80 10 client.0 _ll_register_callbacks cb 0x555c098786d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c099853b0 0x555c09994410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c09993ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c09993ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43696/0 (socket says 172.21.15.7:43696) 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 -- 172.21.15.7:0/151536050 learned_addr learned my addr 172.21.15.7:0/151536050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd2d9d700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c09985d30 0x555c09994950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 -- 172.21.15.7:0/151536050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c09985d30 msgr2=0x555c09994950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c09985d30 0x555c09994950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 -- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c099853b0 msgr2=0x555c09994410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c099853b0 0x555c09994410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd259c700 1 -- 172.21.15.7:0/151536050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd419f8d0 con 0x555c09984520 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.964+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c099853b0 0x555c09994410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadd259c700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c09993ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadc40096b0 tx=0x7fadc4012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be337cb753bf3a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadb9ffb700 1 -- 172.21.15.7:0/151536050 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4017040 con 0x555c09984520 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc809ba90 con 0x555c09984520 2024-01-11T17:57:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadb9ffb700 1 -- 172.21.15.7:0/151536050 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fadc4012e20 con 0x555c09984520 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadb9ffb700 1 -- 172.21.15.7:0/151536050 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc401f810 con 0x555c09984520 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadecdfab80 10 client.25378 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.965+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c098991e0 con 0x555c09984520 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadb9ffb700 1 -- 172.21.15.7:0/151536050 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc809ba90 con 0x555c09984520 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadb9ffb700 1 -- 172.21.15.7:0/151536050 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadc40275b0 con 0x555c09984520 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadecdfab80 10 client.25378 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadecdfab80 10 client.25378 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadecdfab80 20 client.25378 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadecdfab80 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c099f89e0 0x555c099f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadecdfab80 4 client.25378 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c098d0890 con 0x555c099f89e0 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.966+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c099f89e0 0x555c099f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadd1d9b700 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c099f89e0 0x555c099f4e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21931c4efc034193 server_cookie=a2e32f1056c37ee5 in_seq=0 out_seq=0 2024-01-11T17:57:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadb9ffb700 10 client.25378 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadb9ffb700 1 -- 172.21.15.7:0/151536050 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555c098d0890 con 0x555c099f89e0 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadb9ffb700 10 client.25378 handle_command_reply: tid=0 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 client.25378 shutdown 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 msgr2=0x555c09993ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c09993ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadc40096b0 tx=0x7fadc4012990 comp rx=0 tx=0).stop 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 shutdown_connections 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x555c099f89e0 0x555c099f4e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555c09984520 0x555c09993ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c09985d30 0x555c09994950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadb9ffb700 0 client.25378 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 --2- 172.21.15.7:0/151536050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c099853b0 0x555c09994410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 >> 172.21.15.7:0/151536050 conn(0x555c098f9750 msgr2=0x555c0998d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.967+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 shutdown_connections 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadecdfab80 1 -- 172.21.15.7:0/151536050 wait complete. 2024-01-11T17:57:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadecdfab80 20 client.25378 trim_cache size 0 max 16384 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24601, 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 78.590974732999996, 2024-01-11T17:57:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-01-11T17:57:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-01-11T17:57:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-01-11T17:57:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 109.662204582 2024-01-11T17:57:50.977 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fadbc0a1ff0 msgr2=0x7fadbc0a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fadbc0a1ff0 0x7fadbc0a44b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fd0 tx=0x7fadcc009920 comp rx=0 tx=0).stop 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 msgr2=0x7fadd41979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd41979d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fadc8003730 tx=0x7fadc800f690 comp rx=0 tx=0).stop 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 shutdown_connections 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fadbc0a1ff0 0x7fadbc0a44b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadd41015b0 0x7fadd4197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4103140 0x7fadd419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 --2- 172.21.15.7:0/3148659874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadd41027a0 0x7fadd41979d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 >> 172.21.15.7:0/3148659874 conn(0x7fadd40fcb00 msgr2=0x7fadd4105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 shutdown_connections 2024-01-11T17:57:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:50.968+0000 7fadd8fff700 1 -- 172.21.15.7:0/3148659874 wait complete. 2024-01-11T17:57:50.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-11T17:57:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.191+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/3092990733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58074070 msgr2=0x7f1c580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.191+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/3092990733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58074070 0x7f1c580744f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009990 tx=0x7f1c40009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/3092990733 shutdown_connections 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/3092990733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c58109470 0x7f1c5810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/3092990733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58075650 0x7f1c58073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/3092990733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58074070 0x7f1c580744f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/3092990733 >> 172.21.15.7:0/3092990733 conn(0x7f1c580fcb00 msgr2=0x7f1c580fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/3092990733 shutdown_connections 2024-01-11T17:57:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.192+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/3092990733 wait complete. 2024-01-11T17:57:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 Processor -- start 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 -- start start 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 0x7f1c58199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c58075650 0x7f1c58199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58109470 0x7f1c581a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581a1250 con 0x7f1c58109470 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581a1390 con 0x7f1c58074070 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.193+0000 7f1c5d3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581a1520 con 0x7f1c58075650 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 0x7f1c58199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 0x7f1c58199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46482/0 (socket says 172.21.15.7:46482) 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1715117395 learned_addr learned my addr 172.21.15.7:0/1715117395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c577fe700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58109470 0x7f1c581a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c4ffff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c58075650 0x7f1c58199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1715117395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c58075650 msgr2=0x7f1c58199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c58075650 0x7f1c58199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58109470 msgr2=0x7f1c581a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58109470 0x7f1c581a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1715117395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400095f0 con 0x7f1c58074070 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.194+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 0x7f1c58199720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800e9a0 tx=0x7f1c4800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.195+0000 7f1c54ff9700 1 -- 172.21.15.7:0/1715117395 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48009790 con 0x7f1c58074070 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.195+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c581a1800 con 0x7f1c58074070 2024-01-11T17:57:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.195+0000 7f1c54ff9700 1 -- 172.21.15.7:0/1715117395 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c48010430 con 0x7f1c58074070 2024-01-11T17:57:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.195+0000 7f1c54ff9700 1 -- 172.21.15.7:0/1715117395 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48018680 con 0x7f1c58074070 2024-01-11T17:57:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.195+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c581a1d60 con 0x7f1c58074070 2024-01-11T17:57:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.198+0000 7f1c54ff9700 1 -- 172.21.15.7:0/1715117395 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7f1c480105a0 con 0x7f1c58074070 2024-01-11T17:57:51.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.199+0000 7f1c71102b80 1 Processor -- start 2024-01-11T17:57:51.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.199+0000 7f1c71102b80 1 -- start start 2024-01-11T17:57:51.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.199+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fb3b0 0x55877c1fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.199+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fbd30 0x55877c1f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.199+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c1fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c71102b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c0f1040 con 0x55877c1fa520 2024-01-11T17:57:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c71102b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c154ad0 con 0x55877c1fbd30 2024-01-11T17:57:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c71102b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c1523f0 con 0x55877c1fb3b0 2024-01-11T17:57:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c54ff9700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1c440a1ff0 0x7f1c440a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c1fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c1fa9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44924/0 (socket says 172.21.15.7:44924) 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c4ffff700 1 -- 172.21.15.7:0/1903061071 learned_addr learned my addr 172.21.15.7:0/1903061071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fbd30 0x55877c1f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c54ff9700 1 -- 172.21.15.7:0/1715117395 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1c48014070 con 0x7f1c58074070 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c577fe700 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fb3b0 0x55877c1fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c577fe700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1c440a1ff0 0x7f1c440a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1903061071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fb3b0 msgr2=0x55877c1fb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fb3b0 0x55877c1fb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1903061071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 msgr2=0x55877c1fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c1fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c56ffd700 1 -- 172.21.15.7:0/1903061071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55877bfc2360 con 0x55877c1fbd30 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.200+0000 7f1c4ffff700 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c1fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.201+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fbd30 0x55877c1f9f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c480cb640 tx=0x7f1c4809a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e421a16558d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.201+0000 7f1c577fe700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1c440a1ff0 0x7f1c440a44b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009ec0 tx=0x7f1c5000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.201+0000 7f1c4d7fa700 1 -- 172.21.15.7:0/1903061071 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4809af90 con 0x55877c1fbd30 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.201+0000 7f1c4d7fa700 1 -- 172.21.15.7:0/1903061071 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4809a890 con 0x55877c1fbd30 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.201+0000 7f1c4d7fa700 1 -- 172.21.15.7:0/1903061071 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480cb9b0 con 0x55877c1fbd30 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 -- 172.21.15.7:0/1903061071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fbd30 msgr2=0x55877c1f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fbd30 0x55877c1f9f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c480cb640 tx=0x7f1c4809a450 comp rx=0 tx=0).stop 2024-01-11T17:57:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 -- 172.21.15.7:0/1903061071 shutdown_connections 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fb3b0 0x55877c1fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c1fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 --2- 172.21.15.7:0/1903061071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fbd30 0x55877c1f9f40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 -- 172.21.15.7:0/1903061071 >> 172.21.15.7:0/1903061071 conn(0x55877c16f750 msgr2=0x55877c1f4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 -- 172.21.15.7:0/1903061071 shutdown_connections 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 -- 172.21.15.7:0/1903061071 wait complete. 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 -- start start 2024-01-11T17:57:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.202+0000 7f1c71102b80 1 Processor -- start 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c203e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c204350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c14dce0 con 0x55877c1fa520 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c0bc350 con 0x55877c1fb3b0 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c0bbff0 con 0x55877c1fbd30 2024-01-11T17:57:51.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c71102b80 10 client.0 _ll_register_callbacks cb 0x55877c0ee6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c203e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43742/0 (socket says 172.21.15.7:43742) 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c577fe700 1 -- 172.21.15.7:0/2274827947 learned_addr learned my addr 172.21.15.7:0/2274827947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c203e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44932/0 (socket says 172.21.15.7:44932) 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.203+0000 7f1c4ffff700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c204350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c577fe700 1 -- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 msgr2=0x55877c204350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c577fe700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c204350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c577fe700 1 -- 172.21.15.7:0/2274827947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 msgr2=0x55877c203e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c577fe700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c203e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c577fe700 1 -- 172.21.15.7:0/2274827947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5810c520 con 0x55877c1fbd30 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c203e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c577fe700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20b390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c50018970 tx=0x7f1c50004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c83a3261a8c48b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:51.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c3e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/2274827947 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000e070 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/2274827947 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1c50007710 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55877c1ecfc0 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/2274827947 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c500078d0 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.204+0000 7f1c71102b80 10 client.25390 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f1c581a1d60 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/2274827947 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55877c1ecfc0 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/2274827947 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c5001ac30 con 0x55877c1fbd30 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 10 client.25390 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 10 client.25390 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 20 client.25390 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c20fab0 0x55877c20e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 4 client.25390 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.205+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55877c146b70 con 0x55877c20fab0 2024-01-11T17:57:51.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.206+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c20fab0 0x55877c20e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.206+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c20fab0 0x55877c20e580 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf175b8790c2d5e server_cookie=94fefcd18d18376 in_seq=0 out_seq=0 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.206+0000 7f1c3e7fc700 10 client.25390 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/2274827947 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55877c146b70 con 0x55877c20fab0 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c3e7fc700 10 client.25390 handle_command_reply: tid=0 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 client.25390 shutdown 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 msgr2=0x55877c20b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20b390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c50018970 tx=0x7f1c50004ad0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 shutdown_connections 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c20fab0 0x55877c20e580 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20b390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c203e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c3e7fc700 0 client.25390 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:51.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 --2- 172.21.15.7:0/2274827947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c204350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 >> 172.21.15.7:0/2274827947 conn(0x55877c16f750 msgr2=0x55877c1f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:51.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 shutdown_connections 2024-01-11T17:57:51.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.207+0000 7f1c71102b80 1 -- 172.21.15.7:0/2274827947 wait complete. 2024-01-11T17:57:51.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.208+0000 7f1c71102b80 20 client.25390 trim_cache size 0 max 16384 2024-01-11T17:57:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.226+0000 7f1c71102b80 1 Processor -- start 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 -- start start 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c20ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c20b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c1fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877bfc4650 con 0x55877c1fa520 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c1f1810 con 0x55877c1fb3b0 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c71102b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c146b70 con 0x55877c1fbd30 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c20ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c20ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44946/0 (socket says 172.21.15.7:44946) 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 -- 172.21.15.7:0/3263192513 learned_addr learned my addr 172.21.15.7:0/3263192513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c4ffff700 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c20b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c577fe700 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c1fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 -- 172.21.15.7:0/3263192513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 msgr2=0x55877c1fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c1fa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 -- 172.21.15.7:0/3263192513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 msgr2=0x55877c20b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.227+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c20b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c56ffd700 1 -- 172.21.15.7:0/3263192513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55877bf6eb10 con 0x55877c1fa520 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c577fe700 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c1fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c20ec90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c480cbbc0 tx=0x7f1c48010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bf79801288617d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c4cff9700 1 -- 172.21.15.7:0/3263192513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4809e9e0 con 0x55877c1fa520 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c4cff9700 1 -- 172.21.15.7:0/3263192513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c480850f0 con 0x55877c1fa520 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c4cff9700 1 -- 172.21.15.7:0/3263192513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480ba0b0 con 0x55877c1fa520 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.228+0000 7f1c71102b80 1 -- 172.21.15.7:0/3263192513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 msgr2=0x55877c20ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c20ec90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c480cbbc0 tx=0x7f1c48010c80 comp rx=0 tx=0).stop 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 -- 172.21.15.7:0/3263192513 shutdown_connections 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c1fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fa520 0x55877c20ec90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 --2- 172.21.15.7:0/3263192513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fb3b0 0x55877c20b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 -- 172.21.15.7:0/3263192513 >> 172.21.15.7:0/3263192513 conn(0x55877c16f750 msgr2=0x55877c20e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 -- 172.21.15.7:0/3263192513 shutdown_connections 2024-01-11T17:57:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 -- 172.21.15.7:0/3263192513 wait complete. 2024-01-11T17:57:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 -- start start 2024-01-11T17:57:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.229+0000 7f1c71102b80 1 Processor -- start 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fa520 0x55877c2075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fb3b0 0x55877c207b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c20b060 con 0x55877c1fb3b0 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c20b1d0 con 0x55877c1fa520 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877c20b390 con 0x55877c1fbd30 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c71102b80 10 client.0 _ll_register_callbacks cb 0x55877c0ee6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fb3b0 0x55877c207b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fb3b0 0x55877c207b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44952/0 (socket says 172.21.15.7:44952) 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c4ffff700 1 -- 172.21.15.7:0/4086595211 learned_addr learned my addr 172.21.15.7:0/4086595211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fa520 0x55877c2075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c577fe700 1 -- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fa520 msgr2=0x55877c2075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c577fe700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fa520 0x55877c2075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c577fe700 1 -- 172.21.15.7:0/4086595211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fb3b0 msgr2=0x55877c207b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c577fe700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fb3b0 0x55877c207b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c577fe700 1 -- 172.21.15.7:0/4086595211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55877bfc2360 con 0x55877c1fbd30 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.230+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fa520 0x55877c2075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c577fe700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20ab20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c50018940 tx=0x7f1c50025870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0a6a30595a3505 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c3e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:57:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/4086595211 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000e040 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c4809a890 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/4086595211 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1c50025e00 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/4086595211 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c500084a0 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c71102b80 10 client.25396 fetch_fsmap learned FSMap version 22 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55877c10f1e0 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.231+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/4086595211 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4809a890 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/4086595211 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c5002d070 con 0x55877c1fbd30 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c71102b80 10 client.25396 fetch_fsmap finished waiting for FSMap version 22 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c71102b80 10 client.25396 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c71102b80 20 client.25396 populate_metadata read hostname 'smithi007' 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c71102b80 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c26a910 0x55877c266d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c71102b80 4 client.25396 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55877c146890 con 0x55877c26a910 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.232+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c26a910 0x55877c266d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c56ffd700 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c26a910 0x55877c266d70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadbec6dcd9212c8 server_cookie=dcd027e0d8971d07 in_seq=0 out_seq=0 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c3e7fc700 10 client.25396 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c3e7fc700 1 -- 172.21.15.7:0/4086595211 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55877c146890 con 0x55877c26a910 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c3e7fc700 10 client.25396 handle_command_reply: tid=0 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 client.25396 shutdown 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 msgr2=0x55877c20ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20ab20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c50018940 tx=0x7f1c50025870 comp rx=0 tx=0).stop 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 shutdown_connections 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55877c26a910 0x55877c266d70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55877c1fbd30 0x55877c20ab20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55877c1fa520 0x55877c2075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 --2- 172.21.15.7:0/4086595211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55877c1fb3b0 0x55877c207b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 >> 172.21.15.7:0/4086595211 conn(0x55877c16f750 msgr2=0x55877c203bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c3e7fc700 0 client.25396 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 shutdown_connections 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 1 -- 172.21.15.7:0/4086595211 wait complete. 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.233+0000 7f1c71102b80 20 client.25396 trim_cache size 0 max 16384 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "a76d0c8a-b0a9-11ee-95ab-87774f69a715", 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T17:57:51.240 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24602, 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 101.997247984, 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 107.96978251100001 2024-01-11T17:57:51.241 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1c440a1ff0 msgr2=0x7f1c440a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1c440a1ff0 0x7f1c440a44b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009ec0 tx=0x7f1c5000b040 comp rx=0 tx=0).stop 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 msgr2=0x7f1c58199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 0x7f1c58199720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800e9a0 tx=0x7f1c4800ecb0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 shutdown_connections 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1c440a1ff0 0x7f1c440a44b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c58109470 0x7f1c581a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58074070 0x7f1c58199720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 --2- 172.21.15.7:0/1715117395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c58075650 0x7f1c58199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 >> 172.21.15.7:0/1715117395 conn(0x7f1c580fcb00 msgr2=0x7f1c58105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 shutdown_connections 2024-01-11T17:57:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:51.234+0000 7f1c5d3ff700 1 -- 172.21.15.7:0/1715117395 wait complete. 2024-01-11T17:57:51.253 INFO:teuthology.run_tasks:Running task kclient... 2024-01-11T17:57:51.267 INFO:tasks.kclient:Mounting kernel clients... 2024-01-11T17:57:51.267 INFO:tasks.kclient:config is {'client.0': {}} 2024-01-11T17:57:51.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:51.267 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T17:57:51.311 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-01-11T17:57:51.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:51.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-11T17:57:51.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:51.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T17:57:51.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:51 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/86198345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:51.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:51 smithi007 ceph-mon[33361]: pgmap v301: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:51.549 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-01-11T17:57:51.549 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-11T17:57:51.549 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-01-11T17:57:51.549 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-01-11T17:57:51.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph fs ls 2024-01-11T17:57:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:51 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/86198345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:51 smithi016 ceph-mon[43195]: pgmap v301: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:51 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/86198345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:51 smithi038 ceph-mon[43582]: pgmap v301: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-11T17:57:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T17:57:53.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 -- 172.21.15.7:0/1371059069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf880692c0 msgr2=0x7fdf88069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 --2- 172.21.15.7:0/1371059069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf880692c0 0x7fdf88069740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0099c0 tx=0x7fdf7c009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 -- 172.21.15.7:0/1371059069 shutdown_connections 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 --2- 172.21.15.7:0/1371059069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf8810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 --2- 172.21.15.7:0/1371059069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88107640 0x7fdf88107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 --2- 172.21.15.7:0/1371059069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf880692c0 0x7fdf88069740 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 -- 172.21.15.7:0/1371059069 >> 172.21.15.7:0/1371059069 conn(0x7fdf88077440 msgr2=0x7fdf88077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 -- 172.21.15.7:0/1371059069 shutdown_connections 2024-01-11T17:57:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.203+0000 7fdf8e897700 1 -- 172.21.15.7:0/1371059069 wait complete. 2024-01-11T17:57:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.204+0000 7fdf8e897700 1 Processor -- start 2024-01-11T17:57:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.204+0000 7fdf8e897700 1 -- start start 2024-01-11T17:57:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8e897700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 0x7fdf881005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8e897700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf88100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8e897700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88107640 0x7fdf88101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8e897700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf881a5d90 con 0x7fdf88069c80 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8e897700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf881a5f00 con 0x7fdf880692c0 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8e897700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf881a60c0 con 0x7fdf88107640 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf88100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf87fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 0x7fdf881005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf88100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48010/0 (socket says 172.21.15.7:48010) 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf87fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 0x7fdf881005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49550/0 (socket says 172.21.15.7:49550) 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf877fe700 1 -- 172.21.15.7:0/51784574 learned_addr learned my addr 172.21.15.7:0/51784574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.205+0000 7fdf8ce34700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88107640 0x7fdf88101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf87fff700 1 -- 172.21.15.7:0/51784574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88107640 msgr2=0x7fdf88101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf87fff700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88107640 0x7fdf88101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf87fff700 1 -- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 msgr2=0x7fdf88100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf87fff700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf88100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf87fff700 1 -- 172.21.15.7:0/51784574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c0095f0 con 0x7fdf880692c0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf877fe700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf88100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf87fff700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 0x7fdf881005e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9f0 tx=0x7fdf7800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800c9c0 con 0x7fdf880692c0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf881a63a0 con 0x7fdf880692c0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf78010430 con 0x7fdf880692c0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78018680 con 0x7fdf880692c0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.206+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf881a68c0 con 0x7fdf880692c0 2024-01-11T17:57:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.207+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf88109b80 con 0x7fdf880692c0 2024-01-11T17:57:53.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.210+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 136675+0+0 (secure 0 0 0) 0x7fdf78010860 con 0x7fdf880692c0 2024-01-11T17:57:53.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.211+0000 7fdf857fa700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf700a1f70 0x7fdf700a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:53.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.211+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdf78014070 con 0x7fdf880692c0 2024-01-11T17:57:53.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.211+0000 7fdf877fe700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf700a1f70 0x7fdf700a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:53.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.212+0000 7fdf877fe700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf700a1f70 0x7fdf700a4430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf88102070 tx=0x7fdf7c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:53.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.213+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdf78096cd0 con 0x7fdf880692c0 2024-01-11T17:57:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.474+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf88101b10 con 0x7fdf880692c0 2024-01-11T17:57:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.475+0000 7fdf857fa700 1 -- 172.21.15.7:0/51784574 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdf7809a2f0 con 0x7fdf880692c0 2024-01-11T17:57:53.476 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf700a1f70 msgr2=0x7fdf700a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf700a1f70 0x7fdf700a4430 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf88102070 tx=0x7fdf7c005ab0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 msgr2=0x7fdf881005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 0x7fdf881005e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9f0 tx=0x7fdf7800ed00 comp rx=0 tx=0).stop 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 shutdown_connections 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdf700a1f70 0x7fdf700a4430 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88107640 0x7fdf88101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf880692c0 0x7fdf881005e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 --2- 172.21.15.7:0/51784574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88069c80 0x7fdf88100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.477+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 >> 172.21.15.7:0/51784574 conn(0x7fdf88077440 msgr2=0x7fdf880ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.478+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 shutdown_connections 2024-01-11T17:57:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:53.478+0000 7fdf8e897700 1 -- 172.21.15.7:0/51784574 wait complete. 2024-01-11T17:57:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:53 smithi007 ceph-mon[33361]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:53.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:53 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/51784574' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-11T17:57:53.911 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-11T17:57:53.911 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T17:57:53.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T17:57:53.912 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T17:57:53.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-01-11T17:57:53.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-11T17:57:53.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-11T17:57:53.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T17:57:53.912 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-01-11T17:57:53.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:53.912 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T17:57:53.958 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T17:57:53.959 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41946sec preferred_lft 41946sec 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T17:57:53.960 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T17:57:53.961 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-11T17:57:53.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:53.961 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:57:53.961 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:57:53.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-01-11T17:57:53.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-01-11T17:57:53.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-01-11T17:57:53.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-11T17:57:53.962 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:57:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:53 smithi016 ceph-mon[43195]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:53 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/51784574' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-11T17:57:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:53 smithi038 ceph-mon[43582]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-01-11T17:57:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:53 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/51784574' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-11T17:57:54.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.217 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T17:57:54.289 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T17:57:54.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.293 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T17:57:54.340 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T17:57:54.340 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T17:57:54.340 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T17:57:54.340 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T17:57:54.340 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T17:57:54.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.341 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:57:54.341 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:57:54.341 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T17:57:54.341 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T17:57:54.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T17:57:54.342 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:57:54.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.519 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T17:57:54.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.563 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-01-11T17:57:54.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.672 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:57:54.672 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:57:54.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--tmp-cephfs 2024-01-11T17:57:54.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-01-11T17:57:54.672 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:57:54.818 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-01-11T17:57:54.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:54.818 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:57:54.818 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:57:54.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-01-11T17:57:54.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-11T17:57:54.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-01-11T17:57:54.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-01-11T17:57:54.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-01-11T17:57:54.819 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:57:55.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:55.079 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T17:57:55.079 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T17:57:55.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T17:57:55.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-01-11T17:57:55.080 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T17:57:55.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T17:57:55.255 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /tmp/cephfs 2024-01-11T17:57:55.299 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/tmp/cephfs' 2024-01-11T17:57:55.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T17:57:55.299 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /tmp/cephfs 2024-01-11T17:57:55.388 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-01-11T17:57:55.388 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-01-11T17:57:55.388 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e 2024-01-11T17:57:55.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:55.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context=system_u:object_r:postgresql_db_t:s0 2024-01-11T17:57:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2024-01-11T17:57:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-01-11T17:57:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T17:57:55.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:55 smithi007 ceph-mon[33361]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:55.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:55 smithi016 ceph-mon[43195]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:55.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:55 smithi038 ceph-mon[43582]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-01-11T17:57:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2024-01-11T17:57:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-01-11T17:57:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T17:57:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.16:6789,172.21.15.38:6789" 2024-01-11T17:57:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.16:6789,172.21.15.38:6789:/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e 2024-01-11T17:57:55.998 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=a76d0c8a-b0a9-11ee-95ab-87774f69a715" will pass to kernel 2024-01-11T17:57:56.004 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T17:57:56.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:56.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /tmp/cephfs 2024-01-11T17:57:56.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:56.077 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T17:57:56.165 INFO:teuthology.orchestra.run.smithi007.stdout:client15405 2024-01-11T17:57:56.165 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T17:57:56.165 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-11T17:57:56.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:56.210 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T17:57:56.337 INFO:teuthology.orchestra.run.smithi007.stdout:client15405 2024-01-11T17:57:56.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:56.338 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T17:57:56.425 INFO:teuthology.orchestra.run.smithi007.stdout:client15405 2024-01-11T17:57:56.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T17:57:56.426 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715.client15405/status') 2024-01-11T17:57:56.535 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.15405 (3)192.168.0.1:0/3909329483 2024-01-11T17:57:56.535 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-01-11T17:57:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-01-11T17:57:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-01-11T17:57:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:70 bytes copied, 5.6427e-05 s, 1.2 MB/s 2024-01-11T17:57:56.538 INFO:teuthology.run_tasks:Running task exec... 2024-01-11T17:57:56.548 INFO:teuthology.task.exec:Executing custom commands... 2024-01-11T17:57:56.549 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:57:56.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-01-11T17:57:56.607 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:56 smithi007 ceph-mon[33361]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-01-11T17:57:56.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:56 smithi016 ceph-mon[43195]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-01-11T17:57:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 -- 172.21.15.7:0/337364929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 msgr2=0x7f29141019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 --2- 172.21.15.7:0/337364929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f29141019d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2904009990 tx=0x7f2904009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 -- 172.21.15.7:0/337364929 shutdown_connections 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 --2- 172.21.15.7:0/337364929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f29141019d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 --2- 172.21.15.7:0/337364929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2914103140 0x7f29141099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 --2- 172.21.15.7:0/337364929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29141027a0 0x7f2914102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 -- 172.21.15.7:0/337364929 >> 172.21.15.7:0/337364929 conn(0x7f29140fcb00 msgr2=0x7f29140fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 -- 172.21.15.7:0/337364929 shutdown_connections 2024-01-11T17:57:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.750+0000 7f29189ff700 1 -- 172.21.15.7:0/337364929 wait complete. 2024-01-11T17:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 Processor -- start 2024-01-11T17:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 -- start start 2024-01-11T17:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f2914075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29141027a0 0x7f2914073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2914103140 0x7f2914073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29140746e0 con 0x7f2914103140 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914074850 con 0x7f29141027a0 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.751+0000 7f29189ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a15d0 con 0x7f29141015b0 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f2914075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f2914075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55758/0 (socket says 172.21.15.7:55758) 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 -- 172.21.15.7:0/1304668356 learned_addr learned my addr 172.21.15.7:0/1304668356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f2912d9d700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2914103140 0x7f2914073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f2911d9b700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29141027a0 0x7f2914073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 -- 172.21.15.7:0/1304668356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29141027a0 msgr2=0x7f2914073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29141027a0 0x7f2914073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 -- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2914103140 msgr2=0x7f2914073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2914103140 0x7f2914073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 -- 172.21.15.7:0/1304668356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040095f0 con 0x7f29141015b0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f2911d9b700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29141027a0 0x7f2914073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.752+0000 7f291259c700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f2914075370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f290400b680 tx=0x7f2904004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.753+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290401d070 con 0x7f29141015b0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.753+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29141a1800 con 0x7f29141015b0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.753+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2904021410 con 0x7f29141015b0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.753+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904017650 con 0x7f29141015b0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.753+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29141a1bc0 con 0x7f29141015b0 2024-01-11T17:57:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.754+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291404fb20 con 0x7f29141015b0 2024-01-11T17:57:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.756+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f2904021580 con 0x7f29141015b0 2024-01-11T17:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.757+0000 7f29037fe700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28fc0a1ed0 0x7f28fc0a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.757+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f29040cf810 con 0x7f29141015b0 2024-01-11T17:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.758+0000 7f2911d9b700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28fc0a1ed0 0x7f28fc0a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.758+0000 7f2911d9b700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28fc0a1ed0 0x7f28fc0a4390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2908009cb0 tx=0x7f2908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:56.760+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f29040985b0 con 0x7f29141015b0 2024-01-11T17:57:56.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:56 smithi038 ceph-mon[43582]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-01-11T17:57:57.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.023+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f2914108a80 con 0x7f29141015b0 2024-01-11T17:57:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.363+0000 7f29037fe700 1 -- 172.21.15.7:0/1304668356 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) v1 ==== 117+0+0 (secure 0 0 0) 0x7f290409bbd0 con 0x7f29141015b0 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28fc0a1ed0 msgr2=0x7f28fc0a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28fc0a1ed0 0x7f28fc0a4390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2908009cb0 tx=0x7f2908009260 comp rx=0 tx=0).stop 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 msgr2=0x7f2914075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f2914075370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f290400b680 tx=0x7f2904004730 comp rx=0 tx=0).stop 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 shutdown_connections 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f28fc0a1ed0 0x7f28fc0a4390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141015b0 0x7f2914075370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2914103140 0x7f2914073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 --2- 172.21.15.7:0/1304668356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29141027a0 0x7f2914073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 >> 172.21.15.7:0/1304668356 conn(0x7f29140fcb00 msgr2=0x7f2914105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:57.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 shutdown_connections 2024-01-11T17:57:57.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.366+0000 7f29189ff700 1 -- 172.21.15.7:0/1304668356 wait complete. 2024-01-11T17:57:57.387 INFO:teuthology.run_tasks:Running task check-counter... 2024-01-11T17:57:57.398 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-01-11T17:57:57.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 -- 172.21.15.7:0/4271730091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac101590 msgr2=0x7fa2ac1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/4271730091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac101590 0x7fa2ac1019b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009990 tx=0x7fa29c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 -- 172.21.15.7:0/4271730091 shutdown_connections 2024-01-11T17:57:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/4271730091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac101590 0x7fa2ac1019b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/4271730091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2ac103120 0x7fa2ac1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/4271730091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac102780 0x7fa2ac102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 -- 172.21.15.7:0/4271730091 >> 172.21.15.7:0/4271730091 conn(0x7fa2ac0fcb00 msgr2=0x7fa2ac0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.628+0000 7fa2b15ff700 1 -- 172.21.15.7:0/4271730091 shutdown_connections 2024-01-11T17:57:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.629+0000 7fa2b15ff700 1 -- 172.21.15.7:0/4271730091 wait complete. 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.629+0000 7fa2b15ff700 1 Processor -- start 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 -- start start 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2ac101590 0x7fa2ac1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac102780 0x7fa2ac199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 0x7fa2ac1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1a1220 con 0x7fa2ac103120 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1a1360 con 0x7fa2ac101590 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2b15ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1a14f0 con 0x7fa2ac102780 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 0x7fa2ac1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2aa7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac102780 0x7fa2ac199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2ac101590 0x7fa2ac1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 0x7fa2ac1a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48036/0 (socket says 172.21.15.7:48036) 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.630+0000 7fa2ab7fe700 1 -- 172.21.15.7:0/3019778137 learned_addr learned my addr 172.21.15.7:0/3019778137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2ab7fe700 1 -- 172.21.15.7:0/3019778137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac102780 msgr2=0x7fa2ac199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2ab7fe700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac102780 0x7fa2ac199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2ab7fe700 1 -- 172.21.15.7:0/3019778137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2ac101590 msgr2=0x7fa2ac1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2ab7fe700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2ac101590 0x7fa2ac1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2ab7fe700 1 -- 172.21.15.7:0/3019778137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c0095f0 con 0x7fa2ac103120 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2ab7fe700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 0x7fa2ac1a0c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400ef30 tx=0x7fa2a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4018070 con 0x7fa2ac103120 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac1a1770 con 0x7fa2ac103120 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a400f040 con 0x7fa2ac103120 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cd00 con 0x7fa2ac103120 2024-01-11T17:57:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.631+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac1a1c80 con 0x7fa2ac103120 2024-01-11T17:57:57.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.633+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac0683f0 con 0x7fa2ac103120 2024-01-11T17:57:57.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.635+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa2a4003a00 con 0x7fa2ac103120 2024-01-11T17:57:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.636+0000 7fa293fff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2940a1ee0 0x7fa2940a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.636+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa2a40cd870 con 0x7fa2ac103120 2024-01-11T17:57:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.636+0000 7fa2aaffd700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2940a1ee0 0x7fa2940a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.637+0000 7fa2aaffd700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2940a1ee0 0x7fa2940a43a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00b680 tx=0x7fa29c00b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.640+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fa2a4096610 con 0x7fa2ac103120 2024-01-11T17:57:57.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:57 smithi016 ceph-mon[43195]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-01-11T17:57:57.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:57 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1304668356' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-01-11T17:57:57.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:57 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-01-11T17:57:57.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:57 smithi038 ceph-mon[43582]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-01-11T17:57:57.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:57 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1304668356' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-01-11T17:57:57.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:57 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-01-11T17:57:57.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:57 smithi007 ceph-mon[33361]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-01-11T17:57:57.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:57 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1304668356' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-01-11T17:57:57.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:57 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-01-11T17:57:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.854+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2ac1a2060 con 0x7fa2ac103120 2024-01-11T17:57:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.855+0000 7fa293fff700 1 -- 172.21.15.7:0/3019778137 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fa2a4099c30 con 0x7fa2ac103120 2024-01-11T17:57:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:57:57.856 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2940a1ee0 msgr2=0x7fa2940a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2940a1ee0 0x7fa2940a43a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00b680 tx=0x7fa29c00b750 comp rx=0 tx=0).stop 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 msgr2=0x7fa2ac1a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 0x7fa2ac1a0c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400ef30 tx=0x7fa2a400c3a0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 shutdown_connections 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa2940a1ee0 0x7fa2940a43a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ac103120 0x7fa2ac1a0c50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2ac101590 0x7fa2ac1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 --2- 172.21.15.7:0/3019778137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2ac102780 0x7fa2ac199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 >> 172.21.15.7:0/3019778137 conn(0x7fa2ac0fcb00 msgr2=0x7fa2ac107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 shutdown_connections 2024-01-11T17:57:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:57.858+0000 7fa2b15ff700 1 -- 172.21.15.7:0/3019778137 wait complete. 2024-01-11T17:57:57.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:57:57.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:57:58.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 -- 172.21.15.7:0/2534374322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 msgr2=0x7fc8b81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/2534374322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b81019d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80099c0 tx=0x7fc8a8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 -- 172.21.15.7:0/2534374322 shutdown_connections 2024-01-11T17:57:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/2534374322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8b8103140 0x7fc8b81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/2534374322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b81019d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/2534374322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b81027a0 0x7fc8b8102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 -- 172.21.15.7:0/2534374322 >> 172.21.15.7:0/2534374322 conn(0x7fc8b80fcb00 msgr2=0x7fc8b80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 -- 172.21.15.7:0/2534374322 shutdown_connections 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.087+0000 7fc8bcfff700 1 -- 172.21.15.7:0/2534374322 wait complete. 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.088+0000 7fc8bcfff700 1 Processor -- start 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.088+0000 7fc8bcfff700 1 -- start start 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8bcfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8bcfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8b81027a0 0x7fc8b8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8bcfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b8103140 0x7fc8b81a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8bcfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81a1200 con 0x7fc8b81015b0 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8bcfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81a1340 con 0x7fc8b8103140 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8bcfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81a14d0 con 0x7fc8b81027a0 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8b81027a0 0x7fc8b8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48066/0 (socket says 172.21.15.7:48066) 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 -- 172.21.15.7:0/1725608459 learned_addr learned my addr 172.21.15.7:0/1725608459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b6d9d700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b8103140 0x7fc8b81a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 -- 172.21.15.7:0/1725608459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8b81027a0 msgr2=0x7fc8b8199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8b81027a0 0x7fc8b8199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 -- 172.21.15.7:0/1725608459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b8103140 msgr2=0x7fc8b81a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b8103140 0x7fc8b81a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 -- 172.21.15.7:0/1725608459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80095f0 con 0x7fc8b81015b0 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b6d9d700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b8103140 0x7fc8b81a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.089+0000 7fc8b659c700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b8199680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8003730 tx=0x7fc8a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.090+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a801d070 con 0x7fc8b81015b0 2024-01-11T17:57:58.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.090+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a8005510 con 0x7fc8b81015b0 2024-01-11T17:57:58.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.090+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b81a1750 con 0x7fc8b81015b0 2024-01-11T17:57:58.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.090+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8004a00 con 0x7fc8b81015b0 2024-01-11T17:57:58.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.090+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b81a1c50 con 0x7fc8b81015b0 2024-01-11T17:57:58.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.091+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b80683f0 con 0x7fc8b81015b0 2024-01-11T17:57:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.093+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc8a8004b60 con 0x7fc8b81015b0 2024-01-11T17:57:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.094+0000 7fc8a77fe700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8a00a1f60 0x7fc8a00a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.094+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc8a80ced50 con 0x7fc8b81015b0 2024-01-11T17:57:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.095+0000 7fc8b5d9b700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8a00a1f60 0x7fc8a00a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.095+0000 7fc8b5d9b700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8a00a1f60 0x7fc8a00a4420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009c60 tx=0x7fc8ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.096+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc8a8097af0 con 0x7fc8b81015b0 2024-01-11T17:57:58.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.309+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8b81a1e70 con 0x7fc8b81015b0 2024-01-11T17:57:58.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.309+0000 7fc8a77fe700 1 -- 172.21.15.7:0/1725608459 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8a809b110 con 0x7fc8b81015b0 2024-01-11T17:57:58.310 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:57:58.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8a00a1f60 msgr2=0x7fc8a00a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8a00a1f60 0x7fc8a00a4420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009c60 tx=0x7fc8ac009260 comp rx=0 tx=0).stop 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 msgr2=0x7fc8b8199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b8199680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8003730 tx=0x7fc8a800f690 comp rx=0 tx=0).stop 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 shutdown_connections 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc8a00a1f60 0x7fc8a00a4420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b8103140 0x7fc8b81a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b81015b0 0x7fc8b8199680 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 --2- 172.21.15.7:0/1725608459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8b81027a0 0x7fc8b8199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 >> 172.21.15.7:0/1725608459 conn(0x7fc8b80fcb00 msgr2=0x7fc8b8107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 shutdown_connections 2024-01-11T17:57:58.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.312+0000 7fc8bcfff700 1 -- 172.21.15.7:0/1725608459 wait complete. 2024-01-11T17:57:58.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:57:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 -- 172.21.15.7:0/1953495779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1015b0 msgr2=0x7f1dfc1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 --2- 172.21.15.7:0/1953495779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1019d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1df80099c0 tx=0x7f1df8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 -- 172.21.15.7:0/1953495779 shutdown_connections 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 --2- 172.21.15.7:0/1953495779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc103140 0x7f1dfc1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 --2- 172.21.15.7:0/1953495779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1019d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 --2- 172.21.15.7:0/1953495779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc1027a0 0x7f1dfc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 -- 172.21.15.7:0/1953495779 >> 172.21.15.7:0/1953495779 conn(0x7f1dfc0fcb00 msgr2=0x7f1dfc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.540+0000 7f1e03dff700 1 -- 172.21.15.7:0/1953495779 shutdown_connections 2024-01-11T17:57:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.541+0000 7f1e03dff700 1 -- 172.21.15.7:0/1953495779 wait complete. 2024-01-11T17:57:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 Processor -- start 2024-01-11T17:57:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 -- start start 2024-01-11T17:57:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1027a0 0x7f1dfc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc103140 0x7f1dfc1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1a1230 con 0x7f1dfc1015b0 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1a1370 con 0x7f1dfc1027a0 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e03dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1a1500 con 0x7f1dfc103140 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e01b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e01b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48084/0 (socket says 172.21.15.7:48084) 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e01b9b700 1 -- 172.21.15.7:0/2595653116 learned_addr learned my addr 172.21.15.7:0/2595653116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e0139a700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1027a0 0x7f1dfc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.542+0000 7f1e0239c700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc103140 0x7f1dfc1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e01b9b700 1 -- 172.21.15.7:0/2595653116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc103140 msgr2=0x7f1dfc1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e01b9b700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc103140 0x7f1dfc1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e01b9b700 1 -- 172.21.15.7:0/2595653116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1027a0 msgr2=0x7f1dfc199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e01b9b700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1027a0 0x7f1dfc199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e01b9b700 1 -- 172.21.15.7:0/2595653116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80095f0 con 0x7f1dfc1015b0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e0239c700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc103140 0x7f1dfc1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e01b9b700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1996b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df801d070 con 0x7f1dfc1015b0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df8005020 con 0x7f1dfc1015b0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc1a1780 con 0x7f1dfc1015b0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004bb0 con 0x7f1dfc1015b0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.543+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc1a1ba0 con 0x7f1dfc1015b0 2024-01-11T17:57:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.545+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc068420 con 0x7f1dfc1015b0 2024-01-11T17:57:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.546+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1df8005190 con 0x7f1dfc1015b0 2024-01-11T17:57:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.549+0000 7f1df2ffd700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1de80a1f60 0x7f1de80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.549+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1df80ce6e0 con 0x7f1dfc1015b0 2024-01-11T17:57:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.549+0000 7f1e0139a700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1de80a1f60 0x7f1de80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.550+0000 7f1e0139a700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1de80a1f60 0x7f1de80a4420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009c60 tx=0x7f1dec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.553+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1df8097480 con 0x7f1dfc1015b0 2024-01-11T17:57:58.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:58 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-01-11T17:57:58.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:58 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:57:58.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:58 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3019778137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:58.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:58 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1725608459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.762+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1dfc1a1e20 con 0x7f1dfc1015b0 2024-01-11T17:57:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.763+0000 7f1df2ffd700 1 -- 172.21.15.7:0/2595653116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1df809aaa0 con 0x7f1dfc1015b0 2024-01-11T17:57:58.764 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1de80a1f60 msgr2=0x7f1de80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1de80a1f60 0x7f1de80a4420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009c60 tx=0x7f1dec009260 comp rx=0 tx=0).stop 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 msgr2=0x7f1dfc1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1996b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df800f690 comp rx=0 tx=0).stop 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 shutdown_connections 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1de80a1f60 0x7f1de80a4420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dfc103140 0x7f1dfc1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dfc1015b0 0x7f1dfc1996b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 --2- 172.21.15.7:0/2595653116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dfc1027a0 0x7f1dfc199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 >> 172.21.15.7:0/2595653116 conn(0x7f1dfc0fcb00 msgr2=0x7f1dfc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 shutdown_connections 2024-01-11T17:57:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.765+0000 7f1e03dff700 1 -- 172.21.15.7:0/2595653116 wait complete. 2024-01-11T17:57:58.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:58 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-01-11T17:57:58.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:58 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:57:58.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:58 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3019778137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:58.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:58 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1725608459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:58 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-01-11T17:57:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:58 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:57:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:58 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3019778137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:57:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:58 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1725608459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:57:58.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:57:58.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/139076715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4102010 msgr2=0x7fbee4104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/139076715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4102010 0x7fbee4104400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098f0 tx=0x7fbee0009c00 comp rx=0 tx=0).stop 2024-01-11T17:57:58.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/139076715 shutdown_connections 2024-01-11T17:57:58.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/139076715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4107270 0x7fbee41096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/139076715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4102010 0x7fbee4104400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/139076715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee4104940 0x7fbee4106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/139076715 >> 172.21.15.7:0/139076715 conn(0x7fbee40fba00 msgr2=0x7fbee40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/139076715 shutdown_connections 2024-01-11T17:57:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.993+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/139076715 wait complete. 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.994+0000 7fbeeb9ff700 1 Processor -- start 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.994+0000 7fbeeb9ff700 1 -- start start 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbeeb9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 0x7fbee4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbeeb9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4104940 0x7fbee40fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbeeb9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee4107270 0x7fbee4100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbeeb9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4100ca0 con 0x7fbee4107270 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbeeb9ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4100e10 con 0x7fbee4104940 2024-01-11T17:57:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbeeb9ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee41a14f0 con 0x7fbee4102010 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 0x7fbee4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee9f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee4107270 0x7fbee4100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 0x7fbee4101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55828/0 (socket says 172.21.15.7:55828) 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 -- 172.21.15.7:0/530329487 learned_addr learned my addr 172.21.15.7:0/530329487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee8f9a700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4104940 0x7fbee40fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 -- 172.21.15.7:0/530329487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4104940 msgr2=0x7fbee40fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4104940 0x7fbee40fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 -- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee4107270 msgr2=0x7fbee4100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee4107270 0x7fbee4100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.995+0000 7fbee979b700 1 -- 172.21.15.7:0/530329487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0009520 con 0x7fbee4102010 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.996+0000 7fbee979b700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 0x7fbee4101980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009810 tx=0x7fbee000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.996+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee001c070 con 0x7fbee4102010 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.996+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbee000f7d0 con 0x7fbee4102010 2024-01-11T17:57:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.996+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee41a1770 con 0x7fbee4102010 2024-01-11T17:57:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.996+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0003a60 con 0x7fbee4102010 2024-01-11T17:57:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.996+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee41a1b90 con 0x7fbee4102010 2024-01-11T17:57:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.997+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee40fd580 con 0x7fbee4102010 2024-01-11T17:57:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:58.999+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fbee0020030 con 0x7fbee4102010 2024-01-11T17:57:59.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.000+0000 7fbeda7fc700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbed00a1ee0 0x7fbed00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.001+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbee00ceee0 con 0x7fbee4102010 2024-01-11T17:57:59.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.001+0000 7fbee8f9a700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbed00a1ee0 0x7fbed00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.001+0000 7fbee8f9a700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbed00a1ee0 0x7fbed00a43a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbed4005fd0 tx=0x7fbed4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:59.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.003+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fbee0097da0 con 0x7fbee4102010 2024-01-11T17:57:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.218+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbee410ce80 con 0x7fbee4102010 2024-01-11T17:57:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.219+0000 7fbeda7fc700 1 -- 172.21.15.7:0/530329487 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbee009b3c0 con 0x7fbee4102010 2024-01-11T17:57:59.221 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbed00a1ee0 msgr2=0x7fbed00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbed00a1ee0 0x7fbed00a43a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbed4005fd0 tx=0x7fbed4009310 comp rx=0 tx=0).stop 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 msgr2=0x7fbee4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 0x7fbee4101980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009810 tx=0x7fbee000b790 comp rx=0 tx=0).stop 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 shutdown_connections 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbed00a1ee0 0x7fbed00a43a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbee4102010 0x7fbee4101980 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee4107270 0x7fbee4100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 --2- 172.21.15.7:0/530329487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee4104940 0x7fbee40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 >> 172.21.15.7:0/530329487 conn(0x7fbee40fba00 msgr2=0x7fbee4102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 shutdown_connections 2024-01-11T17:57:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.222+0000 7fbeeb9ff700 1 -- 172.21.15.7:0/530329487 wait complete. 2024-01-11T17:57:59.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:57:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 -- 172.21.15.7:0/3276785015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998109470 msgr2=0x7f399810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3276785015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998109470 0x7f399810b900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f399000b0a0 tx=0x7f399000b3b0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 -- 172.21.15.7:0/3276785015 shutdown_connections 2024-01-11T17:57:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3276785015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998109470 0x7f399810b900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3276785015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3998075650 0x7f3998073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3276785015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3998074070 0x7f39980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 -- 172.21.15.7:0/3276785015 >> 172.21.15.7:0/3276785015 conn(0x7f39980fcb00 msgr2=0x7f39980fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 -- 172.21.15.7:0/3276785015 shutdown_connections 2024-01-11T17:57:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.449+0000 7f399e7ff700 1 -- 172.21.15.7:0/3276785015 wait complete. 2024-01-11T17:57:59.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.450+0000 7f399e7ff700 1 Processor -- start 2024-01-11T17:57:59.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.450+0000 7f399e7ff700 1 -- start start 2024-01-11T17:57:59.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399e7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3998074070 0x7f3998199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 0x7f3998199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399e7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3998109470 0x7f39981a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39981a12f0 con 0x7f3998075650 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399e7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39981a1430 con 0x7f3998074070 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399e7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39981a15c0 con 0x7f3998109470 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 0x7f3998199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 0x7f3998199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48128/0 (socket says 172.21.15.7:48128) 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 -- 172.21.15.7:0/3771230019 learned_addr learned my addr 172.21.15.7:0/3771230019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f399cd9c700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3998109470 0x7f39981a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f3997fff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3998074070 0x7f3998199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 -- 172.21.15.7:0/3771230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3998109470 msgr2=0x7f39981a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3998109470 0x7f39981a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 -- 172.21.15.7:0/3771230019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3998074070 msgr2=0x7f3998199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3998074070 0x7f3998199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.451+0000 7f39977fe700 1 -- 172.21.15.7:0/3771230019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009d00 con 0x7f3998075650 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.452+0000 7f39977fe700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 0x7f3998199cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f398c00e970 tx=0x7f398c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.452+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c009790 con 0x7f3998075650 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.452+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39981a18a0 con 0x7f3998075650 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.452+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f398c00c7e0 con 0x7f3998075650 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.452+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c003c10 con 0x7f3998075650 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.452+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39981a1e00 con 0x7f3998075650 2024-01-11T17:57:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.453+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3998108230 con 0x7f3998075650 2024-01-11T17:57:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.455+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f398c00cc50 con 0x7f3998075650 2024-01-11T17:57:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.456+0000 7f39957fa700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39800a1ee0 0x7f39800a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.456+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f398c014070 con 0x7f3998075650 2024-01-11T17:57:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.457+0000 7f3997fff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39800a1ee0 0x7f39800a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.457+0000 7f3997fff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39800a1ee0 0x7f39800a43a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3988009c60 tx=0x7f3988009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:59.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.460+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f398c096b70 con 0x7f3998075650 2024-01-11T17:57:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:59 smithi007 ceph-mon[33361]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-01-11T17:57:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:59 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2595653116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:57:59 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/530329487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.671+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f399802d3f0 con 0x7f3998075650 2024-01-11T17:57:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.672+0000 7f39957fa700 1 -- 172.21.15.7:0/3771230019 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f398c09a190 con 0x7f3998075650 2024-01-11T17:57:59.673 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39800a1ee0 msgr2=0x7f39800a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39800a1ee0 0x7f39800a43a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3988009c60 tx=0x7f3988009260 comp rx=0 tx=0).stop 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 msgr2=0x7f3998199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 0x7f3998199cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f398c00e970 tx=0x7f398c00ec80 comp rx=0 tx=0).stop 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 shutdown_connections 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f39800a1ee0 0x7f39800a43a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3998109470 0x7f39981a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3998074070 0x7f3998199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 --2- 172.21.15.7:0/3771230019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3998075650 0x7f3998199cb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 >> 172.21.15.7:0/3771230019 conn(0x7f39980fcb00 msgr2=0x7f3998105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 shutdown_connections 2024-01-11T17:57:59.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.674+0000 7f399e7ff700 1 -- 172.21.15.7:0/3771230019 wait complete. 2024-01-11T17:57:59.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:57:59.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:59 smithi016 ceph-mon[43195]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-01-11T17:57:59.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:59 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2595653116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:59.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:57:59 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/530329487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:59.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:59 smithi038 ceph-mon[43582]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-01-11T17:57:59.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:59 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2595653116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:57:59.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:57:59 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/530329487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:57:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 -- 172.21.15.7:0/441935612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634075650 msgr2=0x7fc634073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 --2- 172.21.15.7:0/441935612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634075650 0x7fc634073aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc624009990 tx=0x7fc624009ca0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 -- 172.21.15.7:0/441935612 shutdown_connections 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 --2- 172.21.15.7:0/441935612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634109450 0x7fc63410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 --2- 172.21.15.7:0/441935612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634075650 0x7fc634073aa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 --2- 172.21.15.7:0/441935612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 0x7fc6340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 -- 172.21.15.7:0/441935612 >> 172.21.15.7:0/441935612 conn(0x7fc6340fcb00 msgr2=0x7fc6340fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 -- 172.21.15.7:0/441935612 shutdown_connections 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.899+0000 7fc639dff700 1 -- 172.21.15.7:0/441935612 wait complete. 2024-01-11T17:57:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.900+0000 7fc639dff700 1 Processor -- start 2024-01-11T17:57:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.900+0000 7fc639dff700 1 -- start start 2024-01-11T17:57:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc639dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 0x7fc6341996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc639dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 0x7fc634199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc639dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109450 0x7fc6341a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc639dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6341a1200 con 0x7fc634109450 2024-01-11T17:57:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc639dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6341a1340 con 0x7fc634074070 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc639dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6341a14d0 con 0x7fc634075650 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc632ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 0x7fc634199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc633fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109450 0x7fc6341a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc632ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 0x7fc634199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55858/0 (socket says 172.21.15.7:55858) 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc632ffd700 1 -- 172.21.15.7:0/4163253475 learned_addr learned my addr 172.21.15.7:0/4163253475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc6337fe700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 0x7fc6341996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc633fff700 1 -- 172.21.15.7:0/4163253475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 msgr2=0x7fc634199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc633fff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 0x7fc634199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc633fff700 1 -- 172.21.15.7:0/4163253475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 msgr2=0x7fc6341996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc633fff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 0x7fc6341996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc633fff700 1 -- 172.21.15.7:0/4163253475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6240095f0 con 0x7fc634109450 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc632ffd700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 0x7fc634199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.901+0000 7fc6337fe700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 0x7fc6341996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.902+0000 7fc633fff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109450 0x7fc6341a0c30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc62c007f10 tx=0x7fc62c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.902+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c004db0 con 0x7fc634109450 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.902+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6341a17b0 con 0x7fc634109450 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.902+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc62c00f040 con 0x7fc634109450 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.902+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c005990 con 0x7fc634109450 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.902+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6341a1c70 con 0x7fc634109450 2024-01-11T17:57:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.903+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc634108210 con 0x7fc634109450 2024-01-11T17:57:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.905+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc62c00db20 con 0x7fc634109450 2024-01-11T17:57:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.906+0000 7fc630ff9700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc61c0a1ee0 0x7fc61c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:57:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.907+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc62c0cd5e0 con 0x7fc634109450 2024-01-11T17:57:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.907+0000 7fc6337fe700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc61c0a1ee0 0x7fc61c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:57:59.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.907+0000 7fc6337fe700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc61c0a1ee0 0x7fc61c0a43a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc62400b2e0 tx=0x7fc62400b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:57:59.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:57:59.910+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc62c04f380 con 0x7fc634109450 2024-01-11T17:58:00.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.123+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6340683f0 con 0x7fc634109450 2024-01-11T17:58:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.125+0000 7fc630ff9700 1 -- 172.21.15.7:0/4163253475 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fc62c0962f0 con 0x7fc634109450 2024-01-11T17:58:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:00.128 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:58:00.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.127+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc61c0a1ee0 msgr2=0x7fc61c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.127+0000 7fc639dff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc61c0a1ee0 0x7fc61c0a43a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc62400b2e0 tx=0x7fc62400b750 comp rx=0 tx=0).stop 2024-01-11T17:58:00.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109450 msgr2=0x7fc6341a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109450 0x7fc6341a0c30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc62c007f10 tx=0x7fc62c00d3b0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 shutdown_connections 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc61c0a1ee0 0x7fc61c0a43a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109450 0x7fc6341a0c30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc634074070 0x7fc6341996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 --2- 172.21.15.7:0/4163253475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc634075650 0x7fc634199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 >> 172.21.15.7:0/4163253475 conn(0x7fc6340fcb00 msgr2=0x7fc634105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 shutdown_connections 2024-01-11T17:58:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.128+0000 7fc639dff700 1 -- 172.21.15.7:0/4163253475 wait complete. 2024-01-11T17:58:00.150 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:58:00.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:58:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/1871878745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541027a0 msgr2=0x7f1b54102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/1871878745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541027a0 0x7f1b54102c00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1b440099c0 tx=0x7f1b44009cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/1871878745 shutdown_connections 2024-01-11T17:58:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/1871878745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54103140 0x7f1b541099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/1871878745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541015b0 0x7f1b541019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/1871878745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541027a0 0x7f1b54102c00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/1871878745 >> 172.21.15.7:0/1871878745 conn(0x7f1b540fcb00 msgr2=0x7f1b540fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/1871878745 shutdown_connections 2024-01-11T17:58:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.357+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/1871878745 wait complete. 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.358+0000 7f1b5b7ff700 1 Processor -- start 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.358+0000 7f1b5b7ff700 1 -- start start 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5b7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 0x7f1b54199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5b7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541027a0 0x7f1b54199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5b7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54103140 0x7f1b541a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5b7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541a11f0 con 0x7f1b541015b0 2024-01-11T17:58:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5b7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541a1330 con 0x7f1b541027a0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5b7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541a14c0 con 0x7f1b54103140 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 0x7f1b54199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 0x7f1b54199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48158/0 (socket says 172.21.15.7:48158) 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 -- 172.21.15.7:0/4110500389 learned_addr learned my addr 172.21.15.7:0/4110500389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b58d9a700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541027a0 0x7f1b54199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b59d9c700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54103140 0x7f1b541a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 -- 172.21.15.7:0/4110500389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54103140 msgr2=0x7f1b541a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54103140 0x7f1b541a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 -- 172.21.15.7:0/4110500389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541027a0 msgr2=0x7f1b54199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541027a0 0x7f1b54199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.359+0000 7f1b5959b700 1 -- 172.21.15.7:0/4110500389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440095f0 con 0x7f1b541015b0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b58d9a700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541027a0 0x7f1b54199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b5959b700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 0x7f1b54199670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e970 tx=0x7f1b5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50009790 con 0x7f1b541015b0 2024-01-11T17:58:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b5000c7e0 con 0x7f1b541015b0 2024-01-11T17:58:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b541a17a0 con 0x7f1b541015b0 2024-01-11T17:58:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50003c70 con 0x7f1b541015b0 2024-01-11T17:58:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.360+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b541a1d00 con 0x7f1b541015b0 2024-01-11T17:58:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.361+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b540683f0 con 0x7f1b541015b0 2024-01-11T17:58:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.363+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1b5000cc50 con 0x7f1b541015b0 2024-01-11T17:58:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.365+0000 7f1b4a7fc700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1b400a1f60 0x7f1b400a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.365+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1b50014070 con 0x7f1b541015b0 2024-01-11T17:58:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.365+0000 7f1b58d9a700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1b400a1f60 0x7f1b400a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.366+0000 7f1b58d9a700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1b400a1f60 0x7f1b400a4420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009990 tx=0x7f1b4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:00.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.371+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1b50096b60 con 0x7f1b541015b0 2024-01-11T17:58:00.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:00 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3771230019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:00.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:00 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4163253475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:00.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.580+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b541a1fb0 con 0x7f1b541015b0 2024-01-11T17:58:00.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.581+0000 7f1b4a7fc700 1 -- 172.21.15.7:0/4110500389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b50003dd0 con 0x7f1b541015b0 2024-01-11T17:58:00.582 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1b400a1f60 msgr2=0x7f1b400a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1b400a1f60 0x7f1b400a4420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009990 tx=0x7f1b4401a040 comp rx=0 tx=0).stop 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 msgr2=0x7f1b54199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 0x7f1b54199670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e970 tx=0x7f1b5000ec80 comp rx=0 tx=0).stop 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 shutdown_connections 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1b400a1f60 0x7f1b400a4420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1b54103140 0x7f1b541a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b541015b0 0x7f1b54199670 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 --2- 172.21.15.7:0/4110500389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b541027a0 0x7f1b54199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 >> 172.21.15.7:0/4110500389 conn(0x7f1b540fcb00 msgr2=0x7f1b54073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 shutdown_connections 2024-01-11T17:58:00.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.583+0000 7f1b5b7ff700 1 -- 172.21.15.7:0/4110500389 wait complete. 2024-01-11T17:58:00.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:58:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:00 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3771230019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:00 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4163253475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:00.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:00 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3771230019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:00.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:00 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4163253475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 -- 172.21.15.7:0/1026561461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1015c0 msgr2=0x7f381c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 --2- 172.21.15.7:0/1026561461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1015c0 0x7f381c1019e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38180099c0 tx=0x7f3818009cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 -- 172.21.15.7:0/1026561461 shutdown_connections 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 --2- 172.21.15.7:0/1026561461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f381c103150 0x7f381c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 --2- 172.21.15.7:0/1026561461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1015c0 0x7f381c1019e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 --2- 172.21.15.7:0/1026561461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f381c1027b0 0x7f381c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 -- 172.21.15.7:0/1026561461 >> 172.21.15.7:0/1026561461 conn(0x7f381c0fcb30 msgr2=0x7f381c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 -- 172.21.15.7:0/1026561461 shutdown_connections 2024-01-11T17:58:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.813+0000 7f3823bff700 1 -- 172.21.15.7:0/1026561461 wait complete. 2024-01-11T17:58:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 Processor -- start 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 -- start start 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f381c1015c0 0x7f381c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 0x7f381c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f381c103150 0x7f381c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19f060 con 0x7f381c1027b0 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19f1a0 con 0x7f381c1015c0 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f3823bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19f330 con 0x7f381c103150 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.814+0000 7f382119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 0x7f381c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 0x7f381c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48184/0 (socket says 172.21.15.7:48184) 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 -- 172.21.15.7:0/338935640 learned_addr learned my addr 172.21.15.7:0/338935640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382219c700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f381c103150 0x7f381c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382199b700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f381c1015c0 0x7f381c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 -- 172.21.15.7:0/338935640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f381c103150 msgr2=0x7f381c19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f381c103150 0x7f381c19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 -- 172.21.15.7:0/338935640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f381c1015c0 msgr2=0x7f381c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f381c1015c0 0x7f381c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 -- 172.21.15.7:0/338935640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180095f0 con 0x7f381c1027b0 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f382119a700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 0x7f381c197a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f380c00d950 tx=0x7f380c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.815+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00f890 con 0x7f381c1027b0 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.816+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f380c010460 con 0x7f381c1027b0 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.816+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c19f610 con 0x7f381c1027b0 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.816+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00e610 con 0x7f381c1027b0 2024-01-11T17:58:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.816+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c19fb20 con 0x7f381c1027b0 2024-01-11T17:58:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.817+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c108970 con 0x7f381c1027b0 2024-01-11T17:58:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.819+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f380c0108d0 con 0x7f381c1027b0 2024-01-11T17:58:00.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.820+0000 7f3812ffd700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38080a1ee0 0x7f38080a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:00.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.820+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f380c0cda40 con 0x7f381c1027b0 2024-01-11T17:58:00.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.820+0000 7f382199b700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38080a1ee0 0x7f38080a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:00.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.821+0000 7f382199b700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38080a1ee0 0x7f38080a43a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3818003730 tx=0x7f3818000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:00.824+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f380c0cfe60 con 0x7f381c1027b0 2024-01-11T17:58:01.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.035+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f381c02d3f0 con 0x7f381c1027b0 2024-01-11T17:58:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.035+0000 7f3812ffd700 1 -- 172.21.15.7:0/338935640 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f380c016020 con 0x7f381c1027b0 2024-01-11T17:58:01.037 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:58:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38080a1ee0 msgr2=0x7f38080a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38080a1ee0 0x7f38080a43a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3818003730 tx=0x7f3818000bc0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 msgr2=0x7f381c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 0x7f381c197a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f380c00d950 tx=0x7f380c00dc60 comp rx=0 tx=0).stop 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 shutdown_connections 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f38080a1ee0 0x7f38080a43a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f381c103150 0x7f381c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f381c1015c0 0x7f381c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 --2- 172.21.15.7:0/338935640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c1027b0 0x7f381c197a20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 >> 172.21.15.7:0/338935640 conn(0x7f381c0fcb30 msgr2=0x7f381c105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 shutdown_connections 2024-01-11T17:58:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.038+0000 7f3823bff700 1 -- 172.21.15.7:0/338935640 wait complete. 2024-01-11T17:58:01.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 -- 172.21.15.7:0/2115527607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 msgr2=0x7fdcf01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/2115527607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 0x7fdcf01099f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fc0 tx=0x7fdce800b330 comp rx=0 tx=0).stop 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 -- 172.21.15.7:0/2115527607 shutdown_connections 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/2115527607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcf01015b0 0x7fdcf01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/2115527607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 0x7fdcf01099f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/2115527607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01027a0 0x7fdcf0102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 -- 172.21.15.7:0/2115527607 >> 172.21.15.7:0/2115527607 conn(0x7fdcf00fcb00 msgr2=0x7fdcf00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 -- 172.21.15.7:0/2115527607 shutdown_connections 2024-01-11T17:58:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.266+0000 7fdcf71ff700 1 -- 172.21.15.7:0/2115527607 wait complete. 2024-01-11T17:58:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.267+0000 7fdcf71ff700 1 Processor -- start 2024-01-11T17:58:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 -- start start 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 0x7fdcf00753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcf01027a0 0x7fdcf0073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 0x7fdcf0073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00746e0 con 0x7fdcf01015b0 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf0074850 con 0x7fdcf0103140 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf71ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01a15f0 con 0x7fdcf01027a0 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf4f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 0x7fdcf00753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf4f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 0x7fdcf00753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48202/0 (socket says 172.21.15.7:48202) 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf4f9b700 1 -- 172.21.15.7:0/3791689042 learned_addr learned my addr 172.21.15.7:0/3791689042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdceffff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcf01027a0 0x7fdcf0073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.268+0000 7fdcf579c700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 0x7fdcf0073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf4f9b700 1 -- 172.21.15.7:0/3791689042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcf01027a0 msgr2=0x7fdcf0073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf4f9b700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcf01027a0 0x7fdcf0073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf4f9b700 1 -- 172.21.15.7:0/3791689042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 msgr2=0x7fdcf0073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf4f9b700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 0x7fdcf0073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf4f9b700 1 -- 172.21.15.7:0/3791689042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009c40 con 0x7fdcf01015b0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf4f9b700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 0x7fdcf00753c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdce400e970 tx=0x7fdce400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4009790 con 0x7fdcf01015b0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf01a18d0 con 0x7fdcf01015b0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdce400c7c0 con 0x7fdcf01015b0 2024-01-11T17:58:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003c20 con 0x7fdcf01015b0 2024-01-11T17:58:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.269+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf01a1e30 con 0x7fdcf01015b0 2024-01-11T17:58:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.271+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf0108900 con 0x7fdcf01015b0 2024-01-11T17:58:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.273+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fdce400c930 con 0x7fdcf01015b0 2024-01-11T17:58:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.274+0000 7fdcedffb700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdce00a1f60 0x7fdce00a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.274+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdce4014070 con 0x7fdcf01015b0 2024-01-11T17:58:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.274+0000 7fdceffff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdce00a1f60 0x7fdce00a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.275+0000 7fdceffff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdce00a1f60 0x7fdce00a4420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005fd0 tx=0x7fdcdc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.278+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdce4096bb0 con 0x7fdcf01015b0 2024-01-11T17:58:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.492+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcf002d490 con 0x7fdcf01015b0 2024-01-11T17:58:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.493+0000 7fdcedffb700 1 -- 172.21.15.7:0/3791689042 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdce409a1d0 con 0x7fdcf01015b0 2024-01-11T17:58:01.495 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:58:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdce00a1f60 msgr2=0x7fdce00a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdce00a1f60 0x7fdce00a4420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005fd0 tx=0x7fdcdc009310 comp rx=0 tx=0).stop 2024-01-11T17:58:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 msgr2=0x7fdcf00753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 0x7fdcf00753c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdce400e970 tx=0x7fdce400ec80 comp rx=0 tx=0).stop 2024-01-11T17:58:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 shutdown_connections 2024-01-11T17:58:01.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdce00a1f60 0x7fdce00a4420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf0103140 0x7fdcf0073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcf01015b0 0x7fdcf00753c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 --2- 172.21.15.7:0/3791689042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcf01027a0 0x7fdcf0073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.495+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 >> 172.21.15.7:0/3791689042 conn(0x7fdcf00fcb00 msgr2=0x7fdcf0105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.496+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 shutdown_connections 2024-01-11T17:58:01.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.496+0000 7fdcf71ff700 1 -- 172.21.15.7:0/3791689042 wait complete. 2024-01-11T17:58:01.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:58:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 -- 172.21.15.7:0/801251554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880ffe00 msgr2=0x7f0688100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 --2- 172.21.15.7:0/801251554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880ffe00 0x7f0688100220 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0678009990 tx=0x7f0678009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 -- 172.21.15.7:0/801251554 shutdown_connections 2024-01-11T17:58:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 --2- 172.21.15.7:0/801251554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688101120 0x7f06881015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 --2- 172.21.15.7:0/801251554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880ffe00 0x7f0688100220 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 --2- 172.21.15.7:0/801251554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688100760 0x7f0688100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 -- 172.21.15.7:0/801251554 >> 172.21.15.7:0/801251554 conn(0x7f06880fba00 msgr2=0x7f06880fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 -- 172.21.15.7:0/801251554 shutdown_connections 2024-01-11T17:58:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.722+0000 7f068d9ff700 1 -- 172.21.15.7:0/801251554 wait complete. 2024-01-11T17:58:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.723+0000 7f068d9ff700 1 Processor -- start 2024-01-11T17:58:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 -- start start 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 0x7f0688199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688100760 0x7f0688199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688101120 0x7f06881a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06881a1200 con 0x7f0688101120 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06881a1340 con 0x7f06880ffe00 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f068d9ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06881a14d0 con 0x7f0688100760 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f06877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688101120 0x7f06881a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f0686ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 0x7f0688199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f067ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688100760 0x7f0688199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f06877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688101120 0x7f06881a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48222/0 (socket says 172.21.15.7:48222) 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f067ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688100760 0x7f0688199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55930/0 (socket says 172.21.15.7:55930) 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f067ffff700 1 -- 172.21.15.7:0/2856590930 learned_addr learned my addr 172.21.15.7:0/2856590930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.724+0000 7f0686ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 0x7f0688199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49768/0 (socket says 172.21.15.7:49768) 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f0686ffd700 1 -- 172.21.15.7:0/2856590930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688100760 msgr2=0x7f0688199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f0686ffd700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688100760 0x7f0688199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f0686ffd700 1 -- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688101120 msgr2=0x7f06881a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f0686ffd700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688101120 0x7f06881a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f0686ffd700 1 -- 172.21.15.7:0/2856590930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780095f0 con 0x7f06880ffe00 2024-01-11T17:58:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f0686ffd700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 0x7f0688199680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0678000c00 tx=0x7f0678004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067801d070 con 0x7f06880ffe00 2024-01-11T17:58:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0678021410 con 0x7f06880ffe00 2024-01-11T17:58:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06881a1750 con 0x7f06880ffe00 2024-01-11T17:58:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.725+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678021ea0 con 0x7f06880ffe00 2024-01-11T17:58:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.726+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06881a1b70 con 0x7f06880ffe00 2024-01-11T17:58:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.727+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068810abe0 con 0x7f06880ffe00 2024-01-11T17:58:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.728+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0678021580 con 0x7f06880ffe00 2024-01-11T17:58:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.730+0000 7f068c9fd700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06740a1ee0 0x7f06740a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.730+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f06780ced60 con 0x7f06880ffe00 2024-01-11T17:58:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.730+0000 7f067ffff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06740a1ee0 0x7f06740a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.730+0000 7f067ffff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06740a1ee0 0x7f06740a43a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0670009d20 tx=0x7f0670009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.732+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0678097b60 con 0x7f06880ffe00 2024-01-11T17:58:01.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:01 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4110500389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:58:01.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:01 smithi016 ceph-mon[43195]: pgmap v306: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-01-11T17:58:01.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:01 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/338935640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:58:01.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:01 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4110500389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:58:01.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:01 smithi038 ceph-mon[43582]: pgmap v306: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-01-11T17:58:01.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:01 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/338935640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:58:01.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:01 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4110500389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:58:01.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:01 smithi007 ceph-mon[33361]: pgmap v306: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-01-11T17:58:01.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:01 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/338935640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:58:01.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.949+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f068810f040 con 0x7f06880ffe00 2024-01-11T17:58:01.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.949+0000 7f068c9fd700 1 -- 172.21.15.7:0/2856590930 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f067809b180 con 0x7f06880ffe00 2024-01-11T17:58:01.951 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:58:01.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.951+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06740a1ee0 msgr2=0x7f06740a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.951+0000 7f068d9ff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06740a1ee0 0x7f06740a43a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0670009d20 tx=0x7f0670009260 comp rx=0 tx=0).stop 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.951+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 msgr2=0x7f0688199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.951+0000 7f068d9ff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 0x7f0688199680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0678000c00 tx=0x7f0678004730 comp rx=0 tx=0).stop 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 shutdown_connections 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06740a1ee0 0x7f06740a43a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688101120 0x7f06881a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06880ffe00 0x7f0688199680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 --2- 172.21.15.7:0/2856590930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0688100760 0x7f0688199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 >> 172.21.15.7:0/2856590930 conn(0x7f06880fba00 msgr2=0x7f06880fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 shutdown_connections 2024-01-11T17:58:01.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:01.952+0000 7f068d9ff700 1 -- 172.21.15.7:0/2856590930 wait complete. 2024-01-11T17:58:01.972 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-01-11T17:58:01.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:58:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 -- 172.21.15.7:0/2620033664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 msgr2=0x7f53ac1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 --2- 172.21.15.7:0/2620033664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac1019b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f539c009920 tx=0x7f539c009c30 comp rx=0 tx=0).stop 2024-01-11T17:58:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 -- 172.21.15.7:0/2620033664 shutdown_connections 2024-01-11T17:58:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 --2- 172.21.15.7:0/2620033664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 --2- 172.21.15.7:0/2620033664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac1019b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 --2- 172.21.15.7:0/2620033664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ac102780 0x7f53ac102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 -- 172.21.15.7:0/2620033664 >> 172.21.15.7:0/2620033664 conn(0x7f53ac0fcb00 msgr2=0x7f53ac0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:02.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 -- 172.21.15.7:0/2620033664 shutdown_connections 2024-01-11T17:58:02.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.178+0000 7f53b17ff700 1 -- 172.21.15.7:0/2620033664 wait complete. 2024-01-11T17:58:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.179+0000 7f53b17ff700 1 Processor -- start 2024-01-11T17:58:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.179+0000 7f53b17ff700 1 -- start start 2024-01-11T17:58:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53b17ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53b17ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ac102780 0x7f53ac199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53b17ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53b17ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac1a12c0 con 0x7f53ac103120 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53b17ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac1a1400 con 0x7f53ac101590 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53b17ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac1a1590 con 0x7f53ac102780 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46686/0 (socket says 172.21.15.7:46686) 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53ab7fe700 1 -- 172.21.15.7:0/3921588919 learned_addr learned my addr 172.21.15.7:0/3921588919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53aaffd700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.180+0000 7f53a3fff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ac102780 0x7f53ac199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53ab7fe700 1 -- 172.21.15.7:0/3921588919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ac102780 msgr2=0x7f53ac199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53ab7fe700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ac102780 0x7f53ac199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53ab7fe700 1 -- 172.21.15.7:0/3921588919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 msgr2=0x7f53ac199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53ab7fe700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53ab7fe700 1 -- 172.21.15.7:0/3921588919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c009520 con 0x7f53ac103120 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53aaffd700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53ab7fe700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1a0cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53a400ef30 tx=0x7f53a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4018070 con 0x7f53ac103120 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac1a1810 con 0x7f53ac103120 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53a400f040 con 0x7f53ac103120 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400ccf0 con 0x7f53ac103120 2024-01-11T17:58:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.181+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac1a1cd0 con 0x7f53ac103120 2024-01-11T17:58:02.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.182+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac0683f0 con 0x7f53ac103120 2024-01-11T17:58:02.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.184+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f53a4003a00 con 0x7f53ac103120 2024-01-11T17:58:02.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.186+0000 7f53a8ff9700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53980a1ee0 0x7f53980a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.186+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f53a40ce1b0 con 0x7f53ac103120 2024-01-11T17:58:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.186+0000 7f53aaffd700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53980a1ee0 0x7f53980a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.186+0000 7f53aaffd700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53980a1ee0 0x7f53980a43a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f539c0098f0 tx=0x7f539c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.188+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f53a4096fb0 con 0x7f53ac103120 2024-01-11T17:58:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.450+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53ac10cf80 con 0x7f53ac103120 2024-01-11T17:58:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.451+0000 7f53a8ff9700 1 -- 172.21.15.7:0/3921588919 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11309 (secure 0 0 0) 0x7f53a409a5d0 con 0x7f53ac103120 2024-01-11T17:58:02.453 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:02.453 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:57:57.025520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53980a1ee0 msgr2=0x7f53980a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53980a1ee0 0x7f53980a43a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f539c0098f0 tx=0x7f539c019040 comp rx=0 tx=0).stop 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 msgr2=0x7f53ac1a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1a0cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53a400ef30 tx=0x7f53a400c3a0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 shutdown_connections 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f53980a1ee0 0x7f53980a43a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ac103120 0x7f53ac1a0cf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac101590 0x7f53ac199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 --2- 172.21.15.7:0/3921588919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ac102780 0x7f53ac199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 >> 172.21.15.7:0/3921588919 conn(0x7f53ac0fcb00 msgr2=0x7f53ac107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 shutdown_connections 2024-01-11T17:58:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.453+0000 7f53b17ff700 1 -- 172.21.15.7:0/3921588919 wait complete. 2024-01-11T17:58:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-01-11T17:58:02.474 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-01-11T17:58:02.474 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-01-11T17:58:02.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 -- 172.21.15.7:0/2767306140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 msgr2=0x7f49541019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 --2- 172.21.15.7:0/2767306140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 0x7f49541019b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4950009990 tx=0x7f4950009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 -- 172.21.15.7:0/2767306140 shutdown_connections 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 --2- 172.21.15.7:0/2767306140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 0x7f49541019b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 --2- 172.21.15.7:0/2767306140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954103120 0x7f49541099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 --2- 172.21.15.7:0/2767306140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954102780 0x7f4954102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 -- 172.21.15.7:0/2767306140 >> 172.21.15.7:0/2767306140 conn(0x7f49540fcb00 msgr2=0x7f49540fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 -- 172.21.15.7:0/2767306140 shutdown_connections 2024-01-11T17:58:02.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.678+0000 7f495bfff700 1 -- 172.21.15.7:0/2767306140 wait complete. 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.679+0000 7f495bfff700 1 Processor -- start 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.679+0000 7f495bfff700 1 -- start start 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 0x7f4954199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 0x7f4954199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954103120 0x7f49541a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541a1270 con 0x7f4954102780 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495bfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541a13b0 con 0x7f4954103120 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495bfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541a1540 con 0x7f4954101590 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 0x7f4954199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f4959d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 0x7f4954199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 0x7f4954199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46714/0 (socket says 172.21.15.7:46714) 2024-01-11T17:58:02.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 -- 172.21.15.7:0/414279446 learned_addr learned my addr 172.21.15.7:0/414279446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495a59c700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954103120 0x7f49541a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 -- 172.21.15.7:0/414279446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 msgr2=0x7f4954199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 0x7f4954199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 -- 172.21.15.7:0/414279446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954103120 msgr2=0x7f49541a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954103120 0x7f49541a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.680+0000 7f495959a700 1 -- 172.21.15.7:0/414279446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c009c40 con 0x7f4954102780 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.681+0000 7f495959a700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 0x7f4954199cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f494400e9a0 tx=0x7f494400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.681+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944009790 con 0x7f4954102780 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.681+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49500095f0 con 0x7f4954102780 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.681+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f494400ce80 con 0x7f4954102780 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.681+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944018680 con 0x7f4954102780 2024-01-11T17:58:02.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.681+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49541a1bf0 con 0x7f4954102780 2024-01-11T17:58:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.682+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49540683f0 con 0x7f4954102780 2024-01-11T17:58:02.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.684+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4944010430 con 0x7f4954102780 2024-01-11T17:58:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.685+0000 7f494affd700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f49400a1ee0 0x7f49400a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.686+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4944014070 con 0x7f4954102780 2024-01-11T17:58:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.686+0000 7f4959d9b700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f49400a1ee0 0x7f49400a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:02.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.686+0000 7f4959d9b700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f49400a1ee0 0x7f49400a43a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f495000b680 tx=0x7f495000b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.689+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f4944052380 con 0x7f4954102780 2024-01-11T17:58:02.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:02 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3791689042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:58:02.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:02 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2856590930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:02.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:02 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3791689042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:58:02.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:02 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2856590930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:02.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:02 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3791689042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:58:02.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:02 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2856590930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:02.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.898+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4954109710 con 0x7f49400a1ee0 2024-01-11T17:58:02.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.911+0000 7f494affd700 1 -- 172.21.15.7:0/414279446 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427403 (secure 0 0 0) 0x7f4954109710 con 0x7f49400a1ee0 2024-01-11T17:58:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f49400a1ee0 msgr2=0x7f49400a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f49400a1ee0 0x7f49400a43a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f495000b680 tx=0x7f495000b750 comp rx=0 tx=0).stop 2024-01-11T17:58:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 msgr2=0x7f4954199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 0x7f4954199cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f494400e9a0 tx=0x7f494400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 shutdown_connections 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f49400a1ee0 0x7f49400a43a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4954101590 0x7f4954199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954103120 0x7f49541a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 --2- 172.21.15.7:0/414279446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4954102780 0x7f4954199cd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 >> 172.21.15.7:0/414279446 conn(0x7f49540fcb00 msgr2=0x7f4954107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 shutdown_connections 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:02.917+0000 7f495bfff700 1 -- 172.21.15.7:0/414279446 wait complete. 2024-01-11T17:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:58:02.938 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-01-11T17:58:02.596278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664862,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":679,"num_read_kb":466,"num_write":351,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385892,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555932,"statfs":{"total":1151940427776,"available":1151545274368,"internally_reserved":0,"allocated":12062720,"data_stored":7870112,"data_compressed":396960,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6516,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003397"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639344+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:57:54.639344+0000","last_peered":"2024-01-11T17:57:54.639344+0000","last_clean":"2024-01-11T17:57:54.639344+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:57:54.639344+0000","last_undegraded":"2024-01-11T17:57:54.639344+0000","last_fullsized":"2024-01-11T17:57:54.639344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631254+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:57:54.631254+0000","last_peered":"2024-01-11T17:57:54.631254+0000","last_clean":"2024-01-11T17:57:54.631254+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:57:54.631254+0000","last_undegraded":"2024-01-11T17:57:54.631254+0000","last_fullsized":"2024-01-11T17:57:54.631254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.640000+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:57:54.640000+0000","last_peered":"2024-01-11T17:57:54.640000+0000","last_clean":"2024-01-11T17:57:54.640000+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:57:54.640000+0000","last_undegraded":"2024-01-11T17:57:54.640000+0000","last_fullsized":"2024-01-11T17:57:54.640000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.625117+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:57:54.625117+0000","last_peered":"2024-01-11T17:57:54.625117+0000","last_clean":"2024-01-11T17:57:54.625117+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:57:54.625117+0000","last_undegraded":"2024-01-11T17:57:54.625117+0000","last_fullsized":"2024-01-11T17:57:54.625117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643729+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:57:54.643729+0000","last_peered":"2024-01-11T17:57:54.643729+0000","last_clean":"2024-01-11T17:57:54.643729+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:57:54.643729+0000","last_undegraded":"2024-01-11T17:57:54.643729+0000","last_fullsized":"2024-01-11T17:57:54.643729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637431+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:57:54.637431+0000","last_peered":"2024-01-11T17:57:54.637431+0000","last_clean":"2024-01-11T17:57:54.637431+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:57:54.637431+0000","last_undegraded":"2024-01-11T17:57:54.637431+0000","last_fullsized":"2024-01-11T17:57:54.637431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631584+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:57:54.631584+0000","last_peered":"2024-01-11T17:57:54.631584+0000","last_clean":"2024-01-11T17:57:54.631584+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:57:54.631584+0000","last_undegraded":"2024-01-11T17:57:54.631584+0000","last_fullsized":"2024-01-11T17:57:54.631584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.642078+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:57:54.642078+0000","last_peered":"2024-01-11T17:57:54.642078+0000","last_clean":"2024-01-11T17:57:54.642078+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:57:54.642078+0000","last_undegraded":"2024-01-11T17:57:54.642078+0000","last_fullsized":"2024-01-11T17:57:54.642078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636927+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:57:54.636927+0000","last_peered":"2024-01-11T17:57:54.636927+0000","last_clean":"2024-01-11T17:57:54.636927+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:57:54.636927+0000","last_undegraded":"2024-01-11T17:57:54.636927+0000","last_fullsized":"2024-01-11T17:57:54.636927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631470+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:57:54.631470+0000","last_peered":"2024-01-11T17:57:54.631470+0000","last_clean":"2024-01-11T17:57:54.631470+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:57:54.631470+0000","last_undegraded":"2024-01-11T17:57:54.631470+0000","last_fullsized":"2024-01-11T17:57:54.631470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627375+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:57:54.627375+0000","last_peered":"2024-01-11T17:57:54.627375+0000","last_clean":"2024-01-11T17:57:54.627375+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:57:54.627375+0000","last_undegraded":"2024-01-11T17:57:54.627375+0000","last_fullsized":"2024-01-11T17:57:54.627375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623494+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:57:54.623494+0000","last_peered":"2024-01-11T17:57:54.623494+0000","last_clean":"2024-01-11T17:57:54.623494+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:57:54.623494+0000","last_undegraded":"2024-01-11T17:57:54.623494+0000","last_fullsized":"2024-01-11T17:57:54.623494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645164+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:57:54.645164+0000","last_peered":"2024-01-11T17:57:54.645164+0000","last_clean":"2024-01-11T17:57:54.645164+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:57:54.645164+0000","last_undegraded":"2024-01-11T17:57:54.645164+0000","last_fullsized":"2024-01-11T17:57:54.645164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638342+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:57:54.638342+0000","last_peered":"2024-01-11T17:57:54.638342+0000","last_clean":"2024-01-11T17:57:54.638342+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:57:54.638342+0000","last_undegraded":"2024-01-11T17:57:54.638342+0000","last_fullsized":"2024-01-11T17:57:54.638342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644214+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:57:54.644214+0000","last_peered":"2024-01-11T17:57:54.644214+0000","last_clean":"2024-01-11T17:57:54.644214+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:57:54.644214+0000","last_undegraded":"2024-01-11T17:57:54.644214+0000","last_fullsized":"2024-01-11T17:57:54.644214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637270+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:57:54.637270+0000","last_peered":"2024-01-11T17:57:54.637270+0000","last_clean":"2024-01-11T17:57:54.637270+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:57:54.637270+0000","last_undegraded":"2024-01-11T17:57:54.637270+0000","last_fullsized":"2024-01-11T17:57:54.637270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639445+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:57:54.639445+0000","last_peered":"2024-01-11T17:57:54.639445+0000","last_clean":"2024-01-11T17:57:54.639445+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:57:54.639445+0000","last_undegraded":"2024-01-11T17:57:54.639445+0000","last_fullsized":"2024-01-11T17:57:54.639445+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641839+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:57:54.641839+0000","last_peered":"2024-01-11T17:57:54.641839+0000","last_clean":"2024-01-11T17:57:54.641839+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:57:54.641839+0000","last_undegraded":"2024-01-11T17:57:54.641839+0000","last_fullsized":"2024-01-11T17:57:54.641839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644992+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:57:54.644992+0000","last_peered":"2024-01-11T17:57:54.644992+0000","last_clean":"2024-01-11T17:57:54.644992+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:57:54.644992+0000","last_undegraded":"2024-01-11T17:57:54.644992+0000","last_fullsized":"2024-01-11T17:57:54.644992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685921+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:57:26.685921+0000","last_peered":"2024-01-11T17:57:26.685921+0000","last_clean":"2024-01-11T17:57:26.685921+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:57:26.685921+0000","last_undegraded":"2024-01-11T17:57:26.685921+0000","last_fullsized":"2024-01-11T17:57:26.685921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.690057+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:57:26.690057+0000","last_peered":"2024-01-11T17:57:26.690057+0000","last_clean":"2024-01-11T17:57:26.690057+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:57:26.690057+0000","last_undegraded":"2024-01-11T17:57:26.690057+0000","last_fullsized":"2024-01-11T17:57:26.690057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638522+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:57:54.638522+0000","last_peered":"2024-01-11T17:57:54.638522+0000","last_clean":"2024-01-11T17:57:54.638522+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:57:54.638522+0000","last_undegraded":"2024-01-11T17:57:54.638522+0000","last_fullsized":"2024-01-11T17:57:54.638522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346095+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:57:26.346095+0000","last_peered":"2024-01-11T17:57:26.346095+0000","last_clean":"2024-01-11T17:57:26.346095+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:57:26.346095+0000","last_undegraded":"2024-01-11T17:57:26.346095+0000","last_fullsized":"2024-01-11T17:57:26.346095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.412849+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:57:26.412849+0000","last_peered":"2024-01-11T17:57:26.412849+0000","last_clean":"2024-01-11T17:57:26.412849+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:57:26.412849+0000","last_undegraded":"2024-01-11T17:57:26.412849+0000","last_fullsized":"2024-01-11T17:57:26.412849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.640275+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:57:54.640275+0000","last_peered":"2024-01-11T17:57:54.640275+0000","last_clean":"2024-01-11T17:57:54.640275+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:57:54.640275+0000","last_undegraded":"2024-01-11T17:57:54.640275+0000","last_fullsized":"2024-01-11T17:57:54.640275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346280+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:57:26.346280+0000","last_peered":"2024-01-11T17:57:26.346280+0000","last_clean":"2024-01-11T17:57:26.346280+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:57:26.346280+0000","last_undegraded":"2024-01-11T17:57:26.346280+0000","last_fullsized":"2024-01-11T17:57:26.346280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.349086+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:57:26.349086+0000","last_peered":"2024-01-11T17:57:26.349086+0000","last_clean":"2024-01-11T17:57:26.349086+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:57:26.349086+0000","last_undegraded":"2024-01-11T17:57:26.349086+0000","last_fullsized":"2024-01-11T17:57:26.349086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645057+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:57:54.645057+0000","last_peered":"2024-01-11T17:57:54.645057+0000","last_clean":"2024-01-11T17:57:54.645057+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:57:54.645057+0000","last_undegraded":"2024-01-11T17:57:54.645057+0000","last_fullsized":"2024-01-11T17:57:54.645057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346157+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:57:26.346157+0000","last_peered":"2024-01-11T17:57:26.346157+0000","last_clean":"2024-01-11T17:57:26.346157+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:57:26.346157+0000","last_undegraded":"2024-01-11T17:57:26.346157+0000","last_fullsized":"2024-01-11T17:57:26.346157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689779+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:57:26.689779+0000","last_peered":"2024-01-11T17:57:26.689779+0000","last_clean":"2024-01-11T17:57:26.689779+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:57:26.689779+0000","last_undegraded":"2024-01-11T17:57:26.689779+0000","last_fullsized":"2024-01-11T17:57:26.689779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.632027+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:57:54.632027+0000","last_peered":"2024-01-11T17:57:54.632027+0000","last_clean":"2024-01-11T17:57:54.632027+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:57:54.632027+0000","last_undegraded":"2024-01-11T17:57:54.632027+0000","last_fullsized":"2024-01-11T17:57:54.632027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686233+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:57:26.686233+0000","last_peered":"2024-01-11T17:57:26.686233+0000","last_clean":"2024-01-11T17:57:26.686233+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:57:26.686233+0000","last_undegraded":"2024-01-11T17:57:26.686233+0000","last_fullsized":"2024-01-11T17:57:26.686233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:58.463801+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:57:58.463801+0000","last_peered":"2024-01-11T17:57:58.463801+0000","last_clean":"2024-01-11T17:57:58.463801+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:57:58.463801+0000","last_undegraded":"2024-01-11T17:57:58.463801+0000","last_fullsized":"2024-01-11T17:57:58.463801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1459,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.620867+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:57:54.620867+0000","last_peered":"2024-01-11T17:57:54.620867+0000","last_clean":"2024-01-11T17:57:54.620867+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:57:54.620867+0000","last_undegraded":"2024-01-11T17:57:54.620867+0000","last_fullsized":"2024-01-11T17:57:54.620867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686126+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:57:26.686126+0000","last_peered":"2024-01-11T17:57:26.686126+0000","last_clean":"2024-01-11T17:57:26.686126+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:57:26.686126+0000","last_undegraded":"2024-01-11T17:57:26.686126+0000","last_fullsized":"2024-01-11T17:57:26.686126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406841+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:57:26.406841+0000","last_peered":"2024-01-11T17:57:26.406841+0000","last_clean":"2024-01-11T17:57:26.406841+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:57:26.406841+0000","last_undegraded":"2024-01-11T17:57:26.406841+0000","last_fullsized":"2024-01-11T17:57:26.406841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646041+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:57:54.646041+0000","last_peered":"2024-01-11T17:57:54.646041+0000","last_clean":"2024-01-11T17:57:54.646041+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:57:54.646041+0000","last_undegraded":"2024-01-11T17:57:54.646041+0000","last_fullsized":"2024-01-11T17:57:54.646041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.690008+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:57:26.690008+0000","last_peered":"2024-01-11T17:57:26.690008+0000","last_clean":"2024-01-11T17:57:26.690008+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:57:26.690008+0000","last_undegraded":"2024-01-11T17:57:26.690008+0000","last_fullsized":"2024-01-11T17:57:26.690008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406421+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:57:26.406421+0000","last_peered":"2024-01-11T17:57:26.406421+0000","last_clean":"2024-01-11T17:57:26.406421+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:57:26.406421+0000","last_undegraded":"2024-01-11T17:57:26.406421+0000","last_fullsized":"2024-01-11T17:57:26.406421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646879+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:57:54.646879+0000","last_peered":"2024-01-11T17:57:54.646879+0000","last_clean":"2024-01-11T17:57:54.646879+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:57:54.646879+0000","last_undegraded":"2024-01-11T17:57:54.646879+0000","last_fullsized":"2024-01-11T17:57:54.646879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.345977+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:57:26.345977+0000","last_peered":"2024-01-11T17:57:26.345977+0000","last_clean":"2024-01-11T17:57:26.345977+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:57:26.345977+0000","last_undegraded":"2024-01-11T17:57:26.345977+0000","last_fullsized":"2024-01-11T17:57:26.345977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409651+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:57:26.409651+0000","last_peered":"2024-01-11T17:57:26.409651+0000","last_clean":"2024-01-11T17:57:26.409651+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:57:26.409651+0000","last_undegraded":"2024-01-11T17:57:26.409651+0000","last_fullsized":"2024-01-11T17:57:26.409651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638052+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:57:54.638052+0000","last_peered":"2024-01-11T17:57:54.638052+0000","last_clean":"2024-01-11T17:57:54.638052+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:57:54.638052+0000","last_undegraded":"2024-01-11T17:57:54.638052+0000","last_fullsized":"2024-01-11T17:57:54.638052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:28.039741+0000","last_change":"2024-01-11T17:57:28.039741+0000","last_active":"2024-01-11T17:57:28.039741+0000","last_peered":"2024-01-11T17:57:28.039741+0000","last_clean":"2024-01-11T17:57:28.039741+0000","last_became_active":"2024-01-11T17:57:28.039342+0000","last_became_peered":"2024-01-11T17:57:28.039342+0000","last_unstale":"2024-01-11T17:57:28.039741+0000","last_undegraded":"2024-01-11T17:57:28.039741+0000","last_fullsized":"2024-01-11T17:57:28.039741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:17.687299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,10],"acting":[3,7,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:58.463276+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:57:58.463276+0000","last_peered":"2024-01-11T17:57:58.463276+0000","last_clean":"2024-01-11T17:57:58.463276+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:57:58.463276+0000","last_undegraded":"2024-01-11T17:57:58.463276+0000","last_fullsized":"2024-01-11T17:57:58.463276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639525+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:57:54.639525+0000","last_peered":"2024-01-11T17:57:54.639525+0000","last_clean":"2024-01-11T17:57:54.639525+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:57:54.639525+0000","last_undegraded":"2024-01-11T17:57:54.639525+0000","last_fullsized":"2024-01-11T17:57:54.639525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689484+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:57:26.689484+0000","last_peered":"2024-01-11T17:57:26.689484+0000","last_clean":"2024-01-11T17:57:26.689484+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:57:26.689484+0000","last_undegraded":"2024-01-11T17:57:26.689484+0000","last_fullsized":"2024-01-11T17:57:26.689484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415699+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:57:26.415699+0000","last_peered":"2024-01-11T17:57:26.415699+0000","last_clean":"2024-01-11T17:57:26.415699+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:57:26.415699+0000","last_undegraded":"2024-01-11T17:57:26.415699+0000","last_fullsized":"2024-01-11T17:57:26.415699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636540+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:57:54.636540+0000","last_peered":"2024-01-11T17:57:54.636540+0000","last_clean":"2024-01-11T17:57:54.636540+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:57:54.636540+0000","last_undegraded":"2024-01-11T17:57:54.636540+0000","last_fullsized":"2024-01-11T17:57:54.636540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346392+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:57:26.346392+0000","last_peered":"2024-01-11T17:57:26.346392+0000","last_clean":"2024-01-11T17:57:26.346392+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:57:26.346392+0000","last_undegraded":"2024-01-11T17:57:26.346392+0000","last_fullsized":"2024-01-11T17:57:26.346392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410334+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:57:26.410334+0000","last_peered":"2024-01-11T17:57:26.410334+0000","last_clean":"2024-01-11T17:57:26.410334+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:57:26.410334+0000","last_undegraded":"2024-01-11T17:57:26.410334+0000","last_fullsized":"2024-01-11T17:57:26.410334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639586+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:57:54.639586+0000","last_peered":"2024-01-11T17:57:54.639586+0000","last_clean":"2024-01-11T17:57:54.639586+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:57:54.639586+0000","last_undegraded":"2024-01-11T17:57:54.639586+0000","last_fullsized":"2024-01-11T17:57:54.639586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689581+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:57:26.689581+0000","last_peered":"2024-01-11T17:57:26.689581+0000","last_clean":"2024-01-11T17:57:26.689581+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:57:26.689581+0000","last_undegraded":"2024-01-11T17:57:26.689581+0000","last_fullsized":"2024-01-11T17:57:26.689581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'37","reported_seq":242,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.464956+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:57:59.464956+0000","last_peered":"2024-01-11T17:57:59.464956+0000","last_clean":"2024-01-11T17:57:59.464956+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:57:59.464956+0000","last_undegraded":"2024-01-11T17:57:59.464956+0000","last_fullsized":"2024-01-11T17:57:59.464956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69282,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":184,"num_read_kb":161,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623678+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:57:54.623678+0000","last_peered":"2024-01-11T17:57:54.623678+0000","last_clean":"2024-01-11T17:57:54.623678+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:57:54.623678+0000","last_undegraded":"2024-01-11T17:57:54.623678+0000","last_fullsized":"2024-01-11T17:57:54.623678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410861+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:57:26.410861+0000","last_peered":"2024-01-11T17:57:26.410861+0000","last_clean":"2024-01-11T17:57:26.410861+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:57:26.410861+0000","last_undegraded":"2024-01-11T17:57:26.410861+0000","last_fullsized":"2024-01-11T17:57:26.410861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685932+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:57:26.685932+0000","last_peered":"2024-01-11T17:57:26.685932+0000","last_clean":"2024-01-11T17:57:26.685932+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:57:26.685932+0000","last_undegraded":"2024-01-11T17:57:26.685932+0000","last_fullsized":"2024-01-11T17:57:26.685932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644250+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:57:54.644250+0000","last_peered":"2024-01-11T17:57:54.644250+0000","last_clean":"2024-01-11T17:57:54.644250+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:57:54.644250+0000","last_undegraded":"2024-01-11T17:57:54.644250+0000","last_fullsized":"2024-01-11T17:57:54.644250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689807+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:57:26.689807+0000","last_peered":"2024-01-11T17:57:26.689807+0000","last_clean":"2024-01-11T17:57:26.689807+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:57:26.689807+0000","last_undegraded":"2024-01-11T17:57:26.689807+0000","last_fullsized":"2024-01-11T17:57:26.689807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410045+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:57:26.410045+0000","last_peered":"2024-01-11T17:57:26.410045+0000","last_clean":"2024-01-11T17:57:26.410045+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:57:26.410045+0000","last_undegraded":"2024-01-11T17:57:26.410045+0000","last_fullsized":"2024-01-11T17:57:26.410045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641093+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:57:54.641093+0000","last_peered":"2024-01-11T17:57:54.641093+0000","last_clean":"2024-01-11T17:57:54.641093+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:57:54.641093+0000","last_undegraded":"2024-01-11T17:57:54.641093+0000","last_fullsized":"2024-01-11T17:57:54.641093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348785+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:57:26.348785+0000","last_peered":"2024-01-11T17:57:26.348785+0000","last_clean":"2024-01-11T17:57:26.348785+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:57:26.348785+0000","last_undegraded":"2024-01-11T17:57:26.348785+0000","last_fullsized":"2024-01-11T17:57:26.348785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.465117+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:57:59.465117+0000","last_peered":"2024-01-11T17:57:59.465117+0000","last_clean":"2024-01-11T17:57:59.465117+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:57:59.465117+0000","last_undegraded":"2024-01-11T17:57:59.465117+0000","last_fullsized":"2024-01-11T17:57:59.465117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636390+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:57:54.636390+0000","last_peered":"2024-01-11T17:57:54.636390+0000","last_clean":"2024-01-11T17:57:54.636390+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:57:54.636390+0000","last_undegraded":"2024-01-11T17:57:54.636390+0000","last_fullsized":"2024-01-11T17:57:54.636390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406884+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:57:26.406884+0000","last_peered":"2024-01-11T17:57:26.406884+0000","last_clean":"2024-01-11T17:57:26.406884+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:57:26.406884+0000","last_undegraded":"2024-01-11T17:57:26.406884+0000","last_fullsized":"2024-01-11T17:57:26.406884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348419+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:57:26.348419+0000","last_peered":"2024-01-11T17:57:26.348419+0000","last_clean":"2024-01-11T17:57:26.348419+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:57:26.348419+0000","last_undegraded":"2024-01-11T17:57:26.348419+0000","last_fullsized":"2024-01-11T17:57:26.348419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.629482+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:57:54.629482+0000","last_peered":"2024-01-11T17:57:54.629482+0000","last_clean":"2024-01-11T17:57:54.629482+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:57:54.629482+0000","last_undegraded":"2024-01-11T17:57:54.629482+0000","last_fullsized":"2024-01-11T17:57:54.629482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409801+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:57:26.409801+0000","last_peered":"2024-01-11T17:57:26.409801+0000","last_clean":"2024-01-11T17:57:26.409801+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:57:26.409801+0000","last_undegraded":"2024-01-11T17:57:26.409801+0000","last_fullsized":"2024-01-11T17:57:26.409801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348962+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:57:26.348962+0000","last_peered":"2024-01-11T17:57:26.348962+0000","last_clean":"2024-01-11T17:57:26.348962+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:57:26.348962+0000","last_undegraded":"2024-01-11T17:57:26.348962+0000","last_fullsized":"2024-01-11T17:57:26.348962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630954+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:57:54.630954+0000","last_peered":"2024-01-11T17:57:54.630954+0000","last_clean":"2024-01-11T17:57:54.630954+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:57:54.630954+0000","last_undegraded":"2024-01-11T17:57:54.630954+0000","last_fullsized":"2024-01-11T17:57:54.630954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348611+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:57:26.348611+0000","last_peered":"2024-01-11T17:57:26.348611+0000","last_clean":"2024-01-11T17:57:26.348611+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:57:26.348611+0000","last_undegraded":"2024-01-11T17:57:26.348611+0000","last_fullsized":"2024-01-11T17:57:26.348611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689942+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:57:26.689942+0000","last_peered":"2024-01-11T17:57:26.689942+0000","last_clean":"2024-01-11T17:57:26.689942+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:57:26.689942+0000","last_undegraded":"2024-01-11T17:57:26.689942+0000","last_fullsized":"2024-01-11T17:57:26.689942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643064+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:57:54.643064+0000","last_peered":"2024-01-11T17:57:54.643064+0000","last_clean":"2024-01-11T17:57:54.643064+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:57:54.643064+0000","last_undegraded":"2024-01-11T17:57:54.643064+0000","last_fullsized":"2024-01-11T17:57:54.643064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406557+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:57:26.406557+0000","last_peered":"2024-01-11T17:57:26.406557+0000","last_clean":"2024-01-11T17:57:26.406557+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:57:26.406557+0000","last_undegraded":"2024-01-11T17:57:26.406557+0000","last_fullsized":"2024-01-11T17:57:26.406557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415027+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:57:26.415027+0000","last_peered":"2024-01-11T17:57:26.415027+0000","last_clean":"2024-01-11T17:57:26.415027+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:57:26.415027+0000","last_undegraded":"2024-01-11T17:57:26.415027+0000","last_fullsized":"2024-01-11T17:57:26.415027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.632525+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:57:54.632525+0000","last_peered":"2024-01-11T17:57:54.632525+0000","last_clean":"2024-01-11T17:57:54.632525+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:57:54.632525+0000","last_undegraded":"2024-01-11T17:57:54.632525+0000","last_fullsized":"2024-01-11T17:57:54.632525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406685+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:57:26.406685+0000","last_peered":"2024-01-11T17:57:26.406685+0000","last_clean":"2024-01-11T17:57:26.406685+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:57:26.406685+0000","last_undegraded":"2024-01-11T17:57:26.406685+0000","last_fullsized":"2024-01-11T17:57:26.406685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631689+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:57:54.631689+0000","last_peered":"2024-01-11T17:57:54.631689+0000","last_clean":"2024-01-11T17:57:54.631689+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:57:54.631689+0000","last_undegraded":"2024-01-11T17:57:54.631689+0000","last_fullsized":"2024-01-11T17:57:54.631689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.464545+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:57:59.464545+0000","last_peered":"2024-01-11T17:57:59.464545+0000","last_clean":"2024-01-11T17:57:59.464545+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:57:59.464545+0000","last_undegraded":"2024-01-11T17:57:59.464545+0000","last_fullsized":"2024-01-11T17:57:59.464545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415989+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:57:26.415989+0000","last_peered":"2024-01-11T17:57:26.415989+0000","last_clean":"2024-01-11T17:57:26.415989+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:57:26.415989+0000","last_undegraded":"2024-01-11T17:57:26.415989+0000","last_fullsized":"2024-01-11T17:57:26.415989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.647162+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:57:54.647162+0000","last_peered":"2024-01-11T17:57:54.647162+0000","last_clean":"2024-01-11T17:57:54.647162+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:57:54.647162+0000","last_undegraded":"2024-01-11T17:57:54.647162+0000","last_fullsized":"2024-01-11T17:57:54.647162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346338+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:57:26.346338+0000","last_peered":"2024-01-11T17:57:26.346338+0000","last_clean":"2024-01-11T17:57:26.346338+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:57:26.346338+0000","last_undegraded":"2024-01-11T17:57:26.346338+0000","last_fullsized":"2024-01-11T17:57:26.346338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414453+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:57:26.414453+0000","last_peered":"2024-01-11T17:57:26.414453+0000","last_clean":"2024-01-11T17:57:26.414453+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:57:26.414453+0000","last_undegraded":"2024-01-11T17:57:26.414453+0000","last_fullsized":"2024-01-11T17:57:26.414453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410347+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:57:26.410347+0000","last_peered":"2024-01-11T17:57:26.410347+0000","last_clean":"2024-01-11T17:57:26.410347+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:57:26.410347+0000","last_undegraded":"2024-01-11T17:57:26.410347+0000","last_fullsized":"2024-01-11T17:57:26.410347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.634064+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:57:54.634064+0000","last_peered":"2024-01-11T17:57:54.634064+0000","last_clean":"2024-01-11T17:57:54.634064+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:57:54.634064+0000","last_undegraded":"2024-01-11T17:57:54.634064+0000","last_fullsized":"2024-01-11T17:57:54.634064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414488+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:57:26.414488+0000","last_peered":"2024-01-11T17:57:26.414488+0000","last_clean":"2024-01-11T17:57:26.414488+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:57:26.414488+0000","last_undegraded":"2024-01-11T17:57:26.414488+0000","last_fullsized":"2024-01-11T17:57:26.414488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689755+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:57:26.689755+0000","last_peered":"2024-01-11T17:57:26.689755+0000","last_clean":"2024-01-11T17:57:26.689755+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:57:26.689755+0000","last_undegraded":"2024-01-11T17:57:26.689755+0000","last_fullsized":"2024-01-11T17:57:26.689755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348947+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:57:26.348947+0000","last_peered":"2024-01-11T17:57:26.348947+0000","last_clean":"2024-01-11T17:57:26.348947+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:57:26.348947+0000","last_undegraded":"2024-01-11T17:57:26.348947+0000","last_fullsized":"2024-01-11T17:57:26.348947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633371+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:57:54.633371+0000","last_peered":"2024-01-11T17:57:54.633371+0000","last_clean":"2024-01-11T17:57:54.633371+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:57:54.633371+0000","last_undegraded":"2024-01-11T17:57:54.633371+0000","last_fullsized":"2024-01-11T17:57:54.633371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.411976+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:57:26.411976+0000","last_peered":"2024-01-11T17:57:26.411976+0000","last_clean":"2024-01-11T17:57:26.411976+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:57:26.411976+0000","last_undegraded":"2024-01-11T17:57:26.411976+0000","last_fullsized":"2024-01-11T17:57:26.411976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:27.351805+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:57:27.351805+0000","last_peered":"2024-01-11T17:57:27.351805+0000","last_clean":"2024-01-11T17:57:27.351805+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:57:27.351805+0000","last_undegraded":"2024-01-11T17:57:27.351805+0000","last_fullsized":"2024-01-11T17:57:27.351805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638090+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:57:54.638090+0000","last_peered":"2024-01-11T17:57:54.638090+0000","last_clean":"2024-01-11T17:57:54.638090+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:57:54.638090+0000","last_undegraded":"2024-01-11T17:57:54.638090+0000","last_fullsized":"2024-01-11T17:57:54.638090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685669+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:57:26.685669+0000","last_peered":"2024-01-11T17:57:26.685669+0000","last_clean":"2024-01-11T17:57:26.685669+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:57:26.685669+0000","last_undegraded":"2024-01-11T17:57:26.685669+0000","last_fullsized":"2024-01-11T17:57:26.685669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.347046+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:57:26.347046+0000","last_peered":"2024-01-11T17:57:26.347046+0000","last_clean":"2024-01-11T17:57:26.347046+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:57:26.347046+0000","last_undegraded":"2024-01-11T17:57:26.347046+0000","last_fullsized":"2024-01-11T17:57:26.347046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.642913+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:57:54.642913+0000","last_peered":"2024-01-11T17:57:54.642913+0000","last_clean":"2024-01-11T17:57:54.642913+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:57:54.642913+0000","last_undegraded":"2024-01-11T17:57:54.642913+0000","last_fullsized":"2024-01-11T17:57:54.642913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346710+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:57:26.346710+0000","last_peered":"2024-01-11T17:57:26.346710+0000","last_clean":"2024-01-11T17:57:26.346710+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:57:26.346710+0000","last_undegraded":"2024-01-11T17:57:26.346710+0000","last_fullsized":"2024-01-11T17:57:26.346710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415555+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:57:26.415555+0000","last_peered":"2024-01-11T17:57:26.415555+0000","last_clean":"2024-01-11T17:57:26.415555+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:57:26.415555+0000","last_undegraded":"2024-01-11T17:57:26.415555+0000","last_fullsized":"2024-01-11T17:57:26.415555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.598022+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:57:54.598022+0000","last_peered":"2024-01-11T17:57:54.598022+0000","last_clean":"2024-01-11T17:57:54.598022+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:57:54.598022+0000","last_undegraded":"2024-01-11T17:57:54.598022+0000","last_fullsized":"2024-01-11T17:57:54.598022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689961+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:57:26.689961+0000","last_peered":"2024-01-11T17:57:26.689961+0000","last_clean":"2024-01-11T17:57:26.689961+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:57:26.689961+0000","last_undegraded":"2024-01-11T17:57:26.689961+0000","last_fullsized":"2024-01-11T17:57:26.689961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.412881+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:57:26.412881+0000","last_peered":"2024-01-11T17:57:26.412881+0000","last_clean":"2024-01-11T17:57:26.412881+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:57:26.412881+0000","last_undegraded":"2024-01-11T17:57:26.412881+0000","last_fullsized":"2024-01-11T17:57:26.412881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.635273+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:57:54.635273+0000","last_peered":"2024-01-11T17:57:54.635273+0000","last_clean":"2024-01-11T17:57:54.635273+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:57:54.635273+0000","last_undegraded":"2024-01-11T17:57:54.635273+0000","last_fullsized":"2024-01-11T17:57:54.635273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346278+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:57:26.346278+0000","last_peered":"2024-01-11T17:57:26.346278+0000","last_clean":"2024-01-11T17:57:26.346278+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:57:26.346278+0000","last_undegraded":"2024-01-11T17:57:26.346278+0000","last_fullsized":"2024-01-11T17:57:26.346278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689783+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:57:26.689783+0000","last_peered":"2024-01-11T17:57:26.689783+0000","last_clean":"2024-01-11T17:57:26.689783+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:57:26.689783+0000","last_undegraded":"2024-01-11T17:57:26.689783+0000","last_fullsized":"2024-01-11T17:57:26.689783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641886+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:57:54.641886+0000","last_peered":"2024-01-11T17:57:54.641886+0000","last_clean":"2024-01-11T17:57:54.641886+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:57:54.641886+0000","last_undegraded":"2024-01-11T17:57:54.641886+0000","last_fullsized":"2024-01-11T17:57:54.641886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346709+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:57:26.346709+0000","last_peered":"2024-01-11T17:57:26.346709+0000","last_clean":"2024-01-11T17:57:26.346709+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:57:26.346709+0000","last_undegraded":"2024-01-11T17:57:26.346709+0000","last_fullsized":"2024-01-11T17:57:26.346709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348623+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:57:26.348623+0000","last_peered":"2024-01-11T17:57:26.348623+0000","last_clean":"2024-01-11T17:57:26.348623+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:57:26.348623+0000","last_undegraded":"2024-01-11T17:57:26.348623+0000","last_fullsized":"2024-01-11T17:57:26.348623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633587+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:57:54.633587+0000","last_peered":"2024-01-11T17:57:54.633587+0000","last_clean":"2024-01-11T17:57:54.633587+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:57:54.633587+0000","last_undegraded":"2024-01-11T17:57:54.633587+0000","last_fullsized":"2024-01-11T17:57:54.633587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414998+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:57:26.414998+0000","last_peered":"2024-01-11T17:57:26.414998+0000","last_clean":"2024-01-11T17:57:26.414998+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:57:26.414998+0000","last_undegraded":"2024-01-11T17:57:26.414998+0000","last_fullsized":"2024-01-11T17:57:26.414998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686294+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:57:26.686294+0000","last_peered":"2024-01-11T17:57:26.686294+0000","last_clean":"2024-01-11T17:57:26.686294+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:57:26.686294+0000","last_undegraded":"2024-01-11T17:57:26.686294+0000","last_fullsized":"2024-01-11T17:57:26.686294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627483+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:57:54.627483+0000","last_peered":"2024-01-11T17:57:54.627483+0000","last_clean":"2024-01-11T17:57:54.627483+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:57:54.627483+0000","last_undegraded":"2024-01-11T17:57:54.627483+0000","last_fullsized":"2024-01-11T17:57:54.627483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.345942+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:57:26.345942+0000","last_peered":"2024-01-11T17:57:26.345942+0000","last_clean":"2024-01-11T17:57:26.345942+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:57:26.345942+0000","last_undegraded":"2024-01-11T17:57:26.345942+0000","last_fullsized":"2024-01-11T17:57:26.345942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410583+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:57:26.410583+0000","last_peered":"2024-01-11T17:57:26.410583+0000","last_clean":"2024-01-11T17:57:26.410583+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:57:26.410583+0000","last_undegraded":"2024-01-11T17:57:26.410583+0000","last_fullsized":"2024-01-11T17:57:26.410583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641067+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:57:54.641067+0000","last_peered":"2024-01-11T17:57:54.641067+0000","last_clean":"2024-01-11T17:57:54.641067+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:57:54.641067+0000","last_undegraded":"2024-01-11T17:57:54.641067+0000","last_fullsized":"2024-01-11T17:57:54.641067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409577+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:57:26.409577+0000","last_peered":"2024-01-11T17:57:26.409577+0000","last_clean":"2024-01-11T17:57:26.409577+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:57:26.409577+0000","last_undegraded":"2024-01-11T17:57:26.409577+0000","last_fullsized":"2024-01-11T17:57:26.409577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406896+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:57:26.406896+0000","last_peered":"2024-01-11T17:57:26.406896+0000","last_clean":"2024-01-11T17:57:26.406896+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:57:26.406896+0000","last_undegraded":"2024-01-11T17:57:26.406896+0000","last_fullsized":"2024-01-11T17:57:26.406896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630352+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:57:54.630352+0000","last_peered":"2024-01-11T17:57:54.630352+0000","last_clean":"2024-01-11T17:57:54.630352+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:57:54.630352+0000","last_undegraded":"2024-01-11T17:57:54.630352+0000","last_fullsized":"2024-01-11T17:57:54.630352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633236+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:57:54.633236+0000","last_peered":"2024-01-11T17:57:54.633236+0000","last_clean":"2024-01-11T17:57:54.633236+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:57:54.633236+0000","last_undegraded":"2024-01-11T17:57:54.633236+0000","last_fullsized":"2024-01-11T17:57:54.633236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.634289+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:57:54.634289+0000","last_peered":"2024-01-11T17:57:54.634289+0000","last_clean":"2024-01-11T17:57:54.634289+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:57:54.634289+0000","last_undegraded":"2024-01-11T17:57:54.634289+0000","last_fullsized":"2024-01-11T17:57:54.634289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637772+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:57:54.637772+0000","last_peered":"2024-01-11T17:57:54.637772+0000","last_clean":"2024-01-11T17:57:54.637772+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:57:54.637772+0000","last_undegraded":"2024-01-11T17:57:54.637772+0000","last_fullsized":"2024-01-11T17:57:54.637772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645709+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:57:54.645709+0000","last_peered":"2024-01-11T17:57:54.645709+0000","last_clean":"2024-01-11T17:57:54.645709+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:57:54.645709+0000","last_undegraded":"2024-01-11T17:57:54.645709+0000","last_fullsized":"2024-01-11T17:57:54.645709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643205+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:57:54.643205+0000","last_peered":"2024-01-11T17:57:54.643205+0000","last_clean":"2024-01-11T17:57:54.643205+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:57:54.643205+0000","last_undegraded":"2024-01-11T17:57:54.643205+0000","last_fullsized":"2024-01-11T17:57:54.643205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630113+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:57:54.630113+0000","last_peered":"2024-01-11T17:57:54.630113+0000","last_clean":"2024-01-11T17:57:54.630113+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:57:54.630113+0000","last_undegraded":"2024-01-11T17:57:54.630113+0000","last_fullsized":"2024-01-11T17:57:54.630113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638641+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:57:54.638641+0000","last_peered":"2024-01-11T17:57:54.638641+0000","last_clean":"2024-01-11T17:57:54.638641+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:57:54.638641+0000","last_undegraded":"2024-01-11T17:57:54.638641+0000","last_fullsized":"2024-01-11T17:57:54.638641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627381+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:57:54.627381+0000","last_peered":"2024-01-11T17:57:54.627381+0000","last_clean":"2024-01-11T17:57:54.627381+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:57:54.627381+0000","last_undegraded":"2024-01-11T17:57:54.627381+0000","last_fullsized":"2024-01-11T17:57:54.627381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646027+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:57:54.646027+0000","last_peered":"2024-01-11T17:57:54.646027+0000","last_clean":"2024-01-11T17:57:54.646027+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:57:54.646027+0000","last_undegraded":"2024-01-11T17:57:54.646027+0000","last_fullsized":"2024-01-11T17:57:54.646027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639605+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:57:54.639605+0000","last_peered":"2024-01-11T17:57:54.639605+0000","last_clean":"2024-01-11T17:57:54.639605+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:57:54.639605+0000","last_undegraded":"2024-01-11T17:57:54.639605+0000","last_fullsized":"2024-01-11T17:57:54.639605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.647432+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:57:54.647432+0000","last_peered":"2024-01-11T17:57:54.647432+0000","last_clean":"2024-01-11T17:57:54.647432+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:57:54.647432+0000","last_undegraded":"2024-01-11T17:57:54.647432+0000","last_fullsized":"2024-01-11T17:57:54.647432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644914+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:57:54.644914+0000","last_peered":"2024-01-11T17:57:54.644914+0000","last_clean":"2024-01-11T17:57:54.644914+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:57:54.644914+0000","last_undegraded":"2024-01-11T17:57:54.644914+0000","last_fullsized":"2024-01-11T17:57:54.644914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623596+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:57:54.623596+0000","last_peered":"2024-01-11T17:57:54.623596+0000","last_clean":"2024-01-11T17:57:54.623596+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:57:54.623596+0000","last_undegraded":"2024-01-11T17:57:54.623596+0000","last_fullsized":"2024-01-11T17:57:54.623596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74260,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":553,"num_read_kb":357,"num_write":168,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":222780,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":491633,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.497,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.495,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89800000000000002,"5min":1.109,"15min":1.109},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93600000000000005,"5min":1.048,"15min":1.048},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.53400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.046,"5min":1.405,"15min":1.405},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.748,"15min":0.74299999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93600000000000005,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.755},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88,"15min":0.878},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0680000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88,"15min":0.88100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0329999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.02,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.91500000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.872}]},{"osd":9,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1779999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.1120000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.94599999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.71199999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1279999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.95299999999999996,"15min":0.94699999999999995},"min":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.139,"5min":1.355,"15min":1.355},"last":0.93000000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":1148714,"data_compressed":61367,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.623}]},{"osd":3,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.751},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.32300000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.94999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.254,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.097,"15min":1.097},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.117,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69499999999999995}]},{"osd":9,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.099,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0529999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.73499999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.93899999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.04,"5min":1.244,"15min":1.244},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.075,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.97899999999999998}]}]},{"osd":9,"up_from":63,"seq":270582939688,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":495965,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81200000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.73999999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75700000000000001,"15min":0.77100000000000002},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90500000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86199999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.75700000000000001,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79700000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90600000000000003,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96199999999999997,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.198,"15min":1.198},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93899999999999995,"5min":1.224,"15min":1.224},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88900000000000001,"15min":0.93100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.139,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.91900000000000004,"15min":0.96699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.91900000000000004,"15min":0.97299999999999998},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0069999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93300000000000005,"15min":0.96899999999999997},"min":{"1min":0.68500000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.022,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.94799999999999995},"min":{"1min":0.622,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96099999999999997,"5min":1.284,"15min":1.284},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0309999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85499999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":491789,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.38300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.624}]},{"osd":3,"last update":"Thu Jan 11 17:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":4,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":1.097},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.998,"5min":1,"15min":1},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.877},{"interface":"front","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.83399999999999996},"min":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.135,"5min":1.163,"15min":1.163},"last":0.72699999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89600000000000002,"15min":0.89800000000000002},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.095,"5min":1.125,"15min":1.125},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77600000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.88700000000000001,"15min":0.871},"min":{"1min":0.624,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.81299999999999994,"15min":0.79500000000000004},"min":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.85599999999999998}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1088675,"data_compressed":59744,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.625},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.626,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90300000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81499999999999995,"5min":1.137,"15min":1.137},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.56000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.64500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98699999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":1.04,"15min":1.04},"last":0.53700000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81499999999999995,"15min":0.80300000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.018,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81299999999999994,"15min":0.81200000000000006},"min":{"1min":0.68200000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96899999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.95199999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:57:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0600000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80500000000000005,"15min":0.79600000000000004},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.82999999999999996},"min":{"1min":0.79500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.92500000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0129999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":487045,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.149,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":2.149,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.753,"15min":0.71499999999999997},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.085,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66300000000000003,"15min":0.61499999999999999},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.907,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.161,"15min":1.161},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.72,"5min":2.645,"15min":2.645},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.8029999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.627}]},{"osd":4,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1869999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.84699999999999998,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5349999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.75}]},{"osd":5,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.9770000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.77000000000000002,"15min":0.74099999999999999},"min":{"1min":0.57499999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0819999999999999,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.91600000000000004,"15min":0.90100000000000002},"min":{"1min":0.68799999999999994,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.79700000000000004,"15min":0.75600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.91200000000000003,"15min":0.89500000000000002},"min":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4890000000000001,"5min":2.282,"15min":2.282},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.86699999999999999,"15min":0.85099999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.014,"15min":1.028},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.577,"5min":3.044,"15min":3.044},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.91900000000000004},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2080000000000002,"5min":2.415,"15min":2.415},"last":0.96599999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.92700000000000005,"15min":0.89500000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.093,"15min":1.095},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.2509999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.024}]},{"osd":11,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.98999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":556237,"data_compressed":25274,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.436,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91900000000000004,"5min":1.077,"15min":1.077},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.46200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.98899999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59399999999999997,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61399999999999999,"15min":0.55500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.54200000000000004},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.97299999999999998,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88700000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.46300000000000002},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":1.02,"15min":1.02},"last":0.753}]},{"osd":5,"last update":"Thu Jan 11 17:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94399999999999995,"5min":1.012,"15min":1.012},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68500000000000005,"15min":0.622},"min":{"1min":0.54300000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":1.048,"15min":1.048},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0600000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.53600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81599999999999995,"5min":1.048,"15min":1.048},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.64000000000000001},"min":{"1min":0.59699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.048,"15min":1.048},"last":0.875}]},{"osd":8,"last update":"Thu Jan 11 17:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.745,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0089999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86199999999999999,"15min":0.86399999999999999},"min":{"1min":0.58299999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0880000000000001,"5min":1.21,"15min":1.21},"last":0.95599999999999996}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":488433,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.66600000000000004,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.911,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.502,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.61499999999999999},"min":{"1min":0.55800000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.85799999999999998,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69599999999999995,"15min":0.59599999999999997},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.087,"5min":3.508,"15min":3.508},"last":0.438}]},{"osd":3,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72299999999999998,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3359999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.77400000000000002,"15min":0.65200000000000002},"min":{"1min":0.56200000000000006,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.2320000000000002,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.747}]},{"osd":5,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73399999999999999,"15min":0.68700000000000006},"min":{"1min":0.56899999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.024,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.72099999999999997,"15min":0.64600000000000002},"min":{"1min":0.58299999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2609999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.123},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.67300000000000004},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":3.1019999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.68500000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5629999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88,"15min":0.84399999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1879999999999999,"5min":3.198,"15min":3.198},"last":0.77200000000000002}]},{"osd":9,"last update":"Thu Jan 11 17:57:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.95099999999999996},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.423,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97199999999999998,"15min":0.95899999999999996},"min":{"1min":0.72699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.2390000000000001,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.81599999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.75800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0630000000000002,"5min":3.0630000000000002,"15min":3.0630000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.135,"15min":1.145},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":3.4199999999999999,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.0189999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.79000000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5429999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.022,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.339,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":1.056}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":558424,"data_compressed":25274,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.745},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83099999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.96199999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.65900000000000003,"15min":0.56499999999999995},"min":{"1min":0.51800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1220000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0249999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.022,"15min":1.022},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.61099999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.74399999999999999,"15min":0.71199999999999997},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.76100000000000001}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.78000000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98599999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.87}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.129},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.73499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0469999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78700000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":487239,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.40100000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.249},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.46700000000000003,"15min":0.42599999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.623,"15min":0.53700000000000003},"min":{"1min":0.56299999999999994,"5min":0.308,"15min":0.308},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.64000000000000001,"15min":0.55400000000000005},"min":{"1min":0.59099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65500000000000003,"15min":0.60499999999999998},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.628},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.56699999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69399999999999995,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1599999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.72999999999999998,"15min":0.64200000000000002},"min":{"1min":0.64300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3680000000000001,"5min":1.792,"15min":1.792},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.68600000000000005},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1120000000000001,"5min":1.821,"15min":1.821},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.60799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0660000000000001,"5min":1.379,"15min":1.379},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.86199999999999999,"15min":0.84999999999999998},"min":{"1min":0.66300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.93000000000000005},"min":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3069999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1579999999999999,"5min":1.52,"15min":1.52},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.119,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.02,"15min":1.036},"min":{"1min":0.77500000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2330000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.876}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":492226,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.379},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.438,"15min":0.39500000000000002},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.745,"15min":0.745},"last":0.5}]},{"osd":2,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.76900000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94599999999999995,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.877}]},{"osd":3,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":1.034,"15min":1.034},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.004,"15min":1.004},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68899999999999995,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95699999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66300000000000003,"15min":0.60099999999999998},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.129,"15min":1.129},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69499999999999995,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86499999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89000000000000001,"5min":1.163,"15min":1.163},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.745}]},{"osd":8,"last update":"Thu Jan 11 17:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94099999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.61299999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.97099999999999997,"5min":1.238,"15min":1.238},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.878,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.92400000000000004},"min":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.014,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.629,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.95299999999999996,"5min":1.248,"15min":1.248},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.84899999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.92500000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84999999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1083732,"data_compressed":59744,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59099999999999997,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82599999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45600000000000002,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.45900000000000002},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65500000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":1.548,"15min":1.548},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83099999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90400000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.22,"15min":1.22},"last":0.755},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.91300000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95199999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.0309999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":1.097,"15min":1.097},"last":0.96099999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79700000000000004,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0189999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.012}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":69282,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71469,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71391,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:58:02.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T17:58:03.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 -- 172.21.15.7:0/3472260073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0102780 msgr2=0x7fb3b0102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/3472260073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0102780 0x7fb3b0102be0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009920 tx=0x7fb3a0009c30 comp rx=0 tx=0).stop 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 -- 172.21.15.7:0/3472260073 shutdown_connections 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/3472260073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0101590 0x7fb3b01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/3472260073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0103120 0x7fb3b01099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/3472260073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0102780 0x7fb3b0102be0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 -- 172.21.15.7:0/3472260073 >> 172.21.15.7:0/3472260073 conn(0x7fb3b00fcb00 msgr2=0x7fb3b00fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 -- 172.21.15.7:0/3472260073 shutdown_connections 2024-01-11T17:58:03.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.183+0000 7fb3b79ff700 1 -- 172.21.15.7:0/3472260073 wait complete. 2024-01-11T17:58:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 Processor -- start 2024-01-11T17:58:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 -- start start 2024-01-11T17:58:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0101590 0x7fb3b01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 0x7fb3b0199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 0x7fb3b01a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01a1320 con 0x7fb3b0103120 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01a1460 con 0x7fb3b0101590 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.184+0000 7fb3b79ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01a15f0 con 0x7fb3b0102780 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 0x7fb3b01a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b579b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0101590 0x7fb3b01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b4f9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 0x7fb3b0199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 0x7fb3b01a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46734/0 (socket says 172.21.15.7:46734) 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 -- 172.21.15.7:0/2489517703 learned_addr learned my addr 172.21.15.7:0/2489517703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b4f9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 0x7fb3b0199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39710/0 (socket says 172.21.15.7:39710) 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b579b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0101590 0x7fb3b01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49226/0 (socket says 172.21.15.7:49226) 2024-01-11T17:58:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 -- 172.21.15.7:0/2489517703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 msgr2=0x7fb3b0199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 0x7fb3b0199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 -- 172.21.15.7:0/2489517703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0101590 msgr2=0x7fb3b01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0101590 0x7fb3b01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 -- 172.21.15.7:0/2489517703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0009520 con 0x7fb3b0103120 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b4f9a700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 0x7fb3b0199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.185+0000 7fb3b5f9c700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 0x7fb3b01a0d50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800ef30 tx=0x7fb3a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.186+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8018070 con 0x7fb3b0103120 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.186+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a800f040 con 0x7fb3b0103120 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.186+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b01a1870 con 0x7fb3b0103120 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.186+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800ccf0 con 0x7fb3b0103120 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.186+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b01a1d30 con 0x7fb3b0103120 2024-01-11T17:58:03.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.187+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b00683f0 con 0x7fb3b0103120 2024-01-11T17:58:03.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.189+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fb3a8003a00 con 0x7fb3b0103120 2024-01-11T17:58:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.191+0000 7fb3a67fc700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb39c0a1ee0 0x7fb39c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.191+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb3a80cd860 con 0x7fb3b0103120 2024-01-11T17:58:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.191+0000 7fb3b579b700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb39c0a1ee0 0x7fb39c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.192+0000 7fb3b579b700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb39c0a1ee0 0x7fb39c0a43a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01025e0 tx=0x7fb3ac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.195+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fb3a8096600 con 0x7fb3b0103120 2024-01-11T17:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.405+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3b0074f60 con 0x7fb39c0a1ee0 2024-01-11T17:58:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.418+0000 7fb3a67fc700 1 -- 172.21.15.7:0/2489517703 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427403 (secure 0 0 0) 0x7fb3b0074f60 con 0x7fb39c0a1ee0 2024-01-11T17:58:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb39c0a1ee0 msgr2=0x7fb39c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb39c0a1ee0 0x7fb39c0a43a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01025e0 tx=0x7fb3ac009310 comp rx=0 tx=0).stop 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 msgr2=0x7fb3b01a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 0x7fb3b01a0d50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800ef30 tx=0x7fb3a800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 shutdown_connections 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb39c0a1ee0 0x7fb39c0a43a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b0103120 0x7fb3b01a0d50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b0101590 0x7fb3b01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 --2- 172.21.15.7:0/2489517703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3b0102780 0x7fb3b0199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 >> 172.21.15.7:0/2489517703 conn(0x7fb3b00fcb00 msgr2=0x7fb3b0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 shutdown_connections 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.424+0000 7fb3b79ff700 1 -- 172.21.15.7:0/2489517703 wait complete. 2024-01-11T17:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:58:03.445 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-01-11T17:58:02.596278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664862,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":679,"num_read_kb":466,"num_write":351,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385892,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555932,"statfs":{"total":1151940427776,"available":1151545274368,"internally_reserved":0,"allocated":12062720,"data_stored":7870112,"data_compressed":396960,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6516,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003397"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639344+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:57:54.639344+0000","last_peered":"2024-01-11T17:57:54.639344+0000","last_clean":"2024-01-11T17:57:54.639344+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:57:54.639344+0000","last_undegraded":"2024-01-11T17:57:54.639344+0000","last_fullsized":"2024-01-11T17:57:54.639344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631254+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:57:54.631254+0000","last_peered":"2024-01-11T17:57:54.631254+0000","last_clean":"2024-01-11T17:57:54.631254+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:57:54.631254+0000","last_undegraded":"2024-01-11T17:57:54.631254+0000","last_fullsized":"2024-01-11T17:57:54.631254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.640000+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:57:54.640000+0000","last_peered":"2024-01-11T17:57:54.640000+0000","last_clean":"2024-01-11T17:57:54.640000+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:57:54.640000+0000","last_undegraded":"2024-01-11T17:57:54.640000+0000","last_fullsized":"2024-01-11T17:57:54.640000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.625117+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:57:54.625117+0000","last_peered":"2024-01-11T17:57:54.625117+0000","last_clean":"2024-01-11T17:57:54.625117+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:57:54.625117+0000","last_undegraded":"2024-01-11T17:57:54.625117+0000","last_fullsized":"2024-01-11T17:57:54.625117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643729+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:57:54.643729+0000","last_peered":"2024-01-11T17:57:54.643729+0000","last_clean":"2024-01-11T17:57:54.643729+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:57:54.643729+0000","last_undegraded":"2024-01-11T17:57:54.643729+0000","last_fullsized":"2024-01-11T17:57:54.643729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637431+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:57:54.637431+0000","last_peered":"2024-01-11T17:57:54.637431+0000","last_clean":"2024-01-11T17:57:54.637431+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:57:54.637431+0000","last_undegraded":"2024-01-11T17:57:54.637431+0000","last_fullsized":"2024-01-11T17:57:54.637431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631584+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:57:54.631584+0000","last_peered":"2024-01-11T17:57:54.631584+0000","last_clean":"2024-01-11T17:57:54.631584+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:57:54.631584+0000","last_undegraded":"2024-01-11T17:57:54.631584+0000","last_fullsized":"2024-01-11T17:57:54.631584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.642078+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:57:54.642078+0000","last_peered":"2024-01-11T17:57:54.642078+0000","last_clean":"2024-01-11T17:57:54.642078+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:57:54.642078+0000","last_undegraded":"2024-01-11T17:57:54.642078+0000","last_fullsized":"2024-01-11T17:57:54.642078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636927+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:57:54.636927+0000","last_peered":"2024-01-11T17:57:54.636927+0000","last_clean":"2024-01-11T17:57:54.636927+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:57:54.636927+0000","last_undegraded":"2024-01-11T17:57:54.636927+0000","last_fullsized":"2024-01-11T17:57:54.636927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631470+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:57:54.631470+0000","last_peered":"2024-01-11T17:57:54.631470+0000","last_clean":"2024-01-11T17:57:54.631470+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:57:54.631470+0000","last_undegraded":"2024-01-11T17:57:54.631470+0000","last_fullsized":"2024-01-11T17:57:54.631470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627375+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:57:54.627375+0000","last_peered":"2024-01-11T17:57:54.627375+0000","last_clean":"2024-01-11T17:57:54.627375+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:57:54.627375+0000","last_undegraded":"2024-01-11T17:57:54.627375+0000","last_fullsized":"2024-01-11T17:57:54.627375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623494+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:57:54.623494+0000","last_peered":"2024-01-11T17:57:54.623494+0000","last_clean":"2024-01-11T17:57:54.623494+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:57:54.623494+0000","last_undegraded":"2024-01-11T17:57:54.623494+0000","last_fullsized":"2024-01-11T17:57:54.623494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645164+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:57:54.645164+0000","last_peered":"2024-01-11T17:57:54.645164+0000","last_clean":"2024-01-11T17:57:54.645164+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:57:54.645164+0000","last_undegraded":"2024-01-11T17:57:54.645164+0000","last_fullsized":"2024-01-11T17:57:54.645164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638342+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:57:54.638342+0000","last_peered":"2024-01-11T17:57:54.638342+0000","last_clean":"2024-01-11T17:57:54.638342+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:57:54.638342+0000","last_undegraded":"2024-01-11T17:57:54.638342+0000","last_fullsized":"2024-01-11T17:57:54.638342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644214+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:57:54.644214+0000","last_peered":"2024-01-11T17:57:54.644214+0000","last_clean":"2024-01-11T17:57:54.644214+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:57:54.644214+0000","last_undegraded":"2024-01-11T17:57:54.644214+0000","last_fullsized":"2024-01-11T17:57:54.644214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637270+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:57:54.637270+0000","last_peered":"2024-01-11T17:57:54.637270+0000","last_clean":"2024-01-11T17:57:54.637270+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:57:54.637270+0000","last_undegraded":"2024-01-11T17:57:54.637270+0000","last_fullsized":"2024-01-11T17:57:54.637270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639445+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:57:54.639445+0000","last_peered":"2024-01-11T17:57:54.639445+0000","last_clean":"2024-01-11T17:57:54.639445+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:57:54.639445+0000","last_undegraded":"2024-01-11T17:57:54.639445+0000","last_fullsized":"2024-01-11T17:57:54.639445+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641839+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:57:54.641839+0000","last_peered":"2024-01-11T17:57:54.641839+0000","last_clean":"2024-01-11T17:57:54.641839+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:57:54.641839+0000","last_undegraded":"2024-01-11T17:57:54.641839+0000","last_fullsized":"2024-01-11T17:57:54.641839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644992+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:57:54.644992+0000","last_peered":"2024-01-11T17:57:54.644992+0000","last_clean":"2024-01-11T17:57:54.644992+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:57:54.644992+0000","last_undegraded":"2024-01-11T17:57:54.644992+0000","last_fullsized":"2024-01-11T17:57:54.644992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685921+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:57:26.685921+0000","last_peered":"2024-01-11T17:57:26.685921+0000","last_clean":"2024-01-11T17:57:26.685921+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:57:26.685921+0000","last_undegraded":"2024-01-11T17:57:26.685921+0000","last_fullsized":"2024-01-11T17:57:26.685921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.690057+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:57:26.690057+0000","last_peered":"2024-01-11T17:57:26.690057+0000","last_clean":"2024-01-11T17:57:26.690057+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:57:26.690057+0000","last_undegraded":"2024-01-11T17:57:26.690057+0000","last_fullsized":"2024-01-11T17:57:26.690057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638522+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:57:54.638522+0000","last_peered":"2024-01-11T17:57:54.638522+0000","last_clean":"2024-01-11T17:57:54.638522+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:57:54.638522+0000","last_undegraded":"2024-01-11T17:57:54.638522+0000","last_fullsized":"2024-01-11T17:57:54.638522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346095+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:57:26.346095+0000","last_peered":"2024-01-11T17:57:26.346095+0000","last_clean":"2024-01-11T17:57:26.346095+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:57:26.346095+0000","last_undegraded":"2024-01-11T17:57:26.346095+0000","last_fullsized":"2024-01-11T17:57:26.346095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.412849+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:57:26.412849+0000","last_peered":"2024-01-11T17:57:26.412849+0000","last_clean":"2024-01-11T17:57:26.412849+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:57:26.412849+0000","last_undegraded":"2024-01-11T17:57:26.412849+0000","last_fullsized":"2024-01-11T17:57:26.412849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.640275+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:57:54.640275+0000","last_peered":"2024-01-11T17:57:54.640275+0000","last_clean":"2024-01-11T17:57:54.640275+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:57:54.640275+0000","last_undegraded":"2024-01-11T17:57:54.640275+0000","last_fullsized":"2024-01-11T17:57:54.640275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346280+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:57:26.346280+0000","last_peered":"2024-01-11T17:57:26.346280+0000","last_clean":"2024-01-11T17:57:26.346280+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:57:26.346280+0000","last_undegraded":"2024-01-11T17:57:26.346280+0000","last_fullsized":"2024-01-11T17:57:26.346280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.349086+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:57:26.349086+0000","last_peered":"2024-01-11T17:57:26.349086+0000","last_clean":"2024-01-11T17:57:26.349086+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:57:26.349086+0000","last_undegraded":"2024-01-11T17:57:26.349086+0000","last_fullsized":"2024-01-11T17:57:26.349086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645057+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:57:54.645057+0000","last_peered":"2024-01-11T17:57:54.645057+0000","last_clean":"2024-01-11T17:57:54.645057+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:57:54.645057+0000","last_undegraded":"2024-01-11T17:57:54.645057+0000","last_fullsized":"2024-01-11T17:57:54.645057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346157+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:57:26.346157+0000","last_peered":"2024-01-11T17:57:26.346157+0000","last_clean":"2024-01-11T17:57:26.346157+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:57:26.346157+0000","last_undegraded":"2024-01-11T17:57:26.346157+0000","last_fullsized":"2024-01-11T17:57:26.346157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689779+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:57:26.689779+0000","last_peered":"2024-01-11T17:57:26.689779+0000","last_clean":"2024-01-11T17:57:26.689779+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:57:26.689779+0000","last_undegraded":"2024-01-11T17:57:26.689779+0000","last_fullsized":"2024-01-11T17:57:26.689779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.632027+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:57:54.632027+0000","last_peered":"2024-01-11T17:57:54.632027+0000","last_clean":"2024-01-11T17:57:54.632027+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:57:54.632027+0000","last_undegraded":"2024-01-11T17:57:54.632027+0000","last_fullsized":"2024-01-11T17:57:54.632027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686233+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:57:26.686233+0000","last_peered":"2024-01-11T17:57:26.686233+0000","last_clean":"2024-01-11T17:57:26.686233+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:57:26.686233+0000","last_undegraded":"2024-01-11T17:57:26.686233+0000","last_fullsized":"2024-01-11T17:57:26.686233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:58.463801+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:57:58.463801+0000","last_peered":"2024-01-11T17:57:58.463801+0000","last_clean":"2024-01-11T17:57:58.463801+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:57:58.463801+0000","last_undegraded":"2024-01-11T17:57:58.463801+0000","last_fullsized":"2024-01-11T17:57:58.463801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1459,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.620867+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:57:54.620867+0000","last_peered":"2024-01-11T17:57:54.620867+0000","last_clean":"2024-01-11T17:57:54.620867+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:57:54.620867+0000","last_undegraded":"2024-01-11T17:57:54.620867+0000","last_fullsized":"2024-01-11T17:57:54.620867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686126+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:57:26.686126+0000","last_peered":"2024-01-11T17:57:26.686126+0000","last_clean":"2024-01-11T17:57:26.686126+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:57:26.686126+0000","last_undegraded":"2024-01-11T17:57:26.686126+0000","last_fullsized":"2024-01-11T17:57:26.686126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406841+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:57:26.406841+0000","last_peered":"2024-01-11T17:57:26.406841+0000","last_clean":"2024-01-11T17:57:26.406841+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:57:26.406841+0000","last_undegraded":"2024-01-11T17:57:26.406841+0000","last_fullsized":"2024-01-11T17:57:26.406841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646041+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:57:54.646041+0000","last_peered":"2024-01-11T17:57:54.646041+0000","last_clean":"2024-01-11T17:57:54.646041+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:57:54.646041+0000","last_undegraded":"2024-01-11T17:57:54.646041+0000","last_fullsized":"2024-01-11T17:57:54.646041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.690008+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:57:26.690008+0000","last_peered":"2024-01-11T17:57:26.690008+0000","last_clean":"2024-01-11T17:57:26.690008+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:57:26.690008+0000","last_undegraded":"2024-01-11T17:57:26.690008+0000","last_fullsized":"2024-01-11T17:57:26.690008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406421+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:57:26.406421+0000","last_peered":"2024-01-11T17:57:26.406421+0000","last_clean":"2024-01-11T17:57:26.406421+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:57:26.406421+0000","last_undegraded":"2024-01-11T17:57:26.406421+0000","last_fullsized":"2024-01-11T17:57:26.406421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646879+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:57:54.646879+0000","last_peered":"2024-01-11T17:57:54.646879+0000","last_clean":"2024-01-11T17:57:54.646879+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:57:54.646879+0000","last_undegraded":"2024-01-11T17:57:54.646879+0000","last_fullsized":"2024-01-11T17:57:54.646879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.345977+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:57:26.345977+0000","last_peered":"2024-01-11T17:57:26.345977+0000","last_clean":"2024-01-11T17:57:26.345977+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:57:26.345977+0000","last_undegraded":"2024-01-11T17:57:26.345977+0000","last_fullsized":"2024-01-11T17:57:26.345977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409651+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:57:26.409651+0000","last_peered":"2024-01-11T17:57:26.409651+0000","last_clean":"2024-01-11T17:57:26.409651+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:57:26.409651+0000","last_undegraded":"2024-01-11T17:57:26.409651+0000","last_fullsized":"2024-01-11T17:57:26.409651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638052+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:57:54.638052+0000","last_peered":"2024-01-11T17:57:54.638052+0000","last_clean":"2024-01-11T17:57:54.638052+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:57:54.638052+0000","last_undegraded":"2024-01-11T17:57:54.638052+0000","last_fullsized":"2024-01-11T17:57:54.638052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:28.039741+0000","last_change":"2024-01-11T17:57:28.039741+0000","last_active":"2024-01-11T17:57:28.039741+0000","last_peered":"2024-01-11T17:57:28.039741+0000","last_clean":"2024-01-11T17:57:28.039741+0000","last_became_active":"2024-01-11T17:57:28.039342+0000","last_became_peered":"2024-01-11T17:57:28.039342+0000","last_unstale":"2024-01-11T17:57:28.039741+0000","last_undegraded":"2024-01-11T17:57:28.039741+0000","last_fullsized":"2024-01-11T17:57:28.039741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:17.687299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,10],"acting":[3,7,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:58.463276+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:57:58.463276+0000","last_peered":"2024-01-11T17:57:58.463276+0000","last_clean":"2024-01-11T17:57:58.463276+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:57:58.463276+0000","last_undegraded":"2024-01-11T17:57:58.463276+0000","last_fullsized":"2024-01-11T17:57:58.463276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639525+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:57:54.639525+0000","last_peered":"2024-01-11T17:57:54.639525+0000","last_clean":"2024-01-11T17:57:54.639525+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:57:54.639525+0000","last_undegraded":"2024-01-11T17:57:54.639525+0000","last_fullsized":"2024-01-11T17:57:54.639525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689484+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:57:26.689484+0000","last_peered":"2024-01-11T17:57:26.689484+0000","last_clean":"2024-01-11T17:57:26.689484+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:57:26.689484+0000","last_undegraded":"2024-01-11T17:57:26.689484+0000","last_fullsized":"2024-01-11T17:57:26.689484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415699+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:57:26.415699+0000","last_peered":"2024-01-11T17:57:26.415699+0000","last_clean":"2024-01-11T17:57:26.415699+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:57:26.415699+0000","last_undegraded":"2024-01-11T17:57:26.415699+0000","last_fullsized":"2024-01-11T17:57:26.415699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636540+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:57:54.636540+0000","last_peered":"2024-01-11T17:57:54.636540+0000","last_clean":"2024-01-11T17:57:54.636540+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:57:54.636540+0000","last_undegraded":"2024-01-11T17:57:54.636540+0000","last_fullsized":"2024-01-11T17:57:54.636540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346392+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:57:26.346392+0000","last_peered":"2024-01-11T17:57:26.346392+0000","last_clean":"2024-01-11T17:57:26.346392+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:57:26.346392+0000","last_undegraded":"2024-01-11T17:57:26.346392+0000","last_fullsized":"2024-01-11T17:57:26.346392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410334+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:57:26.410334+0000","last_peered":"2024-01-11T17:57:26.410334+0000","last_clean":"2024-01-11T17:57:26.410334+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:57:26.410334+0000","last_undegraded":"2024-01-11T17:57:26.410334+0000","last_fullsized":"2024-01-11T17:57:26.410334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639586+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:57:54.639586+0000","last_peered":"2024-01-11T17:57:54.639586+0000","last_clean":"2024-01-11T17:57:54.639586+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:57:54.639586+0000","last_undegraded":"2024-01-11T17:57:54.639586+0000","last_fullsized":"2024-01-11T17:57:54.639586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689581+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:57:26.689581+0000","last_peered":"2024-01-11T17:57:26.689581+0000","last_clean":"2024-01-11T17:57:26.689581+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:57:26.689581+0000","last_undegraded":"2024-01-11T17:57:26.689581+0000","last_fullsized":"2024-01-11T17:57:26.689581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'37","reported_seq":242,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.464956+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:57:59.464956+0000","last_peered":"2024-01-11T17:57:59.464956+0000","last_clean":"2024-01-11T17:57:59.464956+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:57:59.464956+0000","last_undegraded":"2024-01-11T17:57:59.464956+0000","last_fullsized":"2024-01-11T17:57:59.464956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69282,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":184,"num_read_kb":161,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623678+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:57:54.623678+0000","last_peered":"2024-01-11T17:57:54.623678+0000","last_clean":"2024-01-11T17:57:54.623678+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:57:54.623678+0000","last_undegraded":"2024-01-11T17:57:54.623678+0000","last_fullsized":"2024-01-11T17:57:54.623678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410861+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:57:26.410861+0000","last_peered":"2024-01-11T17:57:26.410861+0000","last_clean":"2024-01-11T17:57:26.410861+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:57:26.410861+0000","last_undegraded":"2024-01-11T17:57:26.410861+0000","last_fullsized":"2024-01-11T17:57:26.410861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685932+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:57:26.685932+0000","last_peered":"2024-01-11T17:57:26.685932+0000","last_clean":"2024-01-11T17:57:26.685932+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:57:26.685932+0000","last_undegraded":"2024-01-11T17:57:26.685932+0000","last_fullsized":"2024-01-11T17:57:26.685932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644250+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:57:54.644250+0000","last_peered":"2024-01-11T17:57:54.644250+0000","last_clean":"2024-01-11T17:57:54.644250+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:57:54.644250+0000","last_undegraded":"2024-01-11T17:57:54.644250+0000","last_fullsized":"2024-01-11T17:57:54.644250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689807+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:57:26.689807+0000","last_peered":"2024-01-11T17:57:26.689807+0000","last_clean":"2024-01-11T17:57:26.689807+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:57:26.689807+0000","last_undegraded":"2024-01-11T17:57:26.689807+0000","last_fullsized":"2024-01-11T17:57:26.689807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410045+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:57:26.410045+0000","last_peered":"2024-01-11T17:57:26.410045+0000","last_clean":"2024-01-11T17:57:26.410045+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:57:26.410045+0000","last_undegraded":"2024-01-11T17:57:26.410045+0000","last_fullsized":"2024-01-11T17:57:26.410045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641093+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:57:54.641093+0000","last_peered":"2024-01-11T17:57:54.641093+0000","last_clean":"2024-01-11T17:57:54.641093+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:57:54.641093+0000","last_undegraded":"2024-01-11T17:57:54.641093+0000","last_fullsized":"2024-01-11T17:57:54.641093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348785+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:57:26.348785+0000","last_peered":"2024-01-11T17:57:26.348785+0000","last_clean":"2024-01-11T17:57:26.348785+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:57:26.348785+0000","last_undegraded":"2024-01-11T17:57:26.348785+0000","last_fullsized":"2024-01-11T17:57:26.348785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.465117+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:57:59.465117+0000","last_peered":"2024-01-11T17:57:59.465117+0000","last_clean":"2024-01-11T17:57:59.465117+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:57:59.465117+0000","last_undegraded":"2024-01-11T17:57:59.465117+0000","last_fullsized":"2024-01-11T17:57:59.465117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636390+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:57:54.636390+0000","last_peered":"2024-01-11T17:57:54.636390+0000","last_clean":"2024-01-11T17:57:54.636390+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:57:54.636390+0000","last_undegraded":"2024-01-11T17:57:54.636390+0000","last_fullsized":"2024-01-11T17:57:54.636390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406884+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:57:26.406884+0000","last_peered":"2024-01-11T17:57:26.406884+0000","last_clean":"2024-01-11T17:57:26.406884+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:57:26.406884+0000","last_undegraded":"2024-01-11T17:57:26.406884+0000","last_fullsized":"2024-01-11T17:57:26.406884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348419+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:57:26.348419+0000","last_peered":"2024-01-11T17:57:26.348419+0000","last_clean":"2024-01-11T17:57:26.348419+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:57:26.348419+0000","last_undegraded":"2024-01-11T17:57:26.348419+0000","last_fullsized":"2024-01-11T17:57:26.348419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.629482+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:57:54.629482+0000","last_peered":"2024-01-11T17:57:54.629482+0000","last_clean":"2024-01-11T17:57:54.629482+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:57:54.629482+0000","last_undegraded":"2024-01-11T17:57:54.629482+0000","last_fullsized":"2024-01-11T17:57:54.629482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409801+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:57:26.409801+0000","last_peered":"2024-01-11T17:57:26.409801+0000","last_clean":"2024-01-11T17:57:26.409801+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:57:26.409801+0000","last_undegraded":"2024-01-11T17:57:26.409801+0000","last_fullsized":"2024-01-11T17:57:26.409801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348962+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:57:26.348962+0000","last_peered":"2024-01-11T17:57:26.348962+0000","last_clean":"2024-01-11T17:57:26.348962+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:57:26.348962+0000","last_undegraded":"2024-01-11T17:57:26.348962+0000","last_fullsized":"2024-01-11T17:57:26.348962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630954+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:57:54.630954+0000","last_peered":"2024-01-11T17:57:54.630954+0000","last_clean":"2024-01-11T17:57:54.630954+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:57:54.630954+0000","last_undegraded":"2024-01-11T17:57:54.630954+0000","last_fullsized":"2024-01-11T17:57:54.630954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348611+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:57:26.348611+0000","last_peered":"2024-01-11T17:57:26.348611+0000","last_clean":"2024-01-11T17:57:26.348611+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:57:26.348611+0000","last_undegraded":"2024-01-11T17:57:26.348611+0000","last_fullsized":"2024-01-11T17:57:26.348611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689942+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:57:26.689942+0000","last_peered":"2024-01-11T17:57:26.689942+0000","last_clean":"2024-01-11T17:57:26.689942+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:57:26.689942+0000","last_undegraded":"2024-01-11T17:57:26.689942+0000","last_fullsized":"2024-01-11T17:57:26.689942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643064+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:57:54.643064+0000","last_peered":"2024-01-11T17:57:54.643064+0000","last_clean":"2024-01-11T17:57:54.643064+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:57:54.643064+0000","last_undegraded":"2024-01-11T17:57:54.643064+0000","last_fullsized":"2024-01-11T17:57:54.643064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406557+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:57:26.406557+0000","last_peered":"2024-01-11T17:57:26.406557+0000","last_clean":"2024-01-11T17:57:26.406557+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:57:26.406557+0000","last_undegraded":"2024-01-11T17:57:26.406557+0000","last_fullsized":"2024-01-11T17:57:26.406557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415027+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:57:26.415027+0000","last_peered":"2024-01-11T17:57:26.415027+0000","last_clean":"2024-01-11T17:57:26.415027+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:57:26.415027+0000","last_undegraded":"2024-01-11T17:57:26.415027+0000","last_fullsized":"2024-01-11T17:57:26.415027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.632525+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:57:54.632525+0000","last_peered":"2024-01-11T17:57:54.632525+0000","last_clean":"2024-01-11T17:57:54.632525+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:57:54.632525+0000","last_undegraded":"2024-01-11T17:57:54.632525+0000","last_fullsized":"2024-01-11T17:57:54.632525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406685+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:57:26.406685+0000","last_peered":"2024-01-11T17:57:26.406685+0000","last_clean":"2024-01-11T17:57:26.406685+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:57:26.406685+0000","last_undegraded":"2024-01-11T17:57:26.406685+0000","last_fullsized":"2024-01-11T17:57:26.406685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631689+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:57:54.631689+0000","last_peered":"2024-01-11T17:57:54.631689+0000","last_clean":"2024-01-11T17:57:54.631689+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:57:54.631689+0000","last_undegraded":"2024-01-11T17:57:54.631689+0000","last_fullsized":"2024-01-11T17:57:54.631689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.464545+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:57:59.464545+0000","last_peered":"2024-01-11T17:57:59.464545+0000","last_clean":"2024-01-11T17:57:59.464545+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:57:59.464545+0000","last_undegraded":"2024-01-11T17:57:59.464545+0000","last_fullsized":"2024-01-11T17:57:59.464545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415989+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:57:26.415989+0000","last_peered":"2024-01-11T17:57:26.415989+0000","last_clean":"2024-01-11T17:57:26.415989+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:57:26.415989+0000","last_undegraded":"2024-01-11T17:57:26.415989+0000","last_fullsized":"2024-01-11T17:57:26.415989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.647162+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:57:54.647162+0000","last_peered":"2024-01-11T17:57:54.647162+0000","last_clean":"2024-01-11T17:57:54.647162+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:57:54.647162+0000","last_undegraded":"2024-01-11T17:57:54.647162+0000","last_fullsized":"2024-01-11T17:57:54.647162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346338+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:57:26.346338+0000","last_peered":"2024-01-11T17:57:26.346338+0000","last_clean":"2024-01-11T17:57:26.346338+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:57:26.346338+0000","last_undegraded":"2024-01-11T17:57:26.346338+0000","last_fullsized":"2024-01-11T17:57:26.346338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414453+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:57:26.414453+0000","last_peered":"2024-01-11T17:57:26.414453+0000","last_clean":"2024-01-11T17:57:26.414453+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:57:26.414453+0000","last_undegraded":"2024-01-11T17:57:26.414453+0000","last_fullsized":"2024-01-11T17:57:26.414453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410347+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:57:26.410347+0000","last_peered":"2024-01-11T17:57:26.410347+0000","last_clean":"2024-01-11T17:57:26.410347+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:57:26.410347+0000","last_undegraded":"2024-01-11T17:57:26.410347+0000","last_fullsized":"2024-01-11T17:57:26.410347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.634064+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:57:54.634064+0000","last_peered":"2024-01-11T17:57:54.634064+0000","last_clean":"2024-01-11T17:57:54.634064+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:57:54.634064+0000","last_undegraded":"2024-01-11T17:57:54.634064+0000","last_fullsized":"2024-01-11T17:57:54.634064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414488+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:57:26.414488+0000","last_peered":"2024-01-11T17:57:26.414488+0000","last_clean":"2024-01-11T17:57:26.414488+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:57:26.414488+0000","last_undegraded":"2024-01-11T17:57:26.414488+0000","last_fullsized":"2024-01-11T17:57:26.414488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689755+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:57:26.689755+0000","last_peered":"2024-01-11T17:57:26.689755+0000","last_clean":"2024-01-11T17:57:26.689755+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:57:26.689755+0000","last_undegraded":"2024-01-11T17:57:26.689755+0000","last_fullsized":"2024-01-11T17:57:26.689755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348947+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:57:26.348947+0000","last_peered":"2024-01-11T17:57:26.348947+0000","last_clean":"2024-01-11T17:57:26.348947+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:57:26.348947+0000","last_undegraded":"2024-01-11T17:57:26.348947+0000","last_fullsized":"2024-01-11T17:57:26.348947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633371+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:57:54.633371+0000","last_peered":"2024-01-11T17:57:54.633371+0000","last_clean":"2024-01-11T17:57:54.633371+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:57:54.633371+0000","last_undegraded":"2024-01-11T17:57:54.633371+0000","last_fullsized":"2024-01-11T17:57:54.633371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.411976+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:57:26.411976+0000","last_peered":"2024-01-11T17:57:26.411976+0000","last_clean":"2024-01-11T17:57:26.411976+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:57:26.411976+0000","last_undegraded":"2024-01-11T17:57:26.411976+0000","last_fullsized":"2024-01-11T17:57:26.411976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:27.351805+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:57:27.351805+0000","last_peered":"2024-01-11T17:57:27.351805+0000","last_clean":"2024-01-11T17:57:27.351805+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:57:27.351805+0000","last_undegraded":"2024-01-11T17:57:27.351805+0000","last_fullsized":"2024-01-11T17:57:27.351805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638090+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:57:54.638090+0000","last_peered":"2024-01-11T17:57:54.638090+0000","last_clean":"2024-01-11T17:57:54.638090+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:57:54.638090+0000","last_undegraded":"2024-01-11T17:57:54.638090+0000","last_fullsized":"2024-01-11T17:57:54.638090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685669+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:57:26.685669+0000","last_peered":"2024-01-11T17:57:26.685669+0000","last_clean":"2024-01-11T17:57:26.685669+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:57:26.685669+0000","last_undegraded":"2024-01-11T17:57:26.685669+0000","last_fullsized":"2024-01-11T17:57:26.685669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.347046+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:57:26.347046+0000","last_peered":"2024-01-11T17:57:26.347046+0000","last_clean":"2024-01-11T17:57:26.347046+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:57:26.347046+0000","last_undegraded":"2024-01-11T17:57:26.347046+0000","last_fullsized":"2024-01-11T17:57:26.347046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.642913+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:57:54.642913+0000","last_peered":"2024-01-11T17:57:54.642913+0000","last_clean":"2024-01-11T17:57:54.642913+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:57:54.642913+0000","last_undegraded":"2024-01-11T17:57:54.642913+0000","last_fullsized":"2024-01-11T17:57:54.642913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346710+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:57:26.346710+0000","last_peered":"2024-01-11T17:57:26.346710+0000","last_clean":"2024-01-11T17:57:26.346710+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:57:26.346710+0000","last_undegraded":"2024-01-11T17:57:26.346710+0000","last_fullsized":"2024-01-11T17:57:26.346710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415555+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:57:26.415555+0000","last_peered":"2024-01-11T17:57:26.415555+0000","last_clean":"2024-01-11T17:57:26.415555+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:57:26.415555+0000","last_undegraded":"2024-01-11T17:57:26.415555+0000","last_fullsized":"2024-01-11T17:57:26.415555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.598022+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:57:54.598022+0000","last_peered":"2024-01-11T17:57:54.598022+0000","last_clean":"2024-01-11T17:57:54.598022+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:57:54.598022+0000","last_undegraded":"2024-01-11T17:57:54.598022+0000","last_fullsized":"2024-01-11T17:57:54.598022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689961+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:57:26.689961+0000","last_peered":"2024-01-11T17:57:26.689961+0000","last_clean":"2024-01-11T17:57:26.689961+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:57:26.689961+0000","last_undegraded":"2024-01-11T17:57:26.689961+0000","last_fullsized":"2024-01-11T17:57:26.689961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.412881+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:57:26.412881+0000","last_peered":"2024-01-11T17:57:26.412881+0000","last_clean":"2024-01-11T17:57:26.412881+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:57:26.412881+0000","last_undegraded":"2024-01-11T17:57:26.412881+0000","last_fullsized":"2024-01-11T17:57:26.412881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.635273+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:57:54.635273+0000","last_peered":"2024-01-11T17:57:54.635273+0000","last_clean":"2024-01-11T17:57:54.635273+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:57:54.635273+0000","last_undegraded":"2024-01-11T17:57:54.635273+0000","last_fullsized":"2024-01-11T17:57:54.635273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346278+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:57:26.346278+0000","last_peered":"2024-01-11T17:57:26.346278+0000","last_clean":"2024-01-11T17:57:26.346278+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:57:26.346278+0000","last_undegraded":"2024-01-11T17:57:26.346278+0000","last_fullsized":"2024-01-11T17:57:26.346278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689783+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:57:26.689783+0000","last_peered":"2024-01-11T17:57:26.689783+0000","last_clean":"2024-01-11T17:57:26.689783+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:57:26.689783+0000","last_undegraded":"2024-01-11T17:57:26.689783+0000","last_fullsized":"2024-01-11T17:57:26.689783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641886+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:57:54.641886+0000","last_peered":"2024-01-11T17:57:54.641886+0000","last_clean":"2024-01-11T17:57:54.641886+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:57:54.641886+0000","last_undegraded":"2024-01-11T17:57:54.641886+0000","last_fullsized":"2024-01-11T17:57:54.641886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346709+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:57:26.346709+0000","last_peered":"2024-01-11T17:57:26.346709+0000","last_clean":"2024-01-11T17:57:26.346709+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:57:26.346709+0000","last_undegraded":"2024-01-11T17:57:26.346709+0000","last_fullsized":"2024-01-11T17:57:26.346709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348623+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:57:26.348623+0000","last_peered":"2024-01-11T17:57:26.348623+0000","last_clean":"2024-01-11T17:57:26.348623+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:57:26.348623+0000","last_undegraded":"2024-01-11T17:57:26.348623+0000","last_fullsized":"2024-01-11T17:57:26.348623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633587+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:57:54.633587+0000","last_peered":"2024-01-11T17:57:54.633587+0000","last_clean":"2024-01-11T17:57:54.633587+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:57:54.633587+0000","last_undegraded":"2024-01-11T17:57:54.633587+0000","last_fullsized":"2024-01-11T17:57:54.633587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414998+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:57:26.414998+0000","last_peered":"2024-01-11T17:57:26.414998+0000","last_clean":"2024-01-11T17:57:26.414998+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:57:26.414998+0000","last_undegraded":"2024-01-11T17:57:26.414998+0000","last_fullsized":"2024-01-11T17:57:26.414998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686294+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:57:26.686294+0000","last_peered":"2024-01-11T17:57:26.686294+0000","last_clean":"2024-01-11T17:57:26.686294+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:57:26.686294+0000","last_undegraded":"2024-01-11T17:57:26.686294+0000","last_fullsized":"2024-01-11T17:57:26.686294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627483+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:57:54.627483+0000","last_peered":"2024-01-11T17:57:54.627483+0000","last_clean":"2024-01-11T17:57:54.627483+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:57:54.627483+0000","last_undegraded":"2024-01-11T17:57:54.627483+0000","last_fullsized":"2024-01-11T17:57:54.627483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.345942+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:57:26.345942+0000","last_peered":"2024-01-11T17:57:26.345942+0000","last_clean":"2024-01-11T17:57:26.345942+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:57:26.345942+0000","last_undegraded":"2024-01-11T17:57:26.345942+0000","last_fullsized":"2024-01-11T17:57:26.345942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410583+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:57:26.410583+0000","last_peered":"2024-01-11T17:57:26.410583+0000","last_clean":"2024-01-11T17:57:26.410583+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:57:26.410583+0000","last_undegraded":"2024-01-11T17:57:26.410583+0000","last_fullsized":"2024-01-11T17:57:26.410583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641067+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:57:54.641067+0000","last_peered":"2024-01-11T17:57:54.641067+0000","last_clean":"2024-01-11T17:57:54.641067+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:57:54.641067+0000","last_undegraded":"2024-01-11T17:57:54.641067+0000","last_fullsized":"2024-01-11T17:57:54.641067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409577+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:57:26.409577+0000","last_peered":"2024-01-11T17:57:26.409577+0000","last_clean":"2024-01-11T17:57:26.409577+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:57:26.409577+0000","last_undegraded":"2024-01-11T17:57:26.409577+0000","last_fullsized":"2024-01-11T17:57:26.409577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406896+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:57:26.406896+0000","last_peered":"2024-01-11T17:57:26.406896+0000","last_clean":"2024-01-11T17:57:26.406896+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:57:26.406896+0000","last_undegraded":"2024-01-11T17:57:26.406896+0000","last_fullsized":"2024-01-11T17:57:26.406896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630352+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:57:54.630352+0000","last_peered":"2024-01-11T17:57:54.630352+0000","last_clean":"2024-01-11T17:57:54.630352+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:57:54.630352+0000","last_undegraded":"2024-01-11T17:57:54.630352+0000","last_fullsized":"2024-01-11T17:57:54.630352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633236+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:57:54.633236+0000","last_peered":"2024-01-11T17:57:54.633236+0000","last_clean":"2024-01-11T17:57:54.633236+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:57:54.633236+0000","last_undegraded":"2024-01-11T17:57:54.633236+0000","last_fullsized":"2024-01-11T17:57:54.633236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.634289+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:57:54.634289+0000","last_peered":"2024-01-11T17:57:54.634289+0000","last_clean":"2024-01-11T17:57:54.634289+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:57:54.634289+0000","last_undegraded":"2024-01-11T17:57:54.634289+0000","last_fullsized":"2024-01-11T17:57:54.634289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637772+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:57:54.637772+0000","last_peered":"2024-01-11T17:57:54.637772+0000","last_clean":"2024-01-11T17:57:54.637772+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:57:54.637772+0000","last_undegraded":"2024-01-11T17:57:54.637772+0000","last_fullsized":"2024-01-11T17:57:54.637772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645709+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:57:54.645709+0000","last_peered":"2024-01-11T17:57:54.645709+0000","last_clean":"2024-01-11T17:57:54.645709+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:57:54.645709+0000","last_undegraded":"2024-01-11T17:57:54.645709+0000","last_fullsized":"2024-01-11T17:57:54.645709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643205+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:57:54.643205+0000","last_peered":"2024-01-11T17:57:54.643205+0000","last_clean":"2024-01-11T17:57:54.643205+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:57:54.643205+0000","last_undegraded":"2024-01-11T17:57:54.643205+0000","last_fullsized":"2024-01-11T17:57:54.643205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630113+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:57:54.630113+0000","last_peered":"2024-01-11T17:57:54.630113+0000","last_clean":"2024-01-11T17:57:54.630113+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:57:54.630113+0000","last_undegraded":"2024-01-11T17:57:54.630113+0000","last_fullsized":"2024-01-11T17:57:54.630113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638641+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:57:54.638641+0000","last_peered":"2024-01-11T17:57:54.638641+0000","last_clean":"2024-01-11T17:57:54.638641+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:57:54.638641+0000","last_undegraded":"2024-01-11T17:57:54.638641+0000","last_fullsized":"2024-01-11T17:57:54.638641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627381+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:57:54.627381+0000","last_peered":"2024-01-11T17:57:54.627381+0000","last_clean":"2024-01-11T17:57:54.627381+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:57:54.627381+0000","last_undegraded":"2024-01-11T17:57:54.627381+0000","last_fullsized":"2024-01-11T17:57:54.627381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646027+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:57:54.646027+0000","last_peered":"2024-01-11T17:57:54.646027+0000","last_clean":"2024-01-11T17:57:54.646027+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:57:54.646027+0000","last_undegraded":"2024-01-11T17:57:54.646027+0000","last_fullsized":"2024-01-11T17:57:54.646027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639605+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:57:54.639605+0000","last_peered":"2024-01-11T17:57:54.639605+0000","last_clean":"2024-01-11T17:57:54.639605+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:57:54.639605+0000","last_undegraded":"2024-01-11T17:57:54.639605+0000","last_fullsized":"2024-01-11T17:57:54.639605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.647432+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:57:54.647432+0000","last_peered":"2024-01-11T17:57:54.647432+0000","last_clean":"2024-01-11T17:57:54.647432+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:57:54.647432+0000","last_undegraded":"2024-01-11T17:57:54.647432+0000","last_fullsized":"2024-01-11T17:57:54.647432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644914+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:57:54.644914+0000","last_peered":"2024-01-11T17:57:54.644914+0000","last_clean":"2024-01-11T17:57:54.644914+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:57:54.644914+0000","last_undegraded":"2024-01-11T17:57:54.644914+0000","last_fullsized":"2024-01-11T17:57:54.644914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623596+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:57:54.623596+0000","last_peered":"2024-01-11T17:57:54.623596+0000","last_clean":"2024-01-11T17:57:54.623596+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:57:54.623596+0000","last_undegraded":"2024-01-11T17:57:54.623596+0000","last_fullsized":"2024-01-11T17:57:54.623596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74260,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":553,"num_read_kb":357,"num_write":168,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":222780,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":491633,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.497,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.495,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89800000000000002,"5min":1.109,"15min":1.109},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93600000000000005,"5min":1.048,"15min":1.048},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.53400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.046,"5min":1.405,"15min":1.405},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.748,"15min":0.74299999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93600000000000005,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.755},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88,"15min":0.878},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0680000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88,"15min":0.88100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0329999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.02,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.91500000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.872}]},{"osd":9,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1779999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.1120000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.94599999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.71199999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1279999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.95299999999999996,"15min":0.94699999999999995},"min":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.139,"5min":1.355,"15min":1.355},"last":0.93000000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":1148714,"data_compressed":61367,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.623}]},{"osd":3,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.751},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.32300000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.94999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.254,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.097,"15min":1.097},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.117,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69499999999999995}]},{"osd":9,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.099,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0529999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.73499999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.93899999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.04,"5min":1.244,"15min":1.244},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.075,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.97899999999999998}]}]},{"osd":9,"up_from":63,"seq":270582939688,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":495965,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81200000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.73999999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75700000000000001,"15min":0.77100000000000002},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90500000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86199999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.75700000000000001,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79700000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90600000000000003,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96199999999999997,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.198,"15min":1.198},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93899999999999995,"5min":1.224,"15min":1.224},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88900000000000001,"15min":0.93100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.139,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.91900000000000004,"15min":0.96699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.91900000000000004,"15min":0.97299999999999998},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0069999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93300000000000005,"15min":0.96899999999999997},"min":{"1min":0.68500000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.022,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.94799999999999995},"min":{"1min":0.622,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96099999999999997,"5min":1.284,"15min":1.284},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0309999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85499999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":491789,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.38300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.624}]},{"osd":3,"last update":"Thu Jan 11 17:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":4,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":1.097},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.998,"5min":1,"15min":1},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.877},{"interface":"front","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.83399999999999996},"min":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.135,"5min":1.163,"15min":1.163},"last":0.72699999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89600000000000002,"15min":0.89800000000000002},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.095,"5min":1.125,"15min":1.125},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77600000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.88700000000000001,"15min":0.871},"min":{"1min":0.624,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.81299999999999994,"15min":0.79500000000000004},"min":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.85599999999999998}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1088675,"data_compressed":59744,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.625},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.626,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90300000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81499999999999995,"5min":1.137,"15min":1.137},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.56000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.64500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98699999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":1.04,"15min":1.04},"last":0.53700000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81499999999999995,"15min":0.80300000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.018,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81299999999999994,"15min":0.81200000000000006},"min":{"1min":0.68200000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96899999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.95199999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:57:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0600000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80500000000000005,"15min":0.79600000000000004},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.82999999999999996},"min":{"1min":0.79500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.92500000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0129999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":487045,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.149,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":2.149,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.753,"15min":0.71499999999999997},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.085,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66300000000000003,"15min":0.61499999999999999},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.907,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.161,"15min":1.161},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.72,"5min":2.645,"15min":2.645},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.8029999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.627}]},{"osd":4,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1869999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.84699999999999998,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5349999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.75}]},{"osd":5,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.9770000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.77000000000000002,"15min":0.74099999999999999},"min":{"1min":0.57499999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0819999999999999,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.91600000000000004,"15min":0.90100000000000002},"min":{"1min":0.68799999999999994,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.79700000000000004,"15min":0.75600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.91200000000000003,"15min":0.89500000000000002},"min":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4890000000000001,"5min":2.282,"15min":2.282},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.86699999999999999,"15min":0.85099999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.014,"15min":1.028},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.577,"5min":3.044,"15min":3.044},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.91900000000000004},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2080000000000002,"5min":2.415,"15min":2.415},"last":0.96599999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.92700000000000005,"15min":0.89500000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.093,"15min":1.095},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.2509999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.024}]},{"osd":11,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.98999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":556237,"data_compressed":25274,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.436,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91900000000000004,"5min":1.077,"15min":1.077},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.46200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.98899999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59399999999999997,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61399999999999999,"15min":0.55500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.54200000000000004},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.97299999999999998,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88700000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.46300000000000002},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":1.02,"15min":1.02},"last":0.753}]},{"osd":5,"last update":"Thu Jan 11 17:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94399999999999995,"5min":1.012,"15min":1.012},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68500000000000005,"15min":0.622},"min":{"1min":0.54300000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":1.048,"15min":1.048},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0600000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.53600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81599999999999995,"5min":1.048,"15min":1.048},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.64000000000000001},"min":{"1min":0.59699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.048,"15min":1.048},"last":0.875}]},{"osd":8,"last update":"Thu Jan 11 17:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.745,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0089999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86199999999999999,"15min":0.86399999999999999},"min":{"1min":0.58299999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0880000000000001,"5min":1.21,"15min":1.21},"last":0.95599999999999996}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":488433,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.66600000000000004,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.911,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.502,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.61499999999999999},"min":{"1min":0.55800000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.85799999999999998,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69599999999999995,"15min":0.59599999999999997},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.087,"5min":3.508,"15min":3.508},"last":0.438}]},{"osd":3,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72299999999999998,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3359999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.77400000000000002,"15min":0.65200000000000002},"min":{"1min":0.56200000000000006,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.2320000000000002,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.747}]},{"osd":5,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73399999999999999,"15min":0.68700000000000006},"min":{"1min":0.56899999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.024,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.72099999999999997,"15min":0.64600000000000002},"min":{"1min":0.58299999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2609999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.123},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.67300000000000004},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":3.1019999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.68500000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5629999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88,"15min":0.84399999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1879999999999999,"5min":3.198,"15min":3.198},"last":0.77200000000000002}]},{"osd":9,"last update":"Thu Jan 11 17:57:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.95099999999999996},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.423,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97199999999999998,"15min":0.95899999999999996},"min":{"1min":0.72699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.2390000000000001,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.81599999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.75800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0630000000000002,"5min":3.0630000000000002,"15min":3.0630000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.135,"15min":1.145},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":3.4199999999999999,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.0189999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.79000000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5429999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.022,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.339,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":1.056}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":558424,"data_compressed":25274,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.745},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83099999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.96199999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.65900000000000003,"15min":0.56499999999999995},"min":{"1min":0.51800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1220000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0249999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.022,"15min":1.022},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.61099999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.74399999999999999,"15min":0.71199999999999997},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.76100000000000001}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.78000000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98599999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.87}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.129},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.73499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0469999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78700000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":487239,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.40100000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.249},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.46700000000000003,"15min":0.42599999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.623,"15min":0.53700000000000003},"min":{"1min":0.56299999999999994,"5min":0.308,"15min":0.308},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.64000000000000001,"15min":0.55400000000000005},"min":{"1min":0.59099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65500000000000003,"15min":0.60499999999999998},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.628},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.56699999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69399999999999995,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1599999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.72999999999999998,"15min":0.64200000000000002},"min":{"1min":0.64300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3680000000000001,"5min":1.792,"15min":1.792},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.68600000000000005},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1120000000000001,"5min":1.821,"15min":1.821},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.60799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0660000000000001,"5min":1.379,"15min":1.379},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.86199999999999999,"15min":0.84999999999999998},"min":{"1min":0.66300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.93000000000000005},"min":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3069999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1579999999999999,"5min":1.52,"15min":1.52},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.119,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.02,"15min":1.036},"min":{"1min":0.77500000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2330000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.876}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":492226,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.379},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.438,"15min":0.39500000000000002},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.745,"15min":0.745},"last":0.5}]},{"osd":2,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.76900000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94599999999999995,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.877}]},{"osd":3,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":1.034,"15min":1.034},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.004,"15min":1.004},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68899999999999995,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95699999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66300000000000003,"15min":0.60099999999999998},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.129,"15min":1.129},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69499999999999995,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86499999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89000000000000001,"5min":1.163,"15min":1.163},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.745}]},{"osd":8,"last update":"Thu Jan 11 17:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94099999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.61299999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.97099999999999997,"5min":1.238,"15min":1.238},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.878,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.92400000000000004},"min":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.014,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.629,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.95299999999999996,"5min":1.248,"15min":1.248},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.84899999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.92500000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84999999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1083732,"data_compressed":59744,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59099999999999997,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82599999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45600000000000002,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.45900000000000002},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65500000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":1.548,"15min":1.548},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83099999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90400000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.22,"15min":1.22},"last":0.755},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.91300000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95199999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.0309999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":1.097,"15min":1.097},"last":0.96099999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79700000000000004,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0189999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.012}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":69282,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71469,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71391,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:58:03.487 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-01-11T17:58:03.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 -- 172.21.15.7:0/1691998587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c102780 msgr2=0x7faa5c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 --2- 172.21.15.7:0/1691998587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c102780 0x7faa5c102be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faa500099c0 tx=0x7faa50009cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 -- 172.21.15.7:0/1691998587 shutdown_connections 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 --2- 172.21.15.7:0/1691998587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c101590 0x7faa5c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 --2- 172.21.15.7:0/1691998587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c103120 0x7faa5c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 --2- 172.21.15.7:0/1691998587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c102780 0x7faa5c102be0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 -- 172.21.15.7:0/1691998587 >> 172.21.15.7:0/1691998587 conn(0x7faa5c0fcb00 msgr2=0x7faa5c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 -- 172.21.15.7:0/1691998587 shutdown_connections 2024-01-11T17:58:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.692+0000 7faa60bff700 1 -- 172.21.15.7:0/1691998587 wait complete. 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.693+0000 7faa60bff700 1 Processor -- start 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.693+0000 7faa60bff700 1 -- start start 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa60bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c101590 0x7faa5c0753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa60bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 0x7faa5c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa60bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c103120 0x7faa5c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa60bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0746e0 con 0x7faa5c102780 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa60bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c074850 con 0x7faa5c103120 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa60bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c1a1530 con 0x7faa5c101590 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 0x7faa5c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 0x7faa5c073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46764/0 (socket says 172.21.15.7:46764) 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 -- 172.21.15.7:0/2613186944 learned_addr learned my addr 172.21.15.7:0/2613186944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa5ad9d700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c103120 0x7faa5c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa5a59c700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c101590 0x7faa5c0753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 -- 172.21.15.7:0/2613186944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c101590 msgr2=0x7faa5c0753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c101590 0x7faa5c0753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 -- 172.21.15.7:0/2613186944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c103120 msgr2=0x7faa5c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c103120 0x7faa5c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.694+0000 7faa59d9b700 1 -- 172.21.15.7:0/2613186944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500095f0 con 0x7faa5c102780 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.695+0000 7faa59d9b700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 0x7faa5c073a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa50003730 tx=0x7faa5000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.695+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5001d070 con 0x7faa5c102780 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.695+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa5000bda0 con 0x7faa5c102780 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.695+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c1a17b0 con 0x7faa5c102780 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.695+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50004bd0 con 0x7faa5c102780 2024-01-11T17:58:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.695+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c1a1c30 con 0x7faa5c102780 2024-01-11T17:58:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.696+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c108a30 con 0x7faa5c102780 2024-01-11T17:58:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.698+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7faa50004d30 con 0x7faa5c102780 2024-01-11T17:58:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.699+0000 7faa4b7fe700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7faa440a1f60 0x7faa440a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.700+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faa500cea30 con 0x7faa5c102780 2024-01-11T17:58:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.700+0000 7faa5a59c700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7faa440a1f60 0x7faa440a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.700+0000 7faa5a59c700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7faa440a1f60 0x7faa440a4420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa5c1025e0 tx=0x7faa4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:03.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.703+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7faa500977d0 con 0x7faa5c102780 2024-01-11T17:58:03.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:03 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3921588919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:03.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:03 smithi016 ceph-mon[43195]: pgmap v307: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-01-11T17:58:03.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:03 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3921588919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:03.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:03 smithi038 ceph-mon[43582]: pgmap v307: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-01-11T17:58:03.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:03 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3921588919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:03.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:03 smithi007 ceph-mon[33361]: pgmap v307: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-01-11T17:58:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.920+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 --> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa5c02d3c0 con 0x7faa440a1f60 2024-01-11T17:58:03.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.933+0000 7faa4b7fe700 1 -- 172.21.15.7:0/2613186944 <== mgr.14152 v2:172.21.15.7:6800/314922909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427403 (secure 0 0 0) 0x7faa5c02d3c0 con 0x7faa440a1f60 2024-01-11T17:58:03.941 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7faa440a1f60 msgr2=0x7faa440a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7faa440a1f60 0x7faa440a4420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa5c1025e0 tx=0x7faa4c009260 comp rx=0 tx=0).stop 2024-01-11T17:58:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 msgr2=0x7faa5c073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 0x7faa5c073a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa50003730 tx=0x7faa5000fa00 comp rx=0 tx=0).stop 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 shutdown_connections 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7faa440a1f60 0x7faa440a4420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa5c101590 0x7faa5c0753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5c103120 0x7faa5c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 --2- 172.21.15.7:0/2613186944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa5c102780 0x7faa5c073a00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 >> 172.21.15.7:0/2613186944 conn(0x7faa5c0fcb00 msgr2=0x7faa5c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 shutdown_connections 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:03.938+0000 7faa60bff700 1 -- 172.21.15.7:0/2613186944 wait complete. 2024-01-11T17:58:03.946 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T17:58:03.959 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-01-11T17:58:02.596278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664862,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":679,"num_read_kb":466,"num_write":351,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385892,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555932,"statfs":{"total":1151940427776,"available":1151545274368,"internally_reserved":0,"allocated":12062720,"data_stored":7870112,"data_compressed":396960,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6516,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003397"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639344+0000","last_change":"2024-01-11T17:56:08.806553+0000","last_active":"2024-01-11T17:57:54.639344+0000","last_peered":"2024-01-11T17:57:54.639344+0000","last_clean":"2024-01-11T17:57:54.639344+0000","last_became_active":"2024-01-11T17:56:08.806128+0000","last_became_peered":"2024-01-11T17:56:08.806128+0000","last_unstale":"2024-01-11T17:57:54.639344+0000","last_undegraded":"2024-01-11T17:57:54.639344+0000","last_fullsized":"2024-01-11T17:57:54.639344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:42:30.136880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631254+0000","last_change":"2024-01-11T17:56:08.803763+0000","last_active":"2024-01-11T17:57:54.631254+0000","last_peered":"2024-01-11T17:57:54.631254+0000","last_clean":"2024-01-11T17:57:54.631254+0000","last_became_active":"2024-01-11T17:56:08.801834+0000","last_became_peered":"2024-01-11T17:56:08.801834+0000","last_unstale":"2024-01-11T17:57:54.631254+0000","last_undegraded":"2024-01-11T17:57:54.631254+0000","last_fullsized":"2024-01-11T17:57:54.631254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:06:03.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.640000+0000","last_change":"2024-01-11T17:56:08.801575+0000","last_active":"2024-01-11T17:57:54.640000+0000","last_peered":"2024-01-11T17:57:54.640000+0000","last_clean":"2024-01-11T17:57:54.640000+0000","last_became_active":"2024-01-11T17:56:08.801168+0000","last_became_peered":"2024-01-11T17:56:08.801168+0000","last_unstale":"2024-01-11T17:57:54.640000+0000","last_undegraded":"2024-01-11T17:57:54.640000+0000","last_fullsized":"2024-01-11T17:57:54.640000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:36:59.458403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.625117+0000","last_change":"2024-01-11T17:56:08.796009+0000","last_active":"2024-01-11T17:57:54.625117+0000","last_peered":"2024-01-11T17:57:54.625117+0000","last_clean":"2024-01-11T17:57:54.625117+0000","last_became_active":"2024-01-11T17:56:08.795623+0000","last_became_peered":"2024-01-11T17:56:08.795623+0000","last_unstale":"2024-01-11T17:57:54.625117+0000","last_undegraded":"2024-01-11T17:57:54.625117+0000","last_fullsized":"2024-01-11T17:57:54.625117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:04:37.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643729+0000","last_change":"2024-01-11T17:56:26.355950+0000","last_active":"2024-01-11T17:57:54.643729+0000","last_peered":"2024-01-11T17:57:54.643729+0000","last_clean":"2024-01-11T17:57:54.643729+0000","last_became_active":"2024-01-11T17:56:26.355502+0000","last_became_peered":"2024-01-11T17:56:26.355502+0000","last_unstale":"2024-01-11T17:57:54.643729+0000","last_undegraded":"2024-01-11T17:57:54.643729+0000","last_fullsized":"2024-01-11T17:57:54.643729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:15:34.505693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637431+0000","last_change":"2024-01-11T17:56:26.355667+0000","last_active":"2024-01-11T17:57:54.637431+0000","last_peered":"2024-01-11T17:57:54.637431+0000","last_clean":"2024-01-11T17:57:54.637431+0000","last_became_active":"2024-01-11T17:56:26.355339+0000","last_became_peered":"2024-01-11T17:56:26.355339+0000","last_unstale":"2024-01-11T17:57:54.637431+0000","last_undegraded":"2024-01-11T17:57:54.637431+0000","last_fullsized":"2024-01-11T17:57:54.637431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:43:44.477015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631584+0000","last_change":"2024-01-11T17:56:08.801731+0000","last_active":"2024-01-11T17:57:54.631584+0000","last_peered":"2024-01-11T17:57:54.631584+0000","last_clean":"2024-01-11T17:57:54.631584+0000","last_became_active":"2024-01-11T17:56:08.801377+0000","last_became_peered":"2024-01-11T17:56:08.801377+0000","last_unstale":"2024-01-11T17:57:54.631584+0000","last_undegraded":"2024-01-11T17:57:54.631584+0000","last_fullsized":"2024-01-11T17:57:54.631584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:11:07.217774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.642078+0000","last_change":"2024-01-11T17:56:08.811543+0000","last_active":"2024-01-11T17:57:54.642078+0000","last_peered":"2024-01-11T17:57:54.642078+0000","last_clean":"2024-01-11T17:57:54.642078+0000","last_became_active":"2024-01-11T17:56:08.811236+0000","last_became_peered":"2024-01-11T17:56:08.811236+0000","last_unstale":"2024-01-11T17:57:54.642078+0000","last_undegraded":"2024-01-11T17:57:54.642078+0000","last_fullsized":"2024-01-11T17:57:54.642078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:02:05.157311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636927+0000","last_change":"2024-01-11T17:56:08.796429+0000","last_active":"2024-01-11T17:57:54.636927+0000","last_peered":"2024-01-11T17:57:54.636927+0000","last_clean":"2024-01-11T17:57:54.636927+0000","last_became_active":"2024-01-11T17:56:08.795967+0000","last_became_peered":"2024-01-11T17:56:08.795967+0000","last_unstale":"2024-01-11T17:57:54.636927+0000","last_undegraded":"2024-01-11T17:57:54.636927+0000","last_fullsized":"2024-01-11T17:57:54.636927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:28:18.337373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631470+0000","last_change":"2024-01-11T17:56:08.802136+0000","last_active":"2024-01-11T17:57:54.631470+0000","last_peered":"2024-01-11T17:57:54.631470+0000","last_clean":"2024-01-11T17:57:54.631470+0000","last_became_active":"2024-01-11T17:56:08.801699+0000","last_became_peered":"2024-01-11T17:56:08.801699+0000","last_unstale":"2024-01-11T17:57:54.631470+0000","last_undegraded":"2024-01-11T17:57:54.631470+0000","last_fullsized":"2024-01-11T17:57:54.631470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:35:33.204263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627375+0000","last_change":"2024-01-11T17:56:08.799203+0000","last_active":"2024-01-11T17:57:54.627375+0000","last_peered":"2024-01-11T17:57:54.627375+0000","last_clean":"2024-01-11T17:57:54.627375+0000","last_became_active":"2024-01-11T17:56:08.798862+0000","last_became_peered":"2024-01-11T17:56:08.798862+0000","last_unstale":"2024-01-11T17:57:54.627375+0000","last_undegraded":"2024-01-11T17:57:54.627375+0000","last_fullsized":"2024-01-11T17:57:54.627375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:12.297095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623494+0000","last_change":"2024-01-11T17:56:08.798511+0000","last_active":"2024-01-11T17:57:54.623494+0000","last_peered":"2024-01-11T17:57:54.623494+0000","last_clean":"2024-01-11T17:57:54.623494+0000","last_became_active":"2024-01-11T17:56:08.798152+0000","last_became_peered":"2024-01-11T17:56:08.798152+0000","last_unstale":"2024-01-11T17:57:54.623494+0000","last_undegraded":"2024-01-11T17:57:54.623494+0000","last_fullsized":"2024-01-11T17:57:54.623494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:41:53.741260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645164+0000","last_change":"2024-01-11T17:56:08.797971+0000","last_active":"2024-01-11T17:57:54.645164+0000","last_peered":"2024-01-11T17:57:54.645164+0000","last_clean":"2024-01-11T17:57:54.645164+0000","last_became_active":"2024-01-11T17:56:08.797600+0000","last_became_peered":"2024-01-11T17:56:08.797600+0000","last_unstale":"2024-01-11T17:57:54.645164+0000","last_undegraded":"2024-01-11T17:57:54.645164+0000","last_fullsized":"2024-01-11T17:57:54.645164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:49:46.289148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638342+0000","last_change":"2024-01-11T17:56:08.802250+0000","last_active":"2024-01-11T17:57:54.638342+0000","last_peered":"2024-01-11T17:57:54.638342+0000","last_clean":"2024-01-11T17:57:54.638342+0000","last_became_active":"2024-01-11T17:56:08.798804+0000","last_became_peered":"2024-01-11T17:56:08.798804+0000","last_unstale":"2024-01-11T17:57:54.638342+0000","last_undegraded":"2024-01-11T17:57:54.638342+0000","last_fullsized":"2024-01-11T17:57:54.638342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:41:34.446974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644214+0000","last_change":"2024-01-11T17:56:26.355728+0000","last_active":"2024-01-11T17:57:54.644214+0000","last_peered":"2024-01-11T17:57:54.644214+0000","last_clean":"2024-01-11T17:57:54.644214+0000","last_became_active":"2024-01-11T17:56:26.354730+0000","last_became_peered":"2024-01-11T17:56:26.354730+0000","last_unstale":"2024-01-11T17:57:54.644214+0000","last_undegraded":"2024-01-11T17:57:54.644214+0000","last_fullsized":"2024-01-11T17:57:54.644214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:30:05.869618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637270+0000","last_change":"2024-01-11T17:56:08.797496+0000","last_active":"2024-01-11T17:57:54.637270+0000","last_peered":"2024-01-11T17:57:54.637270+0000","last_clean":"2024-01-11T17:57:54.637270+0000","last_became_active":"2024-01-11T17:56:08.797165+0000","last_became_peered":"2024-01-11T17:56:08.797165+0000","last_unstale":"2024-01-11T17:57:54.637270+0000","last_undegraded":"2024-01-11T17:57:54.637270+0000","last_fullsized":"2024-01-11T17:57:54.637270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:36:21.836645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639445+0000","last_change":"2024-01-11T17:56:08.806580+0000","last_active":"2024-01-11T17:57:54.639445+0000","last_peered":"2024-01-11T17:57:54.639445+0000","last_clean":"2024-01-11T17:57:54.639445+0000","last_became_active":"2024-01-11T17:56:08.806221+0000","last_became_peered":"2024-01-11T17:56:08.806221+0000","last_unstale":"2024-01-11T17:57:54.639445+0000","last_undegraded":"2024-01-11T17:57:54.639445+0000","last_fullsized":"2024-01-11T17:57:54.639445+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:15:47.660838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641839+0000","last_change":"2024-01-11T17:56:08.804164+0000","last_active":"2024-01-11T17:57:54.641839+0000","last_peered":"2024-01-11T17:57:54.641839+0000","last_clean":"2024-01-11T17:57:54.641839+0000","last_became_active":"2024-01-11T17:56:08.800290+0000","last_became_peered":"2024-01-11T17:56:08.800290+0000","last_unstale":"2024-01-11T17:57:54.641839+0000","last_undegraded":"2024-01-11T17:57:54.641839+0000","last_fullsized":"2024-01-11T17:57:54.641839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:25:16.872502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644992+0000","last_change":"2024-01-11T17:56:08.802999+0000","last_active":"2024-01-11T17:57:54.644992+0000","last_peered":"2024-01-11T17:57:54.644992+0000","last_clean":"2024-01-11T17:57:54.644992+0000","last_became_active":"2024-01-11T17:56:08.802621+0000","last_became_peered":"2024-01-11T17:56:08.802621+0000","last_unstale":"2024-01-11T17:57:54.644992+0000","last_undegraded":"2024-01-11T17:57:54.644992+0000","last_fullsized":"2024-01-11T17:57:54.644992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:18:37.393083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685921+0000","last_change":"2024-01-11T17:56:12.834876+0000","last_active":"2024-01-11T17:57:26.685921+0000","last_peered":"2024-01-11T17:57:26.685921+0000","last_clean":"2024-01-11T17:57:26.685921+0000","last_became_active":"2024-01-11T17:56:12.834470+0000","last_became_peered":"2024-01-11T17:56:12.834470+0000","last_unstale":"2024-01-11T17:57:26.685921+0000","last_undegraded":"2024-01-11T17:57:26.685921+0000","last_fullsized":"2024-01-11T17:57:26.685921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:26:06.464916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.690057+0000","last_change":"2024-01-11T17:56:07.801088+0000","last_active":"2024-01-11T17:57:26.690057+0000","last_peered":"2024-01-11T17:57:26.690057+0000","last_clean":"2024-01-11T17:57:26.690057+0000","last_became_active":"2024-01-11T17:56:07.800762+0000","last_became_peered":"2024-01-11T17:56:07.800762+0000","last_unstale":"2024-01-11T17:57:26.690057+0000","last_undegraded":"2024-01-11T17:57:26.690057+0000","last_fullsized":"2024-01-11T17:57:26.690057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:40:01.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638522+0000","last_change":"2024-01-11T17:56:08.810773+0000","last_active":"2024-01-11T17:57:54.638522+0000","last_peered":"2024-01-11T17:57:54.638522+0000","last_clean":"2024-01-11T17:57:54.638522+0000","last_became_active":"2024-01-11T17:56:08.809752+0000","last_became_peered":"2024-01-11T17:56:08.809752+0000","last_unstale":"2024-01-11T17:57:54.638522+0000","last_undegraded":"2024-01-11T17:57:54.638522+0000","last_fullsized":"2024-01-11T17:57:54.638522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:41:21.339998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346095+0000","last_change":"2024-01-11T17:56:12.830816+0000","last_active":"2024-01-11T17:57:26.346095+0000","last_peered":"2024-01-11T17:57:26.346095+0000","last_clean":"2024-01-11T17:57:26.346095+0000","last_became_active":"2024-01-11T17:56:12.830286+0000","last_became_peered":"2024-01-11T17:56:12.830286+0000","last_unstale":"2024-01-11T17:57:26.346095+0000","last_undegraded":"2024-01-11T17:57:26.346095+0000","last_fullsized":"2024-01-11T17:57:26.346095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:54:06.121187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.412849+0000","last_change":"2024-01-11T17:56:07.812537+0000","last_active":"2024-01-11T17:57:26.412849+0000","last_peered":"2024-01-11T17:57:26.412849+0000","last_clean":"2024-01-11T17:57:26.412849+0000","last_became_active":"2024-01-11T17:56:07.811486+0000","last_became_peered":"2024-01-11T17:56:07.811486+0000","last_unstale":"2024-01-11T17:57:26.412849+0000","last_undegraded":"2024-01-11T17:57:26.412849+0000","last_fullsized":"2024-01-11T17:57:26.412849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:41:48.230960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.640275+0000","last_change":"2024-01-11T17:56:08.802648+0000","last_active":"2024-01-11T17:57:54.640275+0000","last_peered":"2024-01-11T17:57:54.640275+0000","last_clean":"2024-01-11T17:57:54.640275+0000","last_became_active":"2024-01-11T17:56:08.798247+0000","last_became_peered":"2024-01-11T17:56:08.798247+0000","last_unstale":"2024-01-11T17:57:54.640275+0000","last_undegraded":"2024-01-11T17:57:54.640275+0000","last_fullsized":"2024-01-11T17:57:54.640275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:58:12.760303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346280+0000","last_change":"2024-01-11T17:56:12.831267+0000","last_active":"2024-01-11T17:57:26.346280+0000","last_peered":"2024-01-11T17:57:26.346280+0000","last_clean":"2024-01-11T17:57:26.346280+0000","last_became_active":"2024-01-11T17:56:12.830764+0000","last_became_peered":"2024-01-11T17:56:12.830764+0000","last_unstale":"2024-01-11T17:57:26.346280+0000","last_undegraded":"2024-01-11T17:57:26.346280+0000","last_fullsized":"2024-01-11T17:57:26.346280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:17:52.954965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.349086+0000","last_change":"2024-01-11T17:56:07.810116+0000","last_active":"2024-01-11T17:57:26.349086+0000","last_peered":"2024-01-11T17:57:26.349086+0000","last_clean":"2024-01-11T17:57:26.349086+0000","last_became_active":"2024-01-11T17:56:07.809785+0000","last_became_peered":"2024-01-11T17:56:07.809785+0000","last_unstale":"2024-01-11T17:57:26.349086+0000","last_undegraded":"2024-01-11T17:57:26.349086+0000","last_fullsized":"2024-01-11T17:57:26.349086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:36.675829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645057+0000","last_change":"2024-01-11T17:56:08.799784+0000","last_active":"2024-01-11T17:57:54.645057+0000","last_peered":"2024-01-11T17:57:54.645057+0000","last_clean":"2024-01-11T17:57:54.645057+0000","last_became_active":"2024-01-11T17:56:08.799432+0000","last_became_peered":"2024-01-11T17:56:08.799432+0000","last_unstale":"2024-01-11T17:57:54.645057+0000","last_undegraded":"2024-01-11T17:57:54.645057+0000","last_fullsized":"2024-01-11T17:57:54.645057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:54:01.173424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346157+0000","last_change":"2024-01-11T17:56:12.827104+0000","last_active":"2024-01-11T17:57:26.346157+0000","last_peered":"2024-01-11T17:57:26.346157+0000","last_clean":"2024-01-11T17:57:26.346157+0000","last_became_active":"2024-01-11T17:56:12.826699+0000","last_became_peered":"2024-01-11T17:56:12.826699+0000","last_unstale":"2024-01-11T17:57:26.346157+0000","last_undegraded":"2024-01-11T17:57:26.346157+0000","last_fullsized":"2024-01-11T17:57:26.346157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:37:34.811258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689779+0000","last_change":"2024-01-11T17:56:26.349877+0000","last_active":"2024-01-11T17:57:26.689779+0000","last_peered":"2024-01-11T17:57:26.689779+0000","last_clean":"2024-01-11T17:57:26.689779+0000","last_became_active":"2024-01-11T17:56:26.346373+0000","last_became_peered":"2024-01-11T17:56:26.346373+0000","last_unstale":"2024-01-11T17:57:26.689779+0000","last_undegraded":"2024-01-11T17:57:26.689779+0000","last_fullsized":"2024-01-11T17:57:26.689779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:44:08.146079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.632027+0000","last_change":"2024-01-11T17:56:08.801569+0000","last_active":"2024-01-11T17:57:54.632027+0000","last_peered":"2024-01-11T17:57:54.632027+0000","last_clean":"2024-01-11T17:57:54.632027+0000","last_became_active":"2024-01-11T17:56:08.801121+0000","last_became_peered":"2024-01-11T17:56:08.801121+0000","last_unstale":"2024-01-11T17:57:54.632027+0000","last_undegraded":"2024-01-11T17:57:54.632027+0000","last_fullsized":"2024-01-11T17:57:54.632027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:42:34.897925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686233+0000","last_change":"2024-01-11T17:56:12.827633+0000","last_active":"2024-01-11T17:57:26.686233+0000","last_peered":"2024-01-11T17:57:26.686233+0000","last_clean":"2024-01-11T17:57:26.686233+0000","last_became_active":"2024-01-11T17:56:12.827124+0000","last_became_peered":"2024-01-11T17:56:12.827124+0000","last_unstale":"2024-01-11T17:57:26.686233+0000","last_undegraded":"2024-01-11T17:57:26.686233+0000","last_fullsized":"2024-01-11T17:57:26.686233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:20.106012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:58.463801+0000","last_change":"2024-01-11T17:56:07.809203+0000","last_active":"2024-01-11T17:57:58.463801+0000","last_peered":"2024-01-11T17:57:58.463801+0000","last_clean":"2024-01-11T17:57:58.463801+0000","last_became_active":"2024-01-11T17:56:07.808808+0000","last_became_peered":"2024-01-11T17:56:07.808808+0000","last_unstale":"2024-01-11T17:57:58.463801+0000","last_undegraded":"2024-01-11T17:57:58.463801+0000","last_fullsized":"2024-01-11T17:57:58.463801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:03:41.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1459,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.620867+0000","last_change":"2024-01-11T17:56:08.795048+0000","last_active":"2024-01-11T17:57:54.620867+0000","last_peered":"2024-01-11T17:57:54.620867+0000","last_clean":"2024-01-11T17:57:54.620867+0000","last_became_active":"2024-01-11T17:56:08.794555+0000","last_became_peered":"2024-01-11T17:56:08.794555+0000","last_unstale":"2024-01-11T17:57:54.620867+0000","last_undegraded":"2024-01-11T17:57:54.620867+0000","last_fullsized":"2024-01-11T17:57:54.620867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:39:32.099084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686126+0000","last_change":"2024-01-11T17:56:12.832040+0000","last_active":"2024-01-11T17:57:26.686126+0000","last_peered":"2024-01-11T17:57:26.686126+0000","last_clean":"2024-01-11T17:57:26.686126+0000","last_became_active":"2024-01-11T17:56:12.831582+0000","last_became_peered":"2024-01-11T17:56:12.831582+0000","last_unstale":"2024-01-11T17:57:26.686126+0000","last_undegraded":"2024-01-11T17:57:26.686126+0000","last_fullsized":"2024-01-11T17:57:26.686126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:03.767396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406841+0000","last_change":"2024-01-11T17:56:07.802799+0000","last_active":"2024-01-11T17:57:26.406841+0000","last_peered":"2024-01-11T17:57:26.406841+0000","last_clean":"2024-01-11T17:57:26.406841+0000","last_became_active":"2024-01-11T17:56:07.802392+0000","last_became_peered":"2024-01-11T17:56:07.802392+0000","last_unstale":"2024-01-11T17:57:26.406841+0000","last_undegraded":"2024-01-11T17:57:26.406841+0000","last_fullsized":"2024-01-11T17:57:26.406841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:20:49.457480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646041+0000","last_change":"2024-01-11T17:56:08.799233+0000","last_active":"2024-01-11T17:57:54.646041+0000","last_peered":"2024-01-11T17:57:54.646041+0000","last_clean":"2024-01-11T17:57:54.646041+0000","last_became_active":"2024-01-11T17:56:08.798645+0000","last_became_peered":"2024-01-11T17:56:08.798645+0000","last_unstale":"2024-01-11T17:57:54.646041+0000","last_undegraded":"2024-01-11T17:57:54.646041+0000","last_fullsized":"2024-01-11T17:57:54.646041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:16:04.018139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.690008+0000","last_change":"2024-01-11T17:56:12.837876+0000","last_active":"2024-01-11T17:57:26.690008+0000","last_peered":"2024-01-11T17:57:26.690008+0000","last_clean":"2024-01-11T17:57:26.690008+0000","last_became_active":"2024-01-11T17:56:12.837453+0000","last_became_peered":"2024-01-11T17:56:12.837453+0000","last_unstale":"2024-01-11T17:57:26.690008+0000","last_undegraded":"2024-01-11T17:57:26.690008+0000","last_fullsized":"2024-01-11T17:57:26.690008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:05:37.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406421+0000","last_change":"2024-01-11T17:56:07.788376+0000","last_active":"2024-01-11T17:57:26.406421+0000","last_peered":"2024-01-11T17:57:26.406421+0000","last_clean":"2024-01-11T17:57:26.406421+0000","last_became_active":"2024-01-11T17:56:07.787856+0000","last_became_peered":"2024-01-11T17:56:07.787856+0000","last_unstale":"2024-01-11T17:57:26.406421+0000","last_undegraded":"2024-01-11T17:57:26.406421+0000","last_fullsized":"2024-01-11T17:57:26.406421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:48:19.047895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646879+0000","last_change":"2024-01-11T17:56:08.796398+0000","last_active":"2024-01-11T17:57:54.646879+0000","last_peered":"2024-01-11T17:57:54.646879+0000","last_clean":"2024-01-11T17:57:54.646879+0000","last_became_active":"2024-01-11T17:56:08.795801+0000","last_became_peered":"2024-01-11T17:56:08.795801+0000","last_unstale":"2024-01-11T17:57:54.646879+0000","last_undegraded":"2024-01-11T17:57:54.646879+0000","last_fullsized":"2024-01-11T17:57:54.646879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:21:28.961576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.345977+0000","last_change":"2024-01-11T17:56:12.837870+0000","last_active":"2024-01-11T17:57:26.345977+0000","last_peered":"2024-01-11T17:57:26.345977+0000","last_clean":"2024-01-11T17:57:26.345977+0000","last_became_active":"2024-01-11T17:56:12.837111+0000","last_became_peered":"2024-01-11T17:56:12.837111+0000","last_unstale":"2024-01-11T17:57:26.345977+0000","last_undegraded":"2024-01-11T17:57:26.345977+0000","last_fullsized":"2024-01-11T17:57:26.345977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:07:19.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409651+0000","last_change":"2024-01-11T17:56:07.805607+0000","last_active":"2024-01-11T17:57:26.409651+0000","last_peered":"2024-01-11T17:57:26.409651+0000","last_clean":"2024-01-11T17:57:26.409651+0000","last_became_active":"2024-01-11T17:56:07.805075+0000","last_became_peered":"2024-01-11T17:56:07.805075+0000","last_unstale":"2024-01-11T17:57:26.409651+0000","last_undegraded":"2024-01-11T17:57:26.409651+0000","last_fullsized":"2024-01-11T17:57:26.409651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:03:49.334249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638052+0000","last_change":"2024-01-11T17:56:08.802492+0000","last_active":"2024-01-11T17:57:54.638052+0000","last_peered":"2024-01-11T17:57:54.638052+0000","last_clean":"2024-01-11T17:57:54.638052+0000","last_became_active":"2024-01-11T17:56:08.802125+0000","last_became_peered":"2024-01-11T17:56:08.802125+0000","last_unstale":"2024-01-11T17:57:54.638052+0000","last_undegraded":"2024-01-11T17:57:54.638052+0000","last_fullsized":"2024-01-11T17:57:54.638052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:06:50.213297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:28.039741+0000","last_change":"2024-01-11T17:57:28.039741+0000","last_active":"2024-01-11T17:57:28.039741+0000","last_peered":"2024-01-11T17:57:28.039741+0000","last_clean":"2024-01-11T17:57:28.039741+0000","last_became_active":"2024-01-11T17:57:28.039342+0000","last_became_peered":"2024-01-11T17:57:28.039342+0000","last_unstale":"2024-01-11T17:57:28.039741+0000","last_undegraded":"2024-01-11T17:57:28.039741+0000","last_fullsized":"2024-01-11T17:57:28.039741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:52:17.687299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,10],"acting":[3,7,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:58.463276+0000","last_change":"2024-01-11T17:56:07.804346+0000","last_active":"2024-01-11T17:57:58.463276+0000","last_peered":"2024-01-11T17:57:58.463276+0000","last_clean":"2024-01-11T17:57:58.463276+0000","last_became_active":"2024-01-11T17:56:07.803941+0000","last_became_peered":"2024-01-11T17:56:07.803941+0000","last_unstale":"2024-01-11T17:57:58.463276+0000","last_undegraded":"2024-01-11T17:57:58.463276+0000","last_fullsized":"2024-01-11T17:57:58.463276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:16:21.649999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639525+0000","last_change":"2024-01-11T17:56:08.800469+0000","last_active":"2024-01-11T17:57:54.639525+0000","last_peered":"2024-01-11T17:57:54.639525+0000","last_clean":"2024-01-11T17:57:54.639525+0000","last_became_active":"2024-01-11T17:56:08.800133+0000","last_became_peered":"2024-01-11T17:56:08.800133+0000","last_unstale":"2024-01-11T17:57:54.639525+0000","last_undegraded":"2024-01-11T17:57:54.639525+0000","last_fullsized":"2024-01-11T17:57:54.639525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:01:13.004810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689484+0000","last_change":"2024-01-11T17:56:12.823520+0000","last_active":"2024-01-11T17:57:26.689484+0000","last_peered":"2024-01-11T17:57:26.689484+0000","last_clean":"2024-01-11T17:57:26.689484+0000","last_became_active":"2024-01-11T17:56:12.823139+0000","last_became_peered":"2024-01-11T17:56:12.823139+0000","last_unstale":"2024-01-11T17:57:26.689484+0000","last_undegraded":"2024-01-11T17:57:26.689484+0000","last_fullsized":"2024-01-11T17:57:26.689484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:06:01.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415699+0000","last_change":"2024-01-11T17:56:07.806706+0000","last_active":"2024-01-11T17:57:26.415699+0000","last_peered":"2024-01-11T17:57:26.415699+0000","last_clean":"2024-01-11T17:57:26.415699+0000","last_became_active":"2024-01-11T17:56:07.806376+0000","last_became_peered":"2024-01-11T17:56:07.806376+0000","last_unstale":"2024-01-11T17:57:26.415699+0000","last_undegraded":"2024-01-11T17:57:26.415699+0000","last_fullsized":"2024-01-11T17:57:26.415699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:58:40.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636540+0000","last_change":"2024-01-11T17:56:08.807489+0000","last_active":"2024-01-11T17:57:54.636540+0000","last_peered":"2024-01-11T17:57:54.636540+0000","last_clean":"2024-01-11T17:57:54.636540+0000","last_became_active":"2024-01-11T17:56:08.803628+0000","last_became_peered":"2024-01-11T17:56:08.803628+0000","last_unstale":"2024-01-11T17:57:54.636540+0000","last_undegraded":"2024-01-11T17:57:54.636540+0000","last_fullsized":"2024-01-11T17:57:54.636540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:15:28.632493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346392+0000","last_change":"2024-01-11T17:56:12.830963+0000","last_active":"2024-01-11T17:57:26.346392+0000","last_peered":"2024-01-11T17:57:26.346392+0000","last_clean":"2024-01-11T17:57:26.346392+0000","last_became_active":"2024-01-11T17:56:12.830334+0000","last_became_peered":"2024-01-11T17:56:12.830334+0000","last_unstale":"2024-01-11T17:57:26.346392+0000","last_undegraded":"2024-01-11T17:57:26.346392+0000","last_fullsized":"2024-01-11T17:57:26.346392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:32.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410334+0000","last_change":"2024-01-11T17:56:07.801043+0000","last_active":"2024-01-11T17:57:26.410334+0000","last_peered":"2024-01-11T17:57:26.410334+0000","last_clean":"2024-01-11T17:57:26.410334+0000","last_became_active":"2024-01-11T17:56:07.800690+0000","last_became_peered":"2024-01-11T17:56:07.800690+0000","last_unstale":"2024-01-11T17:57:26.410334+0000","last_undegraded":"2024-01-11T17:57:26.410334+0000","last_fullsized":"2024-01-11T17:57:26.410334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:28.956375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639586+0000","last_change":"2024-01-11T17:56:08.795702+0000","last_active":"2024-01-11T17:57:54.639586+0000","last_peered":"2024-01-11T17:57:54.639586+0000","last_clean":"2024-01-11T17:57:54.639586+0000","last_became_active":"2024-01-11T17:56:08.795307+0000","last_became_peered":"2024-01-11T17:56:08.795307+0000","last_unstale":"2024-01-11T17:57:54.639586+0000","last_undegraded":"2024-01-11T17:57:54.639586+0000","last_fullsized":"2024-01-11T17:57:54.639586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:18:29.537245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689581+0000","last_change":"2024-01-11T17:56:12.824857+0000","last_active":"2024-01-11T17:57:26.689581+0000","last_peered":"2024-01-11T17:57:26.689581+0000","last_clean":"2024-01-11T17:57:26.689581+0000","last_became_active":"2024-01-11T17:56:12.824443+0000","last_became_peered":"2024-01-11T17:56:12.824443+0000","last_unstale":"2024-01-11T17:57:26.689581+0000","last_undegraded":"2024-01-11T17:57:26.689581+0000","last_fullsized":"2024-01-11T17:57:26.689581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:03.102575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'37","reported_seq":242,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.464956+0000","last_change":"2024-01-11T17:56:07.806951+0000","last_active":"2024-01-11T17:57:59.464956+0000","last_peered":"2024-01-11T17:57:59.464956+0000","last_clean":"2024-01-11T17:57:59.464956+0000","last_became_active":"2024-01-11T17:56:07.806557+0000","last_became_peered":"2024-01-11T17:56:07.806557+0000","last_unstale":"2024-01-11T17:57:59.464956+0000","last_undegraded":"2024-01-11T17:57:59.464956+0000","last_fullsized":"2024-01-11T17:57:59.464956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:23:41.020013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69282,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":184,"num_read_kb":161,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623678+0000","last_change":"2024-01-11T17:56:08.797981+0000","last_active":"2024-01-11T17:57:54.623678+0000","last_peered":"2024-01-11T17:57:54.623678+0000","last_clean":"2024-01-11T17:57:54.623678+0000","last_became_active":"2024-01-11T17:56:08.797531+0000","last_became_peered":"2024-01-11T17:56:08.797531+0000","last_unstale":"2024-01-11T17:57:54.623678+0000","last_undegraded":"2024-01-11T17:57:54.623678+0000","last_fullsized":"2024-01-11T17:57:54.623678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:58:31.994813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410861+0000","last_change":"2024-01-11T17:56:12.830263+0000","last_active":"2024-01-11T17:57:26.410861+0000","last_peered":"2024-01-11T17:57:26.410861+0000","last_clean":"2024-01-11T17:57:26.410861+0000","last_became_active":"2024-01-11T17:56:12.829675+0000","last_became_peered":"2024-01-11T17:56:12.829675+0000","last_unstale":"2024-01-11T17:57:26.410861+0000","last_undegraded":"2024-01-11T17:57:26.410861+0000","last_fullsized":"2024-01-11T17:57:26.410861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:28:52.592447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685932+0000","last_change":"2024-01-11T17:56:07.815781+0000","last_active":"2024-01-11T17:57:26.685932+0000","last_peered":"2024-01-11T17:57:26.685932+0000","last_clean":"2024-01-11T17:57:26.685932+0000","last_became_active":"2024-01-11T17:56:07.815455+0000","last_became_peered":"2024-01-11T17:56:07.815455+0000","last_unstale":"2024-01-11T17:57:26.685932+0000","last_undegraded":"2024-01-11T17:57:26.685932+0000","last_fullsized":"2024-01-11T17:57:26.685932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:19.612256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644250+0000","last_change":"2024-01-11T17:56:08.794938+0000","last_active":"2024-01-11T17:57:54.644250+0000","last_peered":"2024-01-11T17:57:54.644250+0000","last_clean":"2024-01-11T17:57:54.644250+0000","last_became_active":"2024-01-11T17:56:08.794574+0000","last_became_peered":"2024-01-11T17:56:08.794574+0000","last_unstale":"2024-01-11T17:57:54.644250+0000","last_undegraded":"2024-01-11T17:57:54.644250+0000","last_fullsized":"2024-01-11T17:57:54.644250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:03:58.118515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689807+0000","last_change":"2024-01-11T17:56:12.832666+0000","last_active":"2024-01-11T17:57:26.689807+0000","last_peered":"2024-01-11T17:57:26.689807+0000","last_clean":"2024-01-11T17:57:26.689807+0000","last_became_active":"2024-01-11T17:56:12.831877+0000","last_became_peered":"2024-01-11T17:56:12.831877+0000","last_unstale":"2024-01-11T17:57:26.689807+0000","last_undegraded":"2024-01-11T17:57:26.689807+0000","last_fullsized":"2024-01-11T17:57:26.689807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:56:44.111136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410045+0000","last_change":"2024-01-11T17:56:07.812512+0000","last_active":"2024-01-11T17:57:26.410045+0000","last_peered":"2024-01-11T17:57:26.410045+0000","last_clean":"2024-01-11T17:57:26.410045+0000","last_became_active":"2024-01-11T17:56:07.812180+0000","last_became_peered":"2024-01-11T17:56:07.812180+0000","last_unstale":"2024-01-11T17:57:26.410045+0000","last_undegraded":"2024-01-11T17:57:26.410045+0000","last_fullsized":"2024-01-11T17:57:26.410045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:14:06.705666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641093+0000","last_change":"2024-01-11T17:56:08.794672+0000","last_active":"2024-01-11T17:57:54.641093+0000","last_peered":"2024-01-11T17:57:54.641093+0000","last_clean":"2024-01-11T17:57:54.641093+0000","last_became_active":"2024-01-11T17:56:08.794301+0000","last_became_peered":"2024-01-11T17:56:08.794301+0000","last_unstale":"2024-01-11T17:57:54.641093+0000","last_undegraded":"2024-01-11T17:57:54.641093+0000","last_fullsized":"2024-01-11T17:57:54.641093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:52:37.849103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348785+0000","last_change":"2024-01-11T17:56:26.342433+0000","last_active":"2024-01-11T17:57:26.348785+0000","last_peered":"2024-01-11T17:57:26.348785+0000","last_clean":"2024-01-11T17:57:26.348785+0000","last_became_active":"2024-01-11T17:56:26.341923+0000","last_became_peered":"2024-01-11T17:56:26.341923+0000","last_unstale":"2024-01-11T17:57:26.348785+0000","last_undegraded":"2024-01-11T17:57:26.348785+0000","last_fullsized":"2024-01-11T17:57:26.348785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:00:16.050362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.465117+0000","last_change":"2024-01-11T17:56:07.800547+0000","last_active":"2024-01-11T17:57:59.465117+0000","last_peered":"2024-01-11T17:57:59.465117+0000","last_clean":"2024-01-11T17:57:59.465117+0000","last_became_active":"2024-01-11T17:56:07.800152+0000","last_became_peered":"2024-01-11T17:56:07.800152+0000","last_unstale":"2024-01-11T17:57:59.465117+0000","last_undegraded":"2024-01-11T17:57:59.465117+0000","last_fullsized":"2024-01-11T17:57:59.465117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:21:53.397293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.636390+0000","last_change":"2024-01-11T17:56:08.804976+0000","last_active":"2024-01-11T17:57:54.636390+0000","last_peered":"2024-01-11T17:57:54.636390+0000","last_clean":"2024-01-11T17:57:54.636390+0000","last_became_active":"2024-01-11T17:56:08.804627+0000","last_became_peered":"2024-01-11T17:56:08.804627+0000","last_unstale":"2024-01-11T17:57:54.636390+0000","last_undegraded":"2024-01-11T17:57:54.636390+0000","last_fullsized":"2024-01-11T17:57:54.636390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:51:20.538102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406884+0000","last_change":"2024-01-11T17:56:12.840285+0000","last_active":"2024-01-11T17:57:26.406884+0000","last_peered":"2024-01-11T17:57:26.406884+0000","last_clean":"2024-01-11T17:57:26.406884+0000","last_became_active":"2024-01-11T17:56:12.839436+0000","last_became_peered":"2024-01-11T17:56:12.839436+0000","last_unstale":"2024-01-11T17:57:26.406884+0000","last_undegraded":"2024-01-11T17:57:26.406884+0000","last_fullsized":"2024-01-11T17:57:26.406884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:35.318632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348419+0000","last_change":"2024-01-11T17:56:07.799826+0000","last_active":"2024-01-11T17:57:26.348419+0000","last_peered":"2024-01-11T17:57:26.348419+0000","last_clean":"2024-01-11T17:57:26.348419+0000","last_became_active":"2024-01-11T17:56:07.799472+0000","last_became_peered":"2024-01-11T17:56:07.799472+0000","last_unstale":"2024-01-11T17:57:26.348419+0000","last_undegraded":"2024-01-11T17:57:26.348419+0000","last_fullsized":"2024-01-11T17:57:26.348419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:36:54.043354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.629482+0000","last_change":"2024-01-11T17:56:08.801302+0000","last_active":"2024-01-11T17:57:54.629482+0000","last_peered":"2024-01-11T17:57:54.629482+0000","last_clean":"2024-01-11T17:57:54.629482+0000","last_became_active":"2024-01-11T17:56:08.800923+0000","last_became_peered":"2024-01-11T17:56:08.800923+0000","last_unstale":"2024-01-11T17:57:54.629482+0000","last_undegraded":"2024-01-11T17:57:54.629482+0000","last_fullsized":"2024-01-11T17:57:54.629482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:12:43.668185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409801+0000","last_change":"2024-01-11T17:56:12.834043+0000","last_active":"2024-01-11T17:57:26.409801+0000","last_peered":"2024-01-11T17:57:26.409801+0000","last_clean":"2024-01-11T17:57:26.409801+0000","last_became_active":"2024-01-11T17:56:12.833552+0000","last_became_peered":"2024-01-11T17:56:12.833552+0000","last_unstale":"2024-01-11T17:57:26.409801+0000","last_undegraded":"2024-01-11T17:57:26.409801+0000","last_fullsized":"2024-01-11T17:57:26.409801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:53:36.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348962+0000","last_change":"2024-01-11T17:56:07.803601+0000","last_active":"2024-01-11T17:57:26.348962+0000","last_peered":"2024-01-11T17:57:26.348962+0000","last_clean":"2024-01-11T17:57:26.348962+0000","last_became_active":"2024-01-11T17:56:07.803017+0000","last_became_peered":"2024-01-11T17:56:07.803017+0000","last_unstale":"2024-01-11T17:57:26.348962+0000","last_undegraded":"2024-01-11T17:57:26.348962+0000","last_fullsized":"2024-01-11T17:57:26.348962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:40:52.216247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630954+0000","last_change":"2024-01-11T17:56:08.803116+0000","last_active":"2024-01-11T17:57:54.630954+0000","last_peered":"2024-01-11T17:57:54.630954+0000","last_clean":"2024-01-11T17:57:54.630954+0000","last_became_active":"2024-01-11T17:56:08.802674+0000","last_became_peered":"2024-01-11T17:56:08.802674+0000","last_unstale":"2024-01-11T17:57:54.630954+0000","last_undegraded":"2024-01-11T17:57:54.630954+0000","last_fullsized":"2024-01-11T17:57:54.630954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:48:41.981187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348611+0000","last_change":"2024-01-11T17:56:12.975152+0000","last_active":"2024-01-11T17:57:26.348611+0000","last_peered":"2024-01-11T17:57:26.348611+0000","last_clean":"2024-01-11T17:57:26.348611+0000","last_became_active":"2024-01-11T17:56:12.974489+0000","last_became_peered":"2024-01-11T17:56:12.974489+0000","last_unstale":"2024-01-11T17:57:26.348611+0000","last_undegraded":"2024-01-11T17:57:26.348611+0000","last_fullsized":"2024-01-11T17:57:26.348611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:46:48.944847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689942+0000","last_change":"2024-01-11T17:56:07.810951+0000","last_active":"2024-01-11T17:57:26.689942+0000","last_peered":"2024-01-11T17:57:26.689942+0000","last_clean":"2024-01-11T17:57:26.689942+0000","last_became_active":"2024-01-11T17:56:07.809721+0000","last_became_peered":"2024-01-11T17:56:07.809721+0000","last_unstale":"2024-01-11T17:57:26.689942+0000","last_undegraded":"2024-01-11T17:57:26.689942+0000","last_fullsized":"2024-01-11T17:57:26.689942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:24:52.290097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643064+0000","last_change":"2024-01-11T17:56:08.807232+0000","last_active":"2024-01-11T17:57:54.643064+0000","last_peered":"2024-01-11T17:57:54.643064+0000","last_clean":"2024-01-11T17:57:54.643064+0000","last_became_active":"2024-01-11T17:56:08.806866+0000","last_became_peered":"2024-01-11T17:56:08.806866+0000","last_unstale":"2024-01-11T17:57:54.643064+0000","last_undegraded":"2024-01-11T17:57:54.643064+0000","last_fullsized":"2024-01-11T17:57:54.643064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:06:11.770483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406557+0000","last_change":"2024-01-11T17:56:12.837109+0000","last_active":"2024-01-11T17:57:26.406557+0000","last_peered":"2024-01-11T17:57:26.406557+0000","last_clean":"2024-01-11T17:57:26.406557+0000","last_became_active":"2024-01-11T17:56:12.836707+0000","last_became_peered":"2024-01-11T17:56:12.836707+0000","last_unstale":"2024-01-11T17:57:26.406557+0000","last_undegraded":"2024-01-11T17:57:26.406557+0000","last_fullsized":"2024-01-11T17:57:26.406557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:52:43.492096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415027+0000","last_change":"2024-01-11T17:56:07.800622+0000","last_active":"2024-01-11T17:57:26.415027+0000","last_peered":"2024-01-11T17:57:26.415027+0000","last_clean":"2024-01-11T17:57:26.415027+0000","last_became_active":"2024-01-11T17:56:07.800298+0000","last_became_peered":"2024-01-11T17:56:07.800298+0000","last_unstale":"2024-01-11T17:57:26.415027+0000","last_undegraded":"2024-01-11T17:57:26.415027+0000","last_fullsized":"2024-01-11T17:57:26.415027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:52:13.882234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.632525+0000","last_change":"2024-01-11T17:56:08.806347+0000","last_active":"2024-01-11T17:57:54.632525+0000","last_peered":"2024-01-11T17:57:54.632525+0000","last_clean":"2024-01-11T17:57:54.632525+0000","last_became_active":"2024-01-11T17:56:08.805985+0000","last_became_peered":"2024-01-11T17:56:08.805985+0000","last_unstale":"2024-01-11T17:57:54.632525+0000","last_undegraded":"2024-01-11T17:57:54.632525+0000","last_fullsized":"2024-01-11T17:57:54.632525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:12.005741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406685+0000","last_change":"2024-01-11T17:56:12.975082+0000","last_active":"2024-01-11T17:57:26.406685+0000","last_peered":"2024-01-11T17:57:26.406685+0000","last_clean":"2024-01-11T17:57:26.406685+0000","last_became_active":"2024-01-11T17:56:12.974661+0000","last_became_peered":"2024-01-11T17:56:12.974661+0000","last_unstale":"2024-01-11T17:57:26.406685+0000","last_undegraded":"2024-01-11T17:57:26.406685+0000","last_fullsized":"2024-01-11T17:57:26.406685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:34.380974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.631689+0000","last_change":"2024-01-11T17:56:08.784047+0000","last_active":"2024-01-11T17:57:54.631689+0000","last_peered":"2024-01-11T17:57:54.631689+0000","last_clean":"2024-01-11T17:57:54.631689+0000","last_became_active":"2024-01-11T17:56:08.783601+0000","last_became_peered":"2024-01-11T17:56:08.783601+0000","last_unstale":"2024-01-11T17:57:54.631689+0000","last_undegraded":"2024-01-11T17:57:54.631689+0000","last_fullsized":"2024-01-11T17:57:54.631689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:28:20.848204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:59.464545+0000","last_change":"2024-01-11T17:56:07.811735+0000","last_active":"2024-01-11T17:57:59.464545+0000","last_peered":"2024-01-11T17:57:59.464545+0000","last_clean":"2024-01-11T17:57:59.464545+0000","last_became_active":"2024-01-11T17:56:07.810364+0000","last_became_peered":"2024-01-11T17:56:07.810364+0000","last_unstale":"2024-01-11T17:57:59.464545+0000","last_undegraded":"2024-01-11T17:57:59.464545+0000","last_fullsized":"2024-01-11T17:57:59.464545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:13:26.722240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415989+0000","last_change":"2024-01-11T17:56:12.828190+0000","last_active":"2024-01-11T17:57:26.415989+0000","last_peered":"2024-01-11T17:57:26.415989+0000","last_clean":"2024-01-11T17:57:26.415989+0000","last_became_active":"2024-01-11T17:56:12.827764+0000","last_became_peered":"2024-01-11T17:56:12.827764+0000","last_unstale":"2024-01-11T17:57:26.415989+0000","last_undegraded":"2024-01-11T17:57:26.415989+0000","last_fullsized":"2024-01-11T17:57:26.415989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:36:06.169208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.647162+0000","last_change":"2024-01-11T17:56:08.807013+0000","last_active":"2024-01-11T17:57:54.647162+0000","last_peered":"2024-01-11T17:57:54.647162+0000","last_clean":"2024-01-11T17:57:54.647162+0000","last_became_active":"2024-01-11T17:56:08.806722+0000","last_became_peered":"2024-01-11T17:56:08.806722+0000","last_unstale":"2024-01-11T17:57:54.647162+0000","last_undegraded":"2024-01-11T17:57:54.647162+0000","last_fullsized":"2024-01-11T17:57:54.647162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:27:53.350403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346338+0000","last_change":"2024-01-11T17:56:07.793528+0000","last_active":"2024-01-11T17:57:26.346338+0000","last_peered":"2024-01-11T17:57:26.346338+0000","last_clean":"2024-01-11T17:57:26.346338+0000","last_became_active":"2024-01-11T17:56:07.793185+0000","last_became_peered":"2024-01-11T17:56:07.793185+0000","last_unstale":"2024-01-11T17:57:26.346338+0000","last_undegraded":"2024-01-11T17:57:26.346338+0000","last_fullsized":"2024-01-11T17:57:26.346338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:48:05.345701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414453+0000","last_change":"2024-01-11T17:55:12.872243+0000","last_active":"2024-01-11T17:57:26.414453+0000","last_peered":"2024-01-11T17:57:26.414453+0000","last_clean":"2024-01-11T17:57:26.414453+0000","last_became_active":"2024-01-11T17:55:12.871579+0000","last_became_peered":"2024-01-11T17:55:12.871579+0000","last_unstale":"2024-01-11T17:57:26.414453+0000","last_undegraded":"2024-01-11T17:57:26.414453+0000","last_fullsized":"2024-01-11T17:57:26.414453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:52:31.273628+0000","last_clean_scrub_stamp":"2024-01-11T17:52:31.273628+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:16:30.906008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410347+0000","last_change":"2024-01-11T17:56:12.975153+0000","last_active":"2024-01-11T17:57:26.410347+0000","last_peered":"2024-01-11T17:57:26.410347+0000","last_clean":"2024-01-11T17:57:26.410347+0000","last_became_active":"2024-01-11T17:56:12.974714+0000","last_became_peered":"2024-01-11T17:56:12.974714+0000","last_unstale":"2024-01-11T17:57:26.410347+0000","last_undegraded":"2024-01-11T17:57:26.410347+0000","last_fullsized":"2024-01-11T17:57:26.410347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:37:45.096670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.634064+0000","last_change":"2024-01-11T17:56:08.784945+0000","last_active":"2024-01-11T17:57:54.634064+0000","last_peered":"2024-01-11T17:57:54.634064+0000","last_clean":"2024-01-11T17:57:54.634064+0000","last_became_active":"2024-01-11T17:56:08.784110+0000","last_became_peered":"2024-01-11T17:56:08.784110+0000","last_unstale":"2024-01-11T17:57:54.634064+0000","last_undegraded":"2024-01-11T17:57:54.634064+0000","last_fullsized":"2024-01-11T17:57:54.634064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:48:28.596470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414488+0000","last_change":"2024-01-11T17:56:07.806535+0000","last_active":"2024-01-11T17:57:26.414488+0000","last_peered":"2024-01-11T17:57:26.414488+0000","last_clean":"2024-01-11T17:57:26.414488+0000","last_became_active":"2024-01-11T17:56:07.806216+0000","last_became_peered":"2024-01-11T17:56:07.806216+0000","last_unstale":"2024-01-11T17:57:26.414488+0000","last_undegraded":"2024-01-11T17:57:26.414488+0000","last_fullsized":"2024-01-11T17:57:26.414488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:30:49.053827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689755+0000","last_change":"2024-01-11T17:56:12.830457+0000","last_active":"2024-01-11T17:57:26.689755+0000","last_peered":"2024-01-11T17:57:26.689755+0000","last_clean":"2024-01-11T17:57:26.689755+0000","last_became_active":"2024-01-11T17:56:12.830052+0000","last_became_peered":"2024-01-11T17:56:12.830052+0000","last_unstale":"2024-01-11T17:57:26.689755+0000","last_undegraded":"2024-01-11T17:57:26.689755+0000","last_fullsized":"2024-01-11T17:57:26.689755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:24:48.314993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348947+0000","last_change":"2024-01-11T17:56:07.807246+0000","last_active":"2024-01-11T17:57:26.348947+0000","last_peered":"2024-01-11T17:57:26.348947+0000","last_clean":"2024-01-11T17:57:26.348947+0000","last_became_active":"2024-01-11T17:56:07.806975+0000","last_became_peered":"2024-01-11T17:56:07.806975+0000","last_unstale":"2024-01-11T17:57:26.348947+0000","last_undegraded":"2024-01-11T17:57:26.348947+0000","last_fullsized":"2024-01-11T17:57:26.348947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:10:02.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633371+0000","last_change":"2024-01-11T17:56:08.787110+0000","last_active":"2024-01-11T17:57:54.633371+0000","last_peered":"2024-01-11T17:57:54.633371+0000","last_clean":"2024-01-11T17:57:54.633371+0000","last_became_active":"2024-01-11T17:56:08.786832+0000","last_became_peered":"2024-01-11T17:56:08.786832+0000","last_unstale":"2024-01-11T17:57:54.633371+0000","last_undegraded":"2024-01-11T17:57:54.633371+0000","last_fullsized":"2024-01-11T17:57:54.633371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:35:52.393954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.411976+0000","last_change":"2024-01-11T17:56:12.974934+0000","last_active":"2024-01-11T17:57:26.411976+0000","last_peered":"2024-01-11T17:57:26.411976+0000","last_clean":"2024-01-11T17:57:26.411976+0000","last_became_active":"2024-01-11T17:56:12.974467+0000","last_became_peered":"2024-01-11T17:56:12.974467+0000","last_unstale":"2024-01-11T17:57:26.411976+0000","last_undegraded":"2024-01-11T17:57:26.411976+0000","last_fullsized":"2024-01-11T17:57:26.411976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:43.617352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:27.351805+0000","last_change":"2024-01-11T17:56:07.813223+0000","last_active":"2024-01-11T17:57:27.351805+0000","last_peered":"2024-01-11T17:57:27.351805+0000","last_clean":"2024-01-11T17:57:27.351805+0000","last_became_active":"2024-01-11T17:56:07.812847+0000","last_became_peered":"2024-01-11T17:56:07.812847+0000","last_unstale":"2024-01-11T17:57:27.351805+0000","last_undegraded":"2024-01-11T17:57:27.351805+0000","last_fullsized":"2024-01-11T17:57:27.351805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:51:24.985958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638090+0000","last_change":"2024-01-11T17:56:08.802869+0000","last_active":"2024-01-11T17:57:54.638090+0000","last_peered":"2024-01-11T17:57:54.638090+0000","last_clean":"2024-01-11T17:57:54.638090+0000","last_became_active":"2024-01-11T17:56:08.802490+0000","last_became_peered":"2024-01-11T17:56:08.802490+0000","last_unstale":"2024-01-11T17:57:54.638090+0000","last_undegraded":"2024-01-11T17:57:54.638090+0000","last_fullsized":"2024-01-11T17:57:54.638090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:11:53.893076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.685669+0000","last_change":"2024-01-11T17:56:12.825740+0000","last_active":"2024-01-11T17:57:26.685669+0000","last_peered":"2024-01-11T17:57:26.685669+0000","last_clean":"2024-01-11T17:57:26.685669+0000","last_became_active":"2024-01-11T17:56:12.824950+0000","last_became_peered":"2024-01-11T17:56:12.824950+0000","last_unstale":"2024-01-11T17:57:26.685669+0000","last_undegraded":"2024-01-11T17:57:26.685669+0000","last_fullsized":"2024-01-11T17:57:26.685669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:38:49.613865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.347046+0000","last_change":"2024-01-11T17:56:07.805511+0000","last_active":"2024-01-11T17:57:26.347046+0000","last_peered":"2024-01-11T17:57:26.347046+0000","last_clean":"2024-01-11T17:57:26.347046+0000","last_became_active":"2024-01-11T17:56:07.805168+0000","last_became_peered":"2024-01-11T17:56:07.805168+0000","last_unstale":"2024-01-11T17:57:26.347046+0000","last_undegraded":"2024-01-11T17:57:26.347046+0000","last_fullsized":"2024-01-11T17:57:26.347046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:15:09.059817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.642913+0000","last_change":"2024-01-11T17:56:08.797811+0000","last_active":"2024-01-11T17:57:54.642913+0000","last_peered":"2024-01-11T17:57:54.642913+0000","last_clean":"2024-01-11T17:57:54.642913+0000","last_became_active":"2024-01-11T17:56:08.796027+0000","last_became_peered":"2024-01-11T17:56:08.796027+0000","last_unstale":"2024-01-11T17:57:54.642913+0000","last_undegraded":"2024-01-11T17:57:54.642913+0000","last_fullsized":"2024-01-11T17:57:54.642913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:57:54.781890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346710+0000","last_change":"2024-01-11T17:56:26.346059+0000","last_active":"2024-01-11T17:57:26.346710+0000","last_peered":"2024-01-11T17:57:26.346710+0000","last_clean":"2024-01-11T17:57:26.346710+0000","last_became_active":"2024-01-11T17:56:26.345411+0000","last_became_peered":"2024-01-11T17:56:26.345411+0000","last_unstale":"2024-01-11T17:57:26.346710+0000","last_undegraded":"2024-01-11T17:57:26.346710+0000","last_fullsized":"2024-01-11T17:57:26.346710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:49:10.108374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.415555+0000","last_change":"2024-01-11T17:56:07.808303+0000","last_active":"2024-01-11T17:57:26.415555+0000","last_peered":"2024-01-11T17:57:26.415555+0000","last_clean":"2024-01-11T17:57:26.415555+0000","last_became_active":"2024-01-11T17:56:07.808019+0000","last_became_peered":"2024-01-11T17:56:07.808019+0000","last_unstale":"2024-01-11T17:57:26.415555+0000","last_undegraded":"2024-01-11T17:57:26.415555+0000","last_fullsized":"2024-01-11T17:57:26.415555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:05:58.003774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.598022+0000","last_change":"2024-01-11T17:56:08.799287+0000","last_active":"2024-01-11T17:57:54.598022+0000","last_peered":"2024-01-11T17:57:54.598022+0000","last_clean":"2024-01-11T17:57:54.598022+0000","last_became_active":"2024-01-11T17:56:08.798950+0000","last_became_peered":"2024-01-11T17:56:08.798950+0000","last_unstale":"2024-01-11T17:57:54.598022+0000","last_undegraded":"2024-01-11T17:57:54.598022+0000","last_fullsized":"2024-01-11T17:57:54.598022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:14:10.824153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689961+0000","last_change":"2024-01-11T17:56:12.835443+0000","last_active":"2024-01-11T17:57:26.689961+0000","last_peered":"2024-01-11T17:57:26.689961+0000","last_clean":"2024-01-11T17:57:26.689961+0000","last_became_active":"2024-01-11T17:56:12.826615+0000","last_became_peered":"2024-01-11T17:56:12.826615+0000","last_unstale":"2024-01-11T17:57:26.689961+0000","last_undegraded":"2024-01-11T17:57:26.689961+0000","last_fullsized":"2024-01-11T17:57:26.689961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T00:20:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.412881+0000","last_change":"2024-01-11T17:56:07.808963+0000","last_active":"2024-01-11T17:57:26.412881+0000","last_peered":"2024-01-11T17:57:26.412881+0000","last_clean":"2024-01-11T17:57:26.412881+0000","last_became_active":"2024-01-11T17:56:07.805284+0000","last_became_peered":"2024-01-11T17:56:07.805284+0000","last_unstale":"2024-01-11T17:57:26.412881+0000","last_undegraded":"2024-01-11T17:57:26.412881+0000","last_fullsized":"2024-01-11T17:57:26.412881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:39:41.219076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.635273+0000","last_change":"2024-01-11T17:56:08.803909+0000","last_active":"2024-01-11T17:57:54.635273+0000","last_peered":"2024-01-11T17:57:54.635273+0000","last_clean":"2024-01-11T17:57:54.635273+0000","last_became_active":"2024-01-11T17:56:08.803518+0000","last_became_peered":"2024-01-11T17:56:08.803518+0000","last_unstale":"2024-01-11T17:57:54.635273+0000","last_undegraded":"2024-01-11T17:57:54.635273+0000","last_fullsized":"2024-01-11T17:57:54.635273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:43:33.721153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346278+0000","last_change":"2024-01-11T17:56:12.821240+0000","last_active":"2024-01-11T17:57:26.346278+0000","last_peered":"2024-01-11T17:57:26.346278+0000","last_clean":"2024-01-11T17:57:26.346278+0000","last_became_active":"2024-01-11T17:56:12.820729+0000","last_became_peered":"2024-01-11T17:56:12.820729+0000","last_unstale":"2024-01-11T17:57:26.346278+0000","last_undegraded":"2024-01-11T17:57:26.346278+0000","last_fullsized":"2024-01-11T17:57:26.346278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:23:25.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.689783+0000","last_change":"2024-01-11T17:56:26.343805+0000","last_active":"2024-01-11T17:57:26.689783+0000","last_peered":"2024-01-11T17:57:26.689783+0000","last_clean":"2024-01-11T17:57:26.689783+0000","last_became_active":"2024-01-11T17:56:26.343382+0000","last_became_peered":"2024-01-11T17:56:26.343382+0000","last_unstale":"2024-01-11T17:57:26.689783+0000","last_undegraded":"2024-01-11T17:57:26.689783+0000","last_fullsized":"2024-01-11T17:57:26.689783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:26:59.814199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641886+0000","last_change":"2024-01-11T17:56:08.810236+0000","last_active":"2024-01-11T17:57:54.641886+0000","last_peered":"2024-01-11T17:57:54.641886+0000","last_clean":"2024-01-11T17:57:54.641886+0000","last_became_active":"2024-01-11T17:56:08.809697+0000","last_became_peered":"2024-01-11T17:56:08.809697+0000","last_unstale":"2024-01-11T17:57:54.641886+0000","last_undegraded":"2024-01-11T17:57:54.641886+0000","last_fullsized":"2024-01-11T17:57:54.641886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:05.732851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.346709+0000","last_change":"2024-01-11T17:56:12.975620+0000","last_active":"2024-01-11T17:57:26.346709+0000","last_peered":"2024-01-11T17:57:26.346709+0000","last_clean":"2024-01-11T17:57:26.346709+0000","last_became_active":"2024-01-11T17:56:12.974579+0000","last_became_peered":"2024-01-11T17:56:12.974579+0000","last_unstale":"2024-01-11T17:57:26.346709+0000","last_undegraded":"2024-01-11T17:57:26.346709+0000","last_fullsized":"2024-01-11T17:57:26.346709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T21:05:16.995218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.348623+0000","last_change":"2024-01-11T17:56:07.810162+0000","last_active":"2024-01-11T17:57:26.348623+0000","last_peered":"2024-01-11T17:57:26.348623+0000","last_clean":"2024-01-11T17:57:26.348623+0000","last_became_active":"2024-01-11T17:56:07.809811+0000","last_became_peered":"2024-01-11T17:56:07.809811+0000","last_unstale":"2024-01-11T17:57:26.348623+0000","last_undegraded":"2024-01-11T17:57:26.348623+0000","last_fullsized":"2024-01-11T17:57:26.348623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:57:03.623798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633587+0000","last_change":"2024-01-11T17:56:08.794951+0000","last_active":"2024-01-11T17:57:54.633587+0000","last_peered":"2024-01-11T17:57:54.633587+0000","last_clean":"2024-01-11T17:57:54.633587+0000","last_became_active":"2024-01-11T17:56:08.794533+0000","last_became_peered":"2024-01-11T17:56:08.794533+0000","last_unstale":"2024-01-11T17:57:54.633587+0000","last_undegraded":"2024-01-11T17:57:54.633587+0000","last_fullsized":"2024-01-11T17:57:54.633587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:45:36.234904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.414998+0000","last_change":"2024-01-11T17:56:12.826495+0000","last_active":"2024-01-11T17:57:26.414998+0000","last_peered":"2024-01-11T17:57:26.414998+0000","last_clean":"2024-01-11T17:57:26.414998+0000","last_became_active":"2024-01-11T17:56:12.825890+0000","last_became_peered":"2024-01-11T17:56:12.825890+0000","last_unstale":"2024-01-11T17:57:26.414998+0000","last_undegraded":"2024-01-11T17:57:26.414998+0000","last_fullsized":"2024-01-11T17:57:26.414998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:42:17.965875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.686294+0000","last_change":"2024-01-11T17:56:07.807206+0000","last_active":"2024-01-11T17:57:26.686294+0000","last_peered":"2024-01-11T17:57:26.686294+0000","last_clean":"2024-01-11T17:57:26.686294+0000","last_became_active":"2024-01-11T17:56:07.806850+0000","last_became_peered":"2024-01-11T17:56:07.806850+0000","last_unstale":"2024-01-11T17:57:26.686294+0000","last_undegraded":"2024-01-11T17:57:26.686294+0000","last_fullsized":"2024-01-11T17:57:26.686294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:14:07.122688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627483+0000","last_change":"2024-01-11T17:56:08.799566+0000","last_active":"2024-01-11T17:57:54.627483+0000","last_peered":"2024-01-11T17:57:54.627483+0000","last_clean":"2024-01-11T17:57:54.627483+0000","last_became_active":"2024-01-11T17:56:08.798341+0000","last_became_peered":"2024-01-11T17:56:08.798341+0000","last_unstale":"2024-01-11T17:57:54.627483+0000","last_undegraded":"2024-01-11T17:57:54.627483+0000","last_fullsized":"2024-01-11T17:57:54.627483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:01:48.446549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.345942+0000","last_change":"2024-01-11T17:56:26.345480+0000","last_active":"2024-01-11T17:57:26.345942+0000","last_peered":"2024-01-11T17:57:26.345942+0000","last_clean":"2024-01-11T17:57:26.345942+0000","last_became_active":"2024-01-11T17:56:26.345043+0000","last_became_peered":"2024-01-11T17:56:26.345043+0000","last_unstale":"2024-01-11T17:57:26.345942+0000","last_undegraded":"2024-01-11T17:57:26.345942+0000","last_fullsized":"2024-01-11T17:57:26.345942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T01:45:26.054424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.410583+0000","last_change":"2024-01-11T17:56:07.815907+0000","last_active":"2024-01-11T17:57:26.410583+0000","last_peered":"2024-01-11T17:57:26.410583+0000","last_clean":"2024-01-11T17:57:26.410583+0000","last_became_active":"2024-01-11T17:56:07.815514+0000","last_became_peered":"2024-01-11T17:56:07.815514+0000","last_unstale":"2024-01-11T17:57:26.410583+0000","last_undegraded":"2024-01-11T17:57:26.410583+0000","last_fullsized":"2024-01-11T17:57:26.410583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:33:45.798428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.641067+0000","last_change":"2024-01-11T17:56:08.803981+0000","last_active":"2024-01-11T17:57:54.641067+0000","last_peered":"2024-01-11T17:57:54.641067+0000","last_clean":"2024-01-11T17:57:54.641067+0000","last_became_active":"2024-01-11T17:56:08.803615+0000","last_became_peered":"2024-01-11T17:56:08.803615+0000","last_unstale":"2024-01-11T17:57:54.641067+0000","last_undegraded":"2024-01-11T17:57:54.641067+0000","last_fullsized":"2024-01-11T17:57:54.641067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T04:16:11.758827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.409577+0000","last_change":"2024-01-11T17:56:12.824466+0000","last_active":"2024-01-11T17:57:26.409577+0000","last_peered":"2024-01-11T17:57:26.409577+0000","last_clean":"2024-01-11T17:57:26.409577+0000","last_became_active":"2024-01-11T17:56:12.822664+0000","last_became_peered":"2024-01-11T17:56:12.822664+0000","last_unstale":"2024-01-11T17:57:26.409577+0000","last_undegraded":"2024-01-11T17:57:26.409577+0000","last_fullsized":"2024-01-11T17:57:26.409577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:11.789701+0000","last_clean_scrub_stamp":"2024-01-11T17:56:11.789701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:09:07.611078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-11T17:57:26.406896+0000","last_change":"2024-01-11T17:56:07.803278+0000","last_active":"2024-01-11T17:57:26.406896+0000","last_peered":"2024-01-11T17:57:26.406896+0000","last_clean":"2024-01-11T17:57:26.406896+0000","last_became_active":"2024-01-11T17:56:07.802325+0000","last_became_peered":"2024-01-11T17:56:07.802325+0000","last_unstale":"2024-01-11T17:57:26.406896+0000","last_undegraded":"2024-01-11T17:57:26.406896+0000","last_fullsized":"2024-01-11T17:57:26.406896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:06.743439+0000","last_clean_scrub_stamp":"2024-01-11T17:56:06.743439+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:46:55.755617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630352+0000","last_change":"2024-01-11T17:56:08.798160+0000","last_active":"2024-01-11T17:57:54.630352+0000","last_peered":"2024-01-11T17:57:54.630352+0000","last_clean":"2024-01-11T17:57:54.630352+0000","last_became_active":"2024-01-11T17:56:08.797788+0000","last_became_peered":"2024-01-11T17:56:08.797788+0000","last_unstale":"2024-01-11T17:57:54.630352+0000","last_undegraded":"2024-01-11T17:57:54.630352+0000","last_fullsized":"2024-01-11T17:57:54.630352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:59:20.429247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.633236+0000","last_change":"2024-01-11T17:56:08.805519+0000","last_active":"2024-01-11T17:57:54.633236+0000","last_peered":"2024-01-11T17:57:54.633236+0000","last_clean":"2024-01-11T17:57:54.633236+0000","last_became_active":"2024-01-11T17:56:08.805140+0000","last_became_peered":"2024-01-11T17:56:08.805140+0000","last_unstale":"2024-01-11T17:57:54.633236+0000","last_undegraded":"2024-01-11T17:57:54.633236+0000","last_fullsized":"2024-01-11T17:57:54.633236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:42:58.994920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.634289+0000","last_change":"2024-01-11T17:56:26.355996+0000","last_active":"2024-01-11T17:57:54.634289+0000","last_peered":"2024-01-11T17:57:54.634289+0000","last_clean":"2024-01-11T17:57:54.634289+0000","last_became_active":"2024-01-11T17:56:26.355539+0000","last_became_peered":"2024-01-11T17:56:26.355539+0000","last_unstale":"2024-01-11T17:57:54.634289+0000","last_undegraded":"2024-01-11T17:57:54.634289+0000","last_fullsized":"2024-01-11T17:57:54.634289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:24:18.088872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.637772+0000","last_change":"2024-01-11T17:56:08.808393+0000","last_active":"2024-01-11T17:57:54.637772+0000","last_peered":"2024-01-11T17:57:54.637772+0000","last_clean":"2024-01-11T17:57:54.637772+0000","last_became_active":"2024-01-11T17:56:08.808035+0000","last_became_peered":"2024-01-11T17:56:08.808035+0000","last_unstale":"2024-01-11T17:57:54.637772+0000","last_undegraded":"2024-01-11T17:57:54.637772+0000","last_fullsized":"2024-01-11T17:57:54.637772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T03:13:07.496571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.645709+0000","last_change":"2024-01-11T17:56:08.793732+0000","last_active":"2024-01-11T17:57:54.645709+0000","last_peered":"2024-01-11T17:57:54.645709+0000","last_clean":"2024-01-11T17:57:54.645709+0000","last_became_active":"2024-01-11T17:56:08.793422+0000","last_became_peered":"2024-01-11T17:56:08.793422+0000","last_unstale":"2024-01-11T17:57:54.645709+0000","last_undegraded":"2024-01-11T17:57:54.645709+0000","last_fullsized":"2024-01-11T17:57:54.645709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:41:49.588022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.643205+0000","last_change":"2024-01-11T17:56:08.804870+0000","last_active":"2024-01-11T17:57:54.643205+0000","last_peered":"2024-01-11T17:57:54.643205+0000","last_clean":"2024-01-11T17:57:54.643205+0000","last_became_active":"2024-01-11T17:56:08.804441+0000","last_became_peered":"2024-01-11T17:56:08.804441+0000","last_unstale":"2024-01-11T17:57:54.643205+0000","last_undegraded":"2024-01-11T17:57:54.643205+0000","last_fullsized":"2024-01-11T17:57:54.643205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T02:09:58.102056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.630113+0000","last_change":"2024-01-11T17:56:08.803460+0000","last_active":"2024-01-11T17:57:54.630113+0000","last_peered":"2024-01-11T17:57:54.630113+0000","last_clean":"2024-01-11T17:57:54.630113+0000","last_became_active":"2024-01-11T17:56:08.802944+0000","last_became_peered":"2024-01-11T17:56:08.802944+0000","last_unstale":"2024-01-11T17:57:54.630113+0000","last_undegraded":"2024-01-11T17:57:54.630113+0000","last_fullsized":"2024-01-11T17:57:54.630113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:49.382919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.638641+0000","last_change":"2024-01-11T17:56:08.793430+0000","last_active":"2024-01-11T17:57:54.638641+0000","last_peered":"2024-01-11T17:57:54.638641+0000","last_clean":"2024-01-11T17:57:54.638641+0000","last_became_active":"2024-01-11T17:56:08.792989+0000","last_became_peered":"2024-01-11T17:56:08.792989+0000","last_unstale":"2024-01-11T17:57:54.638641+0000","last_undegraded":"2024-01-11T17:57:54.638641+0000","last_fullsized":"2024-01-11T17:57:54.638641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T23:05:59.648606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.627381+0000","last_change":"2024-01-11T17:56:08.797389+0000","last_active":"2024-01-11T17:57:54.627381+0000","last_peered":"2024-01-11T17:57:54.627381+0000","last_clean":"2024-01-11T17:57:54.627381+0000","last_became_active":"2024-01-11T17:56:08.796720+0000","last_became_peered":"2024-01-11T17:56:08.796720+0000","last_unstale":"2024-01-11T17:57:54.627381+0000","last_undegraded":"2024-01-11T17:57:54.627381+0000","last_fullsized":"2024-01-11T17:57:54.627381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:42.875050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.646027+0000","last_change":"2024-01-11T17:56:08.802331+0000","last_active":"2024-01-11T17:57:54.646027+0000","last_peered":"2024-01-11T17:57:54.646027+0000","last_clean":"2024-01-11T17:57:54.646027+0000","last_became_active":"2024-01-11T17:56:08.801905+0000","last_became_peered":"2024-01-11T17:56:08.801905+0000","last_unstale":"2024-01-11T17:57:54.646027+0000","last_undegraded":"2024-01-11T17:57:54.646027+0000","last_fullsized":"2024-01-11T17:57:54.646027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T22:56:06.005698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.639605+0000","last_change":"2024-01-11T17:56:08.801571+0000","last_active":"2024-01-11T17:57:54.639605+0000","last_peered":"2024-01-11T17:57:54.639605+0000","last_clean":"2024-01-11T17:57:54.639605+0000","last_became_active":"2024-01-11T17:56:08.801237+0000","last_became_peered":"2024-01-11T17:56:08.801237+0000","last_unstale":"2024-01-11T17:57:54.639605+0000","last_undegraded":"2024-01-11T17:57:54.639605+0000","last_fullsized":"2024-01-11T17:57:54.639605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:44:13.125584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.647432+0000","last_change":"2024-01-11T17:56:08.798876+0000","last_active":"2024-01-11T17:57:54.647432+0000","last_peered":"2024-01-11T17:57:54.647432+0000","last_clean":"2024-01-11T17:57:54.647432+0000","last_became_active":"2024-01-11T17:56:08.797974+0000","last_became_peered":"2024-01-11T17:56:08.797974+0000","last_unstale":"2024-01-11T17:57:54.647432+0000","last_undegraded":"2024-01-11T17:57:54.647432+0000","last_fullsized":"2024-01-11T17:57:54.647432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-13T05:39:55.917918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.644914+0000","last_change":"2024-01-11T17:56:08.797123+0000","last_active":"2024-01-11T17:57:54.644914+0000","last_peered":"2024-01-11T17:57:54.644914+0000","last_clean":"2024-01-11T17:57:54.644914+0000","last_became_active":"2024-01-11T17:56:08.794497+0000","last_became_peered":"2024-01-11T17:56:08.794497+0000","last_unstale":"2024-01-11T17:57:54.644914+0000","last_undegraded":"2024-01-11T17:57:54.644914+0000","last_fullsized":"2024-01-11T17:57:54.644914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:32:50.116342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-11T17:57:54.623596+0000","last_change":"2024-01-11T17:56:26.355928+0000","last_active":"2024-01-11T17:57:54.623596+0000","last_peered":"2024-01-11T17:57:54.623596+0000","last_clean":"2024-01-11T17:57:54.623596+0000","last_became_active":"2024-01-11T17:56:26.355466+0000","last_became_peered":"2024-01-11T17:56:26.355466+0000","last_unstale":"2024-01-11T17:57:54.623596+0000","last_undegraded":"2024-01-11T17:57:54.623596+0000","last_fullsized":"2024-01-11T17:57:54.623596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T17:56:07.770020+0000","last_clean_scrub_stamp":"2024-01-11T17:56:07.770020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T20:47:48.942958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283821,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74260,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":553,"num_read_kb":357,"num_write":168,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":222780,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":491633,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.497,"5min":0.432,"15min":0.432},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.495,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89800000000000002,"5min":1.109,"15min":1.109},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93600000000000005,"5min":1.048,"15min":1.048},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.53400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.046,"5min":1.405,"15min":1.405},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.748,"15min":0.74299999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93600000000000005,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.755},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88,"15min":0.878},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0680000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.88,"15min":0.88100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0329999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.02,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.91500000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.872}]},{"osd":9,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1779999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.1120000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.94599999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.71199999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1279999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.95299999999999996,"15min":0.94699999999999995},"min":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.139,"5min":1.355,"15min":1.355},"last":0.93000000000000005}]}]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":1148714,"data_compressed":61367,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70099999999999996,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.623}]},{"osd":3,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.751},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82699999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.32300000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.65100000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.93600000000000005,"15min":0.94999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.254,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":8,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.097,"15min":1.097},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.117,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69499999999999995}]},{"osd":9,"last update":"Thu Jan 11 17:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.099,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0529999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.73499999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.93899999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.04,"5min":1.244,"15min":1.244},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.075,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.97899999999999998}]}]},{"osd":9,"up_from":63,"seq":270582939688,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":495965,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81200000000000006,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81200000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.73999999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75700000000000001,"15min":0.77100000000000002},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90500000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86199999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.75700000000000001,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79700000000000004,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90600000000000003,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81100000000000005,"15min":0.81999999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96199999999999997,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.198,"15min":1.198},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.85999999999999999},"min":{"1min":0.61499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93899999999999995,"5min":1.224,"15min":1.224},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88900000000000001,"15min":0.93100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.139,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.91900000000000004,"15min":0.96699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Jan 11 17:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.91900000000000004,"15min":0.97299999999999998},"min":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0069999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93300000000000005,"15min":0.96899999999999997},"min":{"1min":0.68500000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.022,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.94799999999999995},"min":{"1min":0.622,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96099999999999997,"5min":1.284,"15min":1.284},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0309999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 11 17:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85499999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":491789,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.38300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.624}]},{"osd":3,"last update":"Thu Jan 11 17:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":4,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":1.097},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.998,"5min":1,"15min":1},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.877},{"interface":"front","average":{"1min":0.871,"5min":0.85099999999999998,"15min":0.83399999999999996},"min":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.135,"5min":1.163,"15min":1.163},"last":0.72699999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89600000000000002,"15min":0.89800000000000002},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.095,"5min":1.125,"15min":1.125},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.024,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77600000000000002}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.88700000000000001,"15min":0.871},"min":{"1min":0.624,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.81299999999999994,"15min":0.79500000000000004},"min":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.85599999999999998}]}]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1088675,"data_compressed":59744,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.55800000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.625},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68200000000000005,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.626,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90300000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.81499999999999995,"5min":1.137,"15min":1.137},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.56000000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.755,"15min":0.73899999999999999},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.64500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98699999999999999,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97199999999999998,"5min":1.04,"15min":1.04},"last":0.53700000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81499999999999995,"15min":0.80300000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.018,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81299999999999994,"15min":0.81200000000000006},"min":{"1min":0.68200000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96899999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.95199999999999996}]},{"osd":9,"last update":"Thu Jan 11 17:57:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0600000000000001,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80500000000000005,"15min":0.79600000000000004},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.82999999999999996},"min":{"1min":0.79500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.92500000000000004}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0129999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":487045,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.149,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58199999999999996,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":2.149,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.753,"15min":0.71499999999999997},"min":{"1min":0.45100000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.085,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66300000000000003,"15min":0.61499999999999999},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.907,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1279999999999999,"5min":1.161,"15min":1.161},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73899999999999999,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.72,"5min":2.645,"15min":2.645},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.502,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.8029999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.627}]},{"osd":4,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1869999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.84699999999999998,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5349999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.75}]},{"osd":5,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.9770000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.77000000000000002,"15min":0.74099999999999999},"min":{"1min":0.57499999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0819999999999999,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.91600000000000004,"15min":0.90100000000000002},"min":{"1min":0.68799999999999994,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.79700000000000004,"15min":0.75600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.91200000000000003,"15min":0.89500000000000002},"min":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4890000000000001,"5min":2.282,"15min":2.282},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.86699999999999999,"15min":0.85099999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.014,"15min":1.028},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.577,"5min":3.044,"15min":3.044},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.92700000000000005,"15min":0.91900000000000004},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2080000000000002,"5min":2.415,"15min":2.415},"last":0.96599999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.92700000000000005,"15min":0.89500000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.093,"15min":1.095},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.2509999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.024}]},{"osd":11,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.98999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":556237,"data_compressed":25274,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.436,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91900000000000004,"5min":1.077,"15min":1.077},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.46200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.98899999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 11 17:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59399999999999997,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61399999999999999,"15min":0.55500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88900000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.54200000000000004},"min":{"1min":0.496,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.97299999999999998,"5min":4.0199999999999996,"15min":4.0199999999999996},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.41199999999999998},"min":{"1min":0.34100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88700000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.46300000000000002},"min":{"1min":0.33800000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":1.02,"15min":1.02},"last":0.753}]},{"osd":5,"last update":"Thu Jan 11 17:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94399999999999995,"5min":1.012,"15min":1.012},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68500000000000005,"15min":0.622},"min":{"1min":0.54300000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93000000000000005,"5min":1.048,"15min":1.048},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0600000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.53600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81599999999999995,"5min":1.048,"15min":1.048},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 11 17:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.64000000000000001},"min":{"1min":0.59699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.016,"5min":1.048,"15min":1.048},"last":0.875}]},{"osd":8,"last update":"Thu Jan 11 17:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.745,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0089999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Thu Jan 11 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.86199999999999999,"15min":0.86399999999999999},"min":{"1min":0.58299999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0880000000000001,"5min":1.21,"15min":1.21},"last":0.95599999999999996}]},{"osd":11,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":488433,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.66600000000000004,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.911,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.66000000000000003,"15min":0.58299999999999996},"min":{"1min":0.502,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.61499999999999999},"min":{"1min":0.55800000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.85799999999999998,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69599999999999995,"15min":0.59599999999999997},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.087,"5min":3.508,"15min":3.508},"last":0.438}]},{"osd":3,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 11 17:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72299999999999998,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.3359999999999999,"5min":3.3359999999999999,"15min":3.3359999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.77400000000000002,"15min":0.65200000000000002},"min":{"1min":0.56200000000000006,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.2320000000000002,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.747}]},{"osd":5,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73399999999999999,"15min":0.68700000000000006},"min":{"1min":0.56899999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.024,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.72099999999999997,"15min":0.64600000000000002},"min":{"1min":0.58299999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2609999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":1.123},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71399999999999997,"15min":0.67300000000000004},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":3.1019999999999999,"5min":3.1019999999999999,"15min":3.1019999999999999},"last":0.86399999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.68500000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5629999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.88,"15min":0.84399999999999997},"min":{"1min":0.64400000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1879999999999999,"5min":3.198,"15min":3.198},"last":0.77200000000000002}]},{"osd":9,"last update":"Thu Jan 11 17:57:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.95099999999999996},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.423,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.97199999999999998,"15min":0.95899999999999996},"min":{"1min":0.72699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.2390000000000001,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.81599999999999995}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.75800000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.0630000000000002,"5min":3.0630000000000002,"15min":3.0630000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.135,"15min":1.145},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":3.4199999999999999,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.0189999999999999}]},{"osd":11,"last update":"Thu Jan 11 17:57:34 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.79000000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5429999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.022,"15min":1.036},"min":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.339,"5min":3.4849999999999999,"15min":3.4849999999999999},"last":1.056}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":558424,"data_compressed":25274,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.745},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83099999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.61599999999999999,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.96199999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.65900000000000003,"15min":0.56499999999999995},"min":{"1min":0.51800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1220000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 11 17:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58299999999999996,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 11 17:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.41099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0249999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0089999999999999,"5min":1.022,"15min":1.022},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90700000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.61099999999999999}]},{"osd":8,"last update":"Thu Jan 11 17:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.74399999999999999,"15min":0.71199999999999997},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.76100000000000001}]},{"osd":9,"last update":"Thu Jan 11 17:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.78800000000000003,"15min":0.78000000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98599999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.87}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0329999999999999,"5min":1.129,"15min":1.129},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88100000000000001,"15min":0.88},"min":{"1min":0.73499999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0469999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78700000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":487239,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.40100000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.249},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.46700000000000003,"15min":0.42599999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78900000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.623,"15min":0.53700000000000003},"min":{"1min":0.56299999999999994,"5min":0.308,"15min":0.308},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.64000000000000001,"15min":0.55400000000000005},"min":{"1min":0.59099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89500000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65500000000000003,"15min":0.60499999999999998},"min":{"1min":0.53300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.628},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68200000000000005,"15min":0.627},"min":{"1min":0.56699999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 11 17:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69399999999999995,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1599999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.72999999999999998,"15min":0.64200000000000002},"min":{"1min":0.64300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3680000000000001,"5min":1.792,"15min":1.792},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73699999999999999,"15min":0.68600000000000005},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1120000000000001,"5min":1.821,"15min":1.821},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 11 17:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 11 17:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.60799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0660000000000001,"5min":1.379,"15min":1.379},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.86199999999999999,"15min":0.84999999999999998},"min":{"1min":0.66300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Thu Jan 11 17:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2669999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.93000000000000005},"min":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3069999999999999,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1579999999999999,"5min":1.52,"15min":1.52},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Jan 11 17:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.119,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.02,"15min":1.036},"min":{"1min":0.77500000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2330000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.876}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":492226,"data_compressed":23651,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.379},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.438,"15min":0.39500000000000002},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57899999999999996,"5min":0.745,"15min":0.745},"last":0.5}]},{"osd":2,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.76900000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94599999999999995,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.877}]},{"osd":3,"last update":"Thu Jan 11 17:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48999999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":1.034,"15min":1.034},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.004,"15min":1.004},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 11 17:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68899999999999995,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95699999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66300000000000003,"15min":0.60099999999999998},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.129,"15min":1.129},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69499999999999995,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86499999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72599999999999998,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89000000000000001,"5min":1.163,"15min":1.163},"last":0.79500000000000004}]},{"osd":7,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.745}]},{"osd":8,"last update":"Thu Jan 11 17:57:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94099999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.61299999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.97099999999999997,"5min":1.238,"15min":1.238},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jan 11 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.878,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.90200000000000002,"15min":0.92400000000000004},"min":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.014,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jan 11 17:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.629,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.95299999999999996,"5min":1.248,"15min":1.248},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.84899999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.92500000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 11 17:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84999999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1083732,"data_compressed":59744,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59099999999999997,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82599999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45600000000000002,"15min":0.44600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47699999999999998,"15min":0.45900000000000002},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65500000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82799999999999996,"5min":1.548,"15min":1.548},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 11 17:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83099999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90400000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":1.22,"15min":1.22},"last":0.755},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91500000000000004,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.91300000000000003}]},{"osd":8,"last update":"Thu Jan 11 17:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95199999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.0309999999999999}]},{"osd":9,"last update":"Thu Jan 11 17:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":1.097,"15min":1.097},"last":0.96099999999999997}]},{"osd":10,"last update":"Thu Jan 11 17:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.79700000000000004,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0189999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.012}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34452,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36093,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":69282,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71469,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71391,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-11T17:58:03.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:58:04.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/2237785056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c81007c0 msgr2=0x7fc2c8100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/2237785056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c81007c0 0x7fc2c8100c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009990 tx=0x7fc2bc009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/2237785056 shutdown_connections 2024-01-11T17:58:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/2237785056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2c80ffe60 0x7fc2c8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/2237785056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c8101180 0x7fc2c8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/2237785056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c81007c0 0x7fc2c8100c40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/2237785056 >> 172.21.15.7:0/2237785056 conn(0x7fc2c80fba60 msgr2=0x7fc2c80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/2237785056 shutdown_connections 2024-01-11T17:58:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.203+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/2237785056 wait complete. 2024-01-11T17:58:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.204+0000 7fc2ce5ff700 1 Processor -- start 2024-01-11T17:58:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.204+0000 7fc2ce5ff700 1 -- start start 2024-01-11T17:58:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ce5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 0x7fc2c81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ce5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2c81007c0 0x7fc2c8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ce5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c8101180 0x7fc2c81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ce5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81a1260 con 0x7fc2c80ffe60 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ce5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81a13a0 con 0x7fc2c8101180 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ce5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81a1530 con 0x7fc2c81007c0 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 0x7fc2c81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 0x7fc2c81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46772/0 (socket says 172.21.15.7:46772) 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 -- 172.21.15.7:0/1521446085 learned_addr learned my addr 172.21.15.7:0/1521446085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2ccb9c700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c8101180 0x7fc2c81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c77fe700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2c81007c0 0x7fc2c8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 -- 172.21.15.7:0/1521446085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2c81007c0 msgr2=0x7fc2c8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2c81007c0 0x7fc2c8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 -- 172.21.15.7:0/1521446085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c8101180 msgr2=0x7fc2c81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.205+0000 7fc2c7fff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c8101180 0x7fc2c81a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2c7fff700 1 -- 172.21.15.7:0/1521446085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0095f0 con 0x7fc2c80ffe60 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2c7fff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 0x7fc2c81996e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800e9a0 tx=0x7fc2b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8009790 con 0x7fc2c80ffe60 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b8010430 con 0x7fc2c80ffe60 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8018680 con 0x7fc2c80ffe60 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c81a1810 con 0x7fc2c80ffe60 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.206+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c81a1d70 con 0x7fc2c80ffe60 2024-01-11T17:58:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.207+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c810ac40 con 0x7fc2c80ffe60 2024-01-11T17:58:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.209+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc2b80105a0 con 0x7fc2c80ffe60 2024-01-11T17:58:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.210+0000 7fc2c57fa700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2b00a1f60 0x7fc2b00a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.211+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2b8014070 con 0x7fc2c80ffe60 2024-01-11T17:58:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.211+0000 7fc2c77fe700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2b00a1f60 0x7fc2b00a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.211+0000 7fc2c77fe700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2b00a1f60 0x7fc2b00a4420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b2e0 tx=0x7fc2bc00b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.214+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc2b8052380 con 0x7fc2c80ffe60 2024-01-11T17:58:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.431+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c80683f0 con 0x7fc2c80ffe60 2024-01-11T17:58:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.433+0000 7fc2c57fa700 1 -- 172.21.15.7:0/1521446085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fc2b8096ba0 con 0x7fc2c80ffe60 2024-01-11T17:58:04.435 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:04.435 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.435+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2b00a1f60 msgr2=0x7fc2b00a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.435+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2b00a1f60 0x7fc2b00a4420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b2e0 tx=0x7fc2bc00b750 comp rx=0 tx=0).stop 2024-01-11T17:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.435+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 msgr2=0x7fc2c81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.435+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 0x7fc2c81996e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800e9a0 tx=0x7fc2b800ecb0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 shutdown_connections 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2b00a1f60 0x7fc2b00a4420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c8101180 0x7fc2c81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80ffe60 0x7fc2c81996e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 --2- 172.21.15.7:0/1521446085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2c81007c0 0x7fc2c8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 >> 172.21.15.7:0/1521446085 conn(0x7fc2c80fba60 msgr2=0x7fc2c80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 shutdown_connections 2024-01-11T17:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.436+0000 7fc2ce5ff700 1 -- 172.21.15.7:0/1521446085 wait complete. 2024-01-11T17:58:04.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T17:58:04.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T17:58:04.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:04 smithi007 ceph-mon[33361]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:04.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:04 smithi007 ceph-mon[33361]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3173984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 msgr2=0x7f5e68102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3173984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68102c00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e580099c0 tx=0x7f5e58009cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3173984604 shutdown_connections 2024-01-11T17:58:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3173984604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e68103140 0x7f5e681099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3173984604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e681015b0 0x7f5e681019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3173984604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68102c00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3173984604 >> 172.21.15.7:0/3173984604 conn(0x7f5e680fcb00 msgr2=0x7f5e680fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3173984604 shutdown_connections 2024-01-11T17:58:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.665+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3173984604 wait complete. 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.666+0000 7f5e6fdff700 1 Processor -- start 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.666+0000 7f5e6fdff700 1 -- start start 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6fdff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 0x7f5e68199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6fdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6fdff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68103140 0x7f5e681a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6fdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681a11c0 con 0x7f5e681027a0 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6fdff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681a1300 con 0x7f5e68103140 2024-01-11T17:58:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6fdff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681a1490 con 0x7f5e681015b0 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6db9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 0x7f5e68199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6db9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 0x7f5e68199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39768/0 (socket says 172.21.15.7:39768) 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6db9b700 1 -- 172.21.15.7:0/3847091828 learned_addr learned my addr 172.21.15.7:0/3847091828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6d39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6e39c700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68103140 0x7f5e681a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6d39a700 1 -- 172.21.15.7:0/3847091828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 msgr2=0x7f5e68199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6d39a700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 0x7f5e68199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6d39a700 1 -- 172.21.15.7:0/3847091828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68103140 msgr2=0x7f5e681a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6d39a700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68103140 0x7f5e681a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6d39a700 1 -- 172.21.15.7:0/3847091828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580095f0 con 0x7f5e681027a0 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.667+0000 7f5e6db9b700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 0x7f5e68199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.668+0000 7f5e6d39a700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68199b80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e580058e0 tx=0x7f5e5800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.668+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5801d070 con 0x7f5e681027a0 2024-01-11T17:58:04.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.668+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5800fe40 con 0x7f5e681027a0 2024-01-11T17:58:04.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.668+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e681a1710 con 0x7f5e681027a0 2024-01-11T17:58:04.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.668+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58004a40 con 0x7f5e681027a0 2024-01-11T17:58:04.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.668+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e681a1c10 con 0x7f5e681027a0 2024-01-11T17:58:04.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.669+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e680683f0 con 0x7f5e681027a0 2024-01-11T17:58:04.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.671+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f5e58004bc0 con 0x7f5e681027a0 2024-01-11T17:58:04.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.673+0000 7f5e5effd700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5e540a1f60 0x7f5e540a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:04.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.674+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5e580ce940 con 0x7f5e681027a0 2024-01-11T17:58:04.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.674+0000 7f5e6db9b700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5e540a1f60 0x7f5e540a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:04.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.674+0000 7f5e6db9b700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5e540a1f60 0x7f5e540a4420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5e64005950 tx=0x7f5e64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:04.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.678+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f5e580976e0 con 0x7f5e681027a0 2024-01-11T17:58:04.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:04 smithi016 ceph-mon[43195]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:04.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:04 smithi016 ceph-mon[43195]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:04.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:04 smithi038 ceph-mon[43582]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:04.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:04 smithi038 ceph-mon[43582]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.893+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e681a2080 con 0x7f5e681027a0 2024-01-11T17:58:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.894+0000 7f5e5effd700 1 -- 172.21.15.7:0/3847091828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e5809ad00 con 0x7f5e681027a0 2024-01-11T17:58:04.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T17:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5e540a1f60 msgr2=0x7f5e540a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5e540a1f60 0x7f5e540a4420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5e64005950 tx=0x7f5e64009260 comp rx=0 tx=0).stop 2024-01-11T17:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 msgr2=0x7f5e68199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68199b80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e580058e0 tx=0x7f5e5800fa00 comp rx=0 tx=0).stop 2024-01-11T17:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 shutdown_connections 2024-01-11T17:58:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5e540a1f60 0x7f5e540a4420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e681015b0 0x7f5e68199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68103140 0x7f5e681a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 --2- 172.21.15.7:0/3847091828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e681027a0 0x7f5e68199b80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.896+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 >> 172.21.15.7:0/3847091828 conn(0x7f5e680fcb00 msgr2=0x7f5e68073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.897+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 shutdown_connections 2024-01-11T17:58:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:04.897+0000 7f5e6fdff700 1 -- 172.21.15.7:0/3847091828 wait complete. 2024-01-11T17:58:04.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 -- 172.21.15.7:0/1452543585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef041015b0 msgr2=0x7fef041019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 --2- 172.21.15.7:0/1452543585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef041015b0 0x7fef041019d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feef4009990 tx=0x7feef4009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 -- 172.21.15.7:0/1452543585 shutdown_connections 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 --2- 172.21.15.7:0/1452543585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef041015b0 0x7fef041019d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 --2- 172.21.15.7:0/1452543585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef04103140 0x7fef041099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 --2- 172.21.15.7:0/1452543585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041027a0 0x7fef04102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 -- 172.21.15.7:0/1452543585 >> 172.21.15.7:0/1452543585 conn(0x7fef040fcb00 msgr2=0x7fef040fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 -- 172.21.15.7:0/1452543585 shutdown_connections 2024-01-11T17:58:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.125+0000 7fef08bff700 1 -- 172.21.15.7:0/1452543585 wait complete. 2024-01-11T17:58:05.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 Processor -- start 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 -- start start 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 0x7fef04197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef041027a0 0x7fef041979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef04103140 0x7fef0419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0419f010 con 0x7fef041015b0 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0419f150 con 0x7fef041027a0 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.126+0000 7fef08bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0419f2e0 con 0x7fef04103140 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 0x7fef04197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 0x7fef04197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46808/0 (socket says 172.21.15.7:46808) 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 -- 172.21.15.7:0/2872006448 learned_addr learned my addr 172.21.15.7:0/2872006448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef02d9d700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef04103140 0x7fef0419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7feefbfff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef041027a0 0x7fef041979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 -- 172.21.15.7:0/2872006448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef04103140 msgr2=0x7fef0419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef04103140 0x7fef0419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 -- 172.21.15.7:0/2872006448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef041027a0 msgr2=0x7fef041979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef041027a0 0x7fef041979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 -- 172.21.15.7:0/2872006448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009520 con 0x7fef041015b0 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.127+0000 7fef0259c700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 0x7fef04197490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feef40098e0 tx=0x7feef4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.128+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef401d070 con 0x7fef041015b0 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.128+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feef4022470 con 0x7fef041015b0 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.128+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef40095f0 con 0x7fef041015b0 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.128+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4017650 con 0x7fef041015b0 2024-01-11T17:58:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.128+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0419f7f0 con 0x7fef041015b0 2024-01-11T17:58:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.129+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef040683f0 con 0x7fef041015b0 2024-01-11T17:58:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.131+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7feef400fad0 con 0x7fef041015b0 2024-01-11T17:58:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.132+0000 7feefb7fe700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feef00a1ee0 0x7feef00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.132+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feef40cf1f0 con 0x7fef041015b0 2024-01-11T17:58:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.132+0000 7feefbfff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feef00a1ee0 0x7feef00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.133+0000 7feefbfff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feef00a1ee0 0x7feef00a43a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feeec009d80 tx=0x7feeec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.136+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7feef4097ff0 con 0x7fef041015b0 2024-01-11T17:58:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.348+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef04108a30 con 0x7fef041015b0 2024-01-11T17:58:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.348+0000 7feefb7fe700 1 -- 172.21.15.7:0/2872006448 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7feef409b610 con 0x7fef041015b0 2024-01-11T17:58:05.350 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:58:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feef00a1ee0 msgr2=0x7feef00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feef00a1ee0 0x7feef00a43a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feeec009d80 tx=0x7feeec009260 comp rx=0 tx=0).stop 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 msgr2=0x7fef04197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 0x7fef04197490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feef40098e0 tx=0x7feef4004730 comp rx=0 tx=0).stop 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 shutdown_connections 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feef00a1ee0 0x7feef00a43a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef04103140 0x7fef0419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041015b0 0x7fef04197490 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 --2- 172.21.15.7:0/2872006448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef041027a0 0x7fef041979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 >> 172.21.15.7:0/2872006448 conn(0x7fef040fcb00 msgr2=0x7fef04105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 shutdown_connections 2024-01-11T17:58:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.351+0000 7fef08bff700 1 -- 172.21.15.7:0/2872006448 wait complete. 2024-01-11T17:58:05.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T17:58:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1210160793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0075650 msgr2=0x7f1dc0073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1210160793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0075650 0x7f1dc0073aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0098f0 tx=0x7f1dbc009c00 comp rx=0 tx=0).stop 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1210160793 shutdown_connections 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1210160793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0109470 0x7f1dc010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1210160793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0075650 0x7f1dc0073aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1210160793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0074070 0x7f1dc00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1210160793 >> 172.21.15.7:0/1210160793 conn(0x7f1dc00fcb00 msgr2=0x7f1dc00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.579+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1210160793 shutdown_connections 2024-01-11T17:58:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.580+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1210160793 wait complete. 2024-01-11T17:58:05.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.580+0000 7f1dc87ff700 1 Processor -- start 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 -- start start 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0074070 0x7f1dc0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 0x7f1dc0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0109470 0x7f1dc01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc01a1290 con 0x7f1dc0109470 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc01a13d0 con 0x7f1dc0075650 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc87ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc01a1560 con 0x7f1dc0074070 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc5d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 0x7f1dc0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0109470 0x7f1dc01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc5d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 0x7f1dc0199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49310/0 (socket says 172.21.15.7:49310) 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc5d9a700 1 -- 172.21.15.7:0/1365254022 learned_addr learned my addr 172.21.15.7:0/1365254022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.581+0000 7f1dc659b700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0074070 0x7f1dc0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc5d9a700 1 -- 172.21.15.7:0/1365254022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0074070 msgr2=0x7f1dc0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc5d9a700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0074070 0x7f1dc0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc5d9a700 1 -- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0109470 msgr2=0x7f1dc01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc5d9a700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0109470 0x7f1dc01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc5d9a700 1 -- 172.21.15.7:0/1365254022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc009520 con 0x7f1dc0075650 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc6d9c700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0109470 0x7f1dc01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc5d9a700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 0x7f1dc0199c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e970 tx=0x7f1db000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0009790 con 0x7f1dc0075650 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc01a1840 con 0x7f1dc0075650 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db000c7e0 con 0x7f1dc0075650 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0003c70 con 0x7f1dc0075650 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.582+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc01a1d50 con 0x7f1dc0075650 2024-01-11T17:58:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.583+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc0108230 con 0x7f1dc0075650 2024-01-11T17:58:05.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.585+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1db000cc50 con 0x7f1dc0075650 2024-01-11T17:58:05.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.587+0000 7f1db77fe700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1dac0a1ee0 0x7f1dac0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:05.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.587+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1db0014070 con 0x7f1dc0075650 2024-01-11T17:58:05.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.587+0000 7f1dc659b700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1dac0a1ee0 0x7f1dac0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:05.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.588+0000 7f1dc659b700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1dac0a1ee0 0x7f1dac0a43a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00fa40 tx=0x7f1dbc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:05.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.589+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1db0096bc0 con 0x7f1dc0075650 2024-01-11T17:58:05.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:05 smithi016 ceph-mon[43195]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:05.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:05 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1521446085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:05.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:05 smithi016 ceph-mon[43195]: pgmap v308: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-01-11T17:58:05.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:05 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3847091828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:58:05.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:05 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2872006448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:58:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:05 smithi038 ceph-mon[43582]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:05 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1521446085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:05 smithi038 ceph-mon[43582]: pgmap v308: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-01-11T17:58:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:05 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3847091828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:58:05.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:05 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2872006448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:58:05.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.799+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1dc010f190 con 0x7f1dc0075650 2024-01-11T17:58:05.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.800+0000 7f1db77fe700 1 -- 172.21.15.7:0/1365254022 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1db009a1e0 con 0x7f1dc0075650 2024-01-11T17:58:05.802 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T17:58:05.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1dac0a1ee0 msgr2=0x7f1dac0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1dac0a1ee0 0x7f1dac0a43a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00fa40 tx=0x7f1dbc019040 comp rx=0 tx=0).stop 2024-01-11T17:58:05.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 msgr2=0x7f1dc0199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:05.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 0x7f1dc0199c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e970 tx=0x7f1db000ec80 comp rx=0 tx=0).stop 2024-01-11T17:58:05.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 shutdown_connections 2024-01-11T17:58:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1dac0a1ee0 0x7f1dac0a43a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0074070 0x7f1dc0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0109470 0x7f1dc01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 --2- 172.21.15.7:0/1365254022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dc0075650 0x7f1dc0199c50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 >> 172.21.15.7:0/1365254022 conn(0x7f1dc00fcb00 msgr2=0x7f1dc0105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 shutdown_connections 2024-01-11T17:58:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:05.803+0000 7f1dc87ff700 1 -- 172.21.15.7:0/1365254022 wait complete. 2024-01-11T17:58:05.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:05 smithi007 ceph-mon[33361]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-11T17:58:05.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:05 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1521446085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:05.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:05 smithi007 ceph-mon[33361]: pgmap v308: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-01-11T17:58:05.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:05 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3847091828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T17:58:05.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:05 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2872006448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T17:58:05.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 -- 172.21.15.7:0/76345851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60109470 msgr2=0x7f1d6010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 --2- 172.21.15.7:0/76345851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60109470 0x7f1d6010b900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800b0a0 tx=0x7f1d5800b3b0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 -- 172.21.15.7:0/76345851 shutdown_connections 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 --2- 172.21.15.7:0/76345851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60109470 0x7f1d6010b900 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 --2- 172.21.15.7:0/76345851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60075650 0x7f1d60073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 --2- 172.21.15.7:0/76345851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d60074070 0x7f1d600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 -- 172.21.15.7:0/76345851 >> 172.21.15.7:0/76345851 conn(0x7f1d600fcb00 msgr2=0x7f1d600fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 -- 172.21.15.7:0/76345851 shutdown_connections 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.033+0000 7f1d667ff700 1 -- 172.21.15.7:0/76345851 wait complete. 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 Processor -- start 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 -- start start 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 0x7f1d60199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d60075650 0x7f1d60199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60109470 0x7f1d601a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a12f0 con 0x7f1d60074070 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a1430 con 0x7f1d60109470 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.034+0000 7f1d667ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a15c0 con 0x7f1d60075650 2024-01-11T17:58:06.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 0x7f1d60199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 0x7f1d60199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46846/0 (socket says 172.21.15.7:46846) 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 -- 172.21.15.7:0/3419618306 learned_addr learned my addr 172.21.15.7:0/3419618306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5f7fe700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d60075650 0x7f1d60199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d64d9c700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60109470 0x7f1d601a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 -- 172.21.15.7:0/3419618306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d60075650 msgr2=0x7f1d60199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d60075650 0x7f1d60199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 -- 172.21.15.7:0/3419618306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60109470 msgr2=0x7f1d601a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60109470 0x7f1d601a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 -- 172.21.15.7:0/3419618306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d58009d00 con 0x7f1d60074070 2024-01-11T17:58:06.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d64d9c700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60109470 0x7f1d601a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5ffff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 0x7f1d60199770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e9f0 tx=0x7f1d5000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.035+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000eee0 con 0x7f1d60074070 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.036+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d50004cf0 con 0x7f1d60074070 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.036+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50005540 con 0x7f1d60074070 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.036+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d601a18a0 con 0x7f1d60074070 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.036+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601a1d60 con 0x7f1d60074070 2024-01-11T17:58:06.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.037+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d60108230 con 0x7f1d60074070 2024-01-11T17:58:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.039+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1d5000cbd0 con 0x7f1d60074070 2024-01-11T17:58:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.040+0000 7f1d5d7fa700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1d480a1ee0 0x7f1d480a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.040+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d50014070 con 0x7f1d60074070 2024-01-11T17:58:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.040+0000 7f1d5f7fe700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1d480a1ee0 0x7f1d480a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.041+0000 7f1d5f7fe700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1d480a1ee0 0x7f1d480a43a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d60075370 tx=0x7f1d5400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:06.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.042+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1d50096ef0 con 0x7f1d60074070 2024-01-11T17:58:06.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.254+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1d6010f070 con 0x7f1d60074070 2024-01-11T17:58:06.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.254+0000 7f1d5d7fa700 1 -- 172.21.15.7:0/3419618306 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1d5009a510 con 0x7f1d60074070 2024-01-11T17:58:06.256 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T17:58:06.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.256+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1d480a1ee0 msgr2=0x7f1d480a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.256+0000 7f1d667ff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1d480a1ee0 0x7f1d480a43a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d60075370 tx=0x7f1d5400b410 comp rx=0 tx=0).stop 2024-01-11T17:58:06.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.256+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 msgr2=0x7f1d60199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.256+0000 7f1d667ff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 0x7f1d60199770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e9f0 tx=0x7f1d5000ed00 comp rx=0 tx=0).stop 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 shutdown_connections 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1d480a1ee0 0x7f1d480a43a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d60109470 0x7f1d601a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d60074070 0x7f1d60199770 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 --2- 172.21.15.7:0/3419618306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d60075650 0x7f1d60199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 >> 172.21.15.7:0/3419618306 conn(0x7f1d600fcb00 msgr2=0x7f1d60105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 shutdown_connections 2024-01-11T17:58:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.257+0000 7f1d667ff700 1 -- 172.21.15.7:0/3419618306 wait complete. 2024-01-11T17:58:06.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:58:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 -- 172.21.15.7:0/1361422344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 msgr2=0x7f48c81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 --2- 172.21.15.7:0/1361422344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81019b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009990 tx=0x7f48b8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 -- 172.21.15.7:0/1361422344 shutdown_connections 2024-01-11T17:58:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 --2- 172.21.15.7:0/1361422344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8101590 0x7f48c81019b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 --2- 172.21.15.7:0/1361422344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8103120 0x7f48c81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 --2- 172.21.15.7:0/1361422344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8102780 0x7f48c8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 -- 172.21.15.7:0/1361422344 >> 172.21.15.7:0/1361422344 conn(0x7f48c80fcb00 msgr2=0x7f48c80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 -- 172.21.15.7:0/1361422344 shutdown_connections 2024-01-11T17:58:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.483+0000 7f48cddff700 1 -- 172.21.15.7:0/1361422344 wait complete. 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.484+0000 7f48cddff700 1 Processor -- start 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 -- start start 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 0x7f48c80753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8102780 0x7f48c8073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8103120 0x7f48c8073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c819b350 con 0x7f48c8101590 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8074480 con 0x7f48c8103120 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48cddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8074640 con 0x7f48c8102780 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 0x7f48c80753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 0x7f48c80753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46872/0 (socket says 172.21.15.7:46872) 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 -- 172.21.15.7:0/1224020196 learned_addr learned my addr 172.21.15.7:0/1224020196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c6ffd700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8102780 0x7f48c8073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c7fff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8103120 0x7f48c8073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 -- 172.21.15.7:0/1224020196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8102780 msgr2=0x7f48c8073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8102780 0x7f48c8073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 -- 172.21.15.7:0/1224020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8103120 msgr2=0x7f48c8073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.485+0000 7f48c77fe700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8103120 0x7f48c8073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48c77fe700 1 -- 172.21.15.7:0/1224020196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80095f0 con 0x7f48c8101590 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48c77fe700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 0x7f48c80753c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f48b8000c00 tx=0x7f48b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b801d070 con 0x7f48c8101590 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c81a5d00 con 0x7f48c8101590 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b8021410 con 0x7f48c8101590 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8017650 con 0x7f48c8101590 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.486+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c81a6070 con 0x7f48c8101590 2024-01-11T17:58:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.487+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c8108ab0 con 0x7f48c8101590 2024-01-11T17:58:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.489+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f48b8021580 con 0x7f48c8101590 2024-01-11T17:58:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.491+0000 7f48c4ff9700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b00a1ee0 0x7f48b00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.491+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f48b80cf0b0 con 0x7f48c8101590 2024-01-11T17:58:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.491+0000 7f48c6ffd700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b00a1ee0 0x7f48b00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.491+0000 7f48c6ffd700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b00a1ee0 0x7f48b00a43a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009c60 tx=0x7f48bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.494+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f48b8097e50 con 0x7f48c8101590 2024-01-11T17:58:06.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:06 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1365254022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:58:06.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:06 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3419618306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:06.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:06 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1365254022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:58:06.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:06 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3419618306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.758+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48c8068440 con 0x7f48c8101590 2024-01-11T17:58:06.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.758+0000 7f48c4ff9700 1 -- 172.21.15.7:0/1224020196 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11309 (secure 0 0 0) 0x7f48b809b470 con 0x7f48c8101590 2024-01-11T17:58:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:57:57.025520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b00a1ee0 msgr2=0x7f48b00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b00a1ee0 0x7f48b00a43a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009c60 tx=0x7f48bc009260 comp rx=0 tx=0).stop 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 msgr2=0x7f48c80753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 0x7f48c80753c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f48b8000c00 tx=0x7f48b8004730 comp rx=0 tx=0).stop 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 shutdown_connections 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f48b00a1ee0 0x7f48b00a43a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48c8103120 0x7f48c8073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c8101590 0x7f48c80753c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 --2- 172.21.15.7:0/1224020196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48c8102780 0x7f48c8073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 >> 172.21.15.7:0/1224020196 conn(0x7f48c80fcb00 msgr2=0x7f48c8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 shutdown_connections 2024-01-11T17:58:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.761+0000 7f48cddff700 1 -- 172.21.15.7:0/1224020196 wait complete. 2024-01-11T17:58:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-01-11T17:58:06.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:06 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1365254022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T17:58:06.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:06 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3419618306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T17:58:06.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.984+0000 7fdbbcbff700 1 -- 172.21.15.7:0/4020172236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb81015a0 msgr2=0x7fdbb81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.984+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/4020172236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb81015a0 0x7fdbb81019c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdba80099c0 tx=0x7fdba8009cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.984+0000 7fdbbcbff700 1 -- 172.21.15.7:0/4020172236 shutdown_connections 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.985+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/4020172236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb8103130 0x7fdbb81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.985+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/4020172236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb81015a0 0x7fdbb81019c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.985+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/4020172236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb8102790 0x7fdbb8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.985+0000 7fdbbcbff700 1 -- 172.21.15.7:0/4020172236 >> 172.21.15.7:0/4020172236 conn(0x7fdbb80fcb30 msgr2=0x7fdbb80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.985+0000 7fdbbcbff700 1 -- 172.21.15.7:0/4020172236 shutdown_connections 2024-01-11T17:58:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.985+0000 7fdbbcbff700 1 -- 172.21.15.7:0/4020172236 wait complete. 2024-01-11T17:58:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 Processor -- start 2024-01-11T17:58:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 -- start start 2024-01-11T17:58:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 0x7fdbb8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb8102790 0x7fdbb8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb8103130 0x7fdbb81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81a1210 con 0x7fdbb8103130 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81a1350 con 0x7fdbb8102790 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbbcbff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81a14e0 con 0x7fdbb81015a0 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 0x7fdbb8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 0x7fdbb8199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39844/0 (socket says 172.21.15.7:39844) 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbb659c700 1 -- 172.21.15.7:0/1325995101 learned_addr learned my addr 172.21.15.7:0/1325995101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.986+0000 7fdbb6d9d700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb8103130 0x7fdbb81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb5d9b700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb8102790 0x7fdbb8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb659c700 1 -- 172.21.15.7:0/1325995101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb8102790 msgr2=0x7fdbb8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb659c700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb8102790 0x7fdbb8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb659c700 1 -- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb8103130 msgr2=0x7fdbb81a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb659c700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb8103130 0x7fdbb81a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb659c700 1 -- 172.21.15.7:0/1325995101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80095f0 con 0x7fdbb81015a0 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb6d9d700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb8103130 0x7fdbb81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbb659c700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 0x7fdbb8199690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdba8003730 tx=0x7fdba800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba801d070 con 0x7fdbb81015a0 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb81a1760 con 0x7fdbb81015a0 2024-01-11T17:58:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.987+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba8005050 con 0x7fdbb81015a0 2024-01-11T17:58:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.988+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8004970 con 0x7fdbb81015a0 2024-01-11T17:58:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.988+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb81a1c60 con 0x7fdbb81015a0 2024-01-11T17:58:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.988+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb80683f0 con 0x7fdbb81015a0 2024-01-11T17:58:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.991+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fdba8005470 con 0x7fdbb81015a0 2024-01-11T17:58:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.992+0000 7fdba77fe700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdba00a1f60 0x7fdba00a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.992+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdba80cec50 con 0x7fdbb81015a0 2024-01-11T17:58:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.992+0000 7fdbb5d9b700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdba00a1f60 0x7fdba00a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.993+0000 7fdbb5d9b700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdba00a1f60 0x7fdba00a4420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009c60 tx=0x7fdbac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:06.995+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fdba8097a50 con 0x7fdbb81015a0 2024-01-11T17:58:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.212+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbb810cdd0 con 0x7fdbb81015a0 2024-01-11T17:58:07.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.214+0000 7fdba77fe700 1 -- 172.21.15.7:0/1325995101 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fdba809b070 con 0x7fdbb81015a0 2024-01-11T17:58:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:07.216 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T17:57:27.343922+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdba00a1f60 msgr2=0x7fdba00a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdba00a1f60 0x7fdba00a4420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009c60 tx=0x7fdbac009260 comp rx=0 tx=0).stop 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 msgr2=0x7fdbb8199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 0x7fdbb8199690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdba8003730 tx=0x7fdba800f690 comp rx=0 tx=0).stop 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 shutdown_connections 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdba00a1f60 0x7fdba00a4420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbb81015a0 0x7fdbb8199690 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbb8103130 0x7fdbb81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 --2- 172.21.15.7:0/1325995101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbb8102790 0x7fdbb8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 >> 172.21.15.7:0/1325995101 conn(0x7fdbb80fcb30 msgr2=0x7fdbb8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 shutdown_connections 2024-01-11T17:58:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.216+0000 7fdbbcbff700 1 -- 172.21.15.7:0/1325995101 wait complete. 2024-01-11T17:58:07.237 DEBUG:tasks.fwd_scrub:Yielding 2024-01-11T17:58:07.238 INFO:teuthology.run_tasks:Running task background_exec... 2024-01-11T17:58:07.249 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:58:07.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-01-11T17:58:07.250 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-01-11T17:58:07.250 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-01-11T17:58:07.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-01-11T17:58:07.254 INFO:teuthology.run_tasks:Running task exec... 2024-01-11T17:58:07.263 INFO:teuthology.task.exec:Executing custom commands... 2024-01-11T17:58:07.263 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T17:58:07.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-01-11T17:58:07.369 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:07.412 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem Size Used Avail Use% Mounted on 2024-01-11T17:58:07.412 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6789,172.21.15.16:6789,172.21.15.38:6789:/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e 1.1T 380M 1.1T 1% /tmp/cephfs 2024-01-11T17:58:07.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-01-11T17:58:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 -- 172.21.15.7:0/3282829058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 msgr2=0x7f42e01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 --2- 172.21.15.7:0/3282829058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 0x7f42e01099f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42d800b0a0 tx=0x7f42d800b3b0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 -- 172.21.15.7:0/3282829058 shutdown_connections 2024-01-11T17:58:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 --2- 172.21.15.7:0/3282829058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 0x7f42e01099f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 --2- 172.21.15.7:0/3282829058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01015b0 0x7f42e01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 --2- 172.21.15.7:0/3282829058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01027a0 0x7f42e0102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 -- 172.21.15.7:0/3282829058 >> 172.21.15.7:0/3282829058 conn(0x7f42e00fcb00 msgr2=0x7f42e00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 -- 172.21.15.7:0/3282829058 shutdown_connections 2024-01-11T17:58:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.474+0000 7f42e75ff700 1 -- 172.21.15.7:0/3282829058 wait complete. 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.475+0000 7f42e75ff700 1 Processor -- start 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.475+0000 7f42e75ff700 1 -- start start 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e75ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 0x7f42e01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e75ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01027a0 0x7f42e0199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e75ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 0x7f42e01a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e75ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01a12f0 con 0x7f42e01015b0 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e75ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01a1430 con 0x7f42e01027a0 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e75ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01a15c0 con 0x7f42e0103140 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 0x7f42e01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e4b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01027a0 0x7f42e0199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 0x7f42e01997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46912/0 (socket says 172.21.15.7:46912) 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e539b700 1 -- 172.21.15.7:0/1604659029 learned_addr learned my addr 172.21.15.7:0/1604659029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.476+0000 7f42e5b9c700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 0x7f42e01a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e4b9a700 1 -- 172.21.15.7:0/1604659029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 msgr2=0x7f42e01a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 0x7f42e01a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e4b9a700 1 -- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 msgr2=0x7f42e01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 0x7f42e01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e4b9a700 1 -- 172.21.15.7:0/1604659029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8009d00 con 0x7f42e01027a0 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e539b700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 0x7f42e01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01027a0 0x7f42e0199d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42d000e9f0 tx=0x7f42d000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42d67fc700 1 -- 172.21.15.7:0/1604659029 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000c9c0 con 0x7f42e01027a0 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e01a18a0 con 0x7f42e01027a0 2024-01-11T17:58:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42d67fc700 1 -- 172.21.15.7:0/1604659029 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42d0010430 con 0x7f42e01027a0 2024-01-11T17:58:07.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42d67fc700 1 -- 172.21.15.7:0/1604659029 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0018680 con 0x7f42e01027a0 2024-01-11T17:58:07.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.477+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e01a1e00 con 0x7f42e01027a0 2024-01-11T17:58:07.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.481+0000 7f42d67fc700 1 -- 172.21.15.7:0/1604659029 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f42d0010860 con 0x7f42e01027a0 2024-01-11T17:58:07.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.482+0000 7f42fb37bb80 1 Processor -- start 2024-01-11T17:58:07.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.482+0000 7f42fb37bb80 1 -- start start 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42d67fc700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f42cc0a1f60 0x7f42cc0a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770f0e00 0x5633770ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770ef5f0 0x5633770efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563376fe6110 con 0x5633770f0e00 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563377049ba0 con 0x5633770f0480 2024-01-11T17:58:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633770474c0 con 0x5633770ef5f0 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42d67fc700 1 -- 172.21.15.7:0/1604659029 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f42d0014070 con 0x7f42e01027a0 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42e5b9c700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f42cc0a1f60 0x7f42cc0a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42e539b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42e539b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49394/0 (socket says 172.21.15.7:49394) 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42e539b700 1 -- 172.21.15.7:0/2490407071 learned_addr learned my addr 172.21.15.7:0/2490407071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770f0e00 0x5633770ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.483+0000 7f42e5b9c700 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770ef5f0 0x5633770efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e539b700 1 -- 172.21.15.7:0/2490407071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770ef5f0 msgr2=0x5633770efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e539b700 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770ef5f0 0x5633770efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e539b700 1 -- 172.21.15.7:0/2490407071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770f0e00 msgr2=0x5633770ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e539b700 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770f0e00 0x5633770ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e539b700 1 -- 172.21.15.7:0/2490407071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563376eb7430 con 0x5633770f0480 2024-01-11T17:58:07.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e539b700 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f08a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f42dc011650 tx=0x7f42dc011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add2d64ba29d53d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42e5b9c700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f42cc0a1f60 0x7f42cc0a4420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009fd0 tx=0x7f42d8009420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42cb7fe700 1 -- 172.21.15.7:0/2490407071 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc004a30 con 0x5633770f0480 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42cb7fe700 1 -- 172.21.15.7:0/2490407071 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42dc0054e0 con 0x5633770f0480 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.484+0000 7f42cb7fe700 1 -- 172.21.15.7:0/2490407071 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0198c0 con 0x5633770f0480 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2490407071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 msgr2=0x5633770f08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f08a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f42dc011650 tx=0x7f42dc011960 comp rx=0 tx=0).stop 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2490407071 shutdown_connections 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770ef5f0 0x5633770efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f08a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2490407071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770f0e00 0x5633770ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2490407071 >> 172.21.15.7:0/2490407071 conn(0x563377064820 msgr2=0x5633770e9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2490407071 shutdown_connections 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.485+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2490407071 wait complete. 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 -- start start 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 Processor -- start 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0e00 0x563377100430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563377042db0 con 0x5633770ef5f0 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563376fb1420 con 0x5633770f0480 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563376fb10c0 con 0x5633770f0e00 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.486+0000 7f42fb37bb80 10 client.0 _ll_register_callbacks cb 0x563376fe37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e5b9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0e00 0x563377100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49396/0 (socket says 172.21.15.7:49396) 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46918/0 (socket says 172.21.15.7:46918) 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 -- 172.21.15.7:0/3797896450 learned_addr learned my addr 172.21.15.7:0/3797896450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 -- 172.21.15.7:0/3797896450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0e00 msgr2=0x563377100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0e00 0x563377100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 -- 172.21.15.7:0/3797896450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 msgr2=0x5633770f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 -- 172.21.15.7:0/3797896450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc009e10 con 0x5633770f0480 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f93f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f42d00cb520 tx=0x7f42d0014af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645587f4c54d1956 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.487+0000 7f42c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42c8ff9700 1 -- 172.21.15.7:0/3797896450 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00ba110 con 0x5633770f0480 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633770e2090 con 0x5633770f0480 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42c8ff9700 1 -- 172.21.15.7:0/3797896450 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f42d009ae80 con 0x5633770f0480 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42c8ff9700 1 -- 172.21.15.7:0/3797896450 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42d0014be0 con 0x5633770f0480 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42fb37bb80 10 client.25526 fetch_fsmap learned FSMap version 23 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563376e63be0 con 0x5633770f0480 2024-01-11T17:58:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42c8ff9700 1 -- 172.21.15.7:0/3797896450 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633770e2090 con 0x5633770f0480 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.488+0000 7f42c8ff9700 1 -- 172.21.15.7:0/3797896450 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f42d00a2680 con 0x5633770f0480 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42fb37bb80 10 client.25526 fetch_fsmap finished waiting for FSMap version 23 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42fb37bb80 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42fb37bb80 20 client.25526 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563377104c90 0x5633771070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42fb37bb80 4 client.25526 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56337703bc40 con 0x563377104c90 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.489+0000 7f42e539b700 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563377104c90 0x5633771070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.490+0000 7f42e539b700 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563377104c90 0x5633771070b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d79f1e14c3aef8 server_cookie=d2ac0d78f0a43890 in_seq=0 out_seq=0 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.490+0000 7f42c8ff9700 10 client.25526 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.490+0000 7f42c8ff9700 1 -- 172.21.15.7:0/3797896450 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56337703bc40 con 0x563377104c90 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.490+0000 7f42c8ff9700 10 client.25526 handle_command_reply: tid=0 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.490+0000 7f42fb37bb80 1 client.25526 shutdown 2024-01-11T17:58:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 msgr2=0x5633770f93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f93f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f42d00cb520 tx=0x7f42d0014af0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 shutdown_connections 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563377104c90 0x5633771070b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0e00 0x563377100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/3797896450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770f93f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 >> 172.21.15.7:0/3797896450 conn(0x563377064820 msgr2=0x5633770eded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42c8ff9700 0 client.25526 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 shutdown_connections 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 1 -- 172.21.15.7:0/3797896450 wait complete. 2024-01-11T17:58:07.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.491+0000 7f42fb37bb80 20 client.25526 trim_cache size 0 max 16384 2024-01-11T17:58:07.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:07 smithi007 ceph-mon[33361]: pgmap v309: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-01-11T17:58:07.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1224020196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:07.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:07 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1325995101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 Processor -- start 2024-01-11T17:58:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 -- start start 2024-01-11T17:58:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 0x5633770f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633771003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0480 0x5633770f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563376eb9720 con 0x5633770ef5f0 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633770f9e00 con 0x563377064820 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633770f79b0 con 0x5633770f0480 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42e539b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 0x5633770f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42e539b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 0x5633770f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49406/0 (socket says 172.21.15.7:49406) 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.511+0000 7f42e539b700 1 -- 172.21.15.7:0/974881039 learned_addr learned my addr 172.21.15.7:0/974881039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e5b9c700 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0480 0x5633770f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633771003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e539b700 1 -- 172.21.15.7:0/974881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0480 msgr2=0x5633770f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e539b700 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0480 0x5633770f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e539b700 1 -- 172.21.15.7:0/974881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 msgr2=0x5633771003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e539b700 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633771003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e539b700 1 -- 172.21.15.7:0/974881039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633770042b0 con 0x563377064820 2024-01-11T17:58:07.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42e539b700 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 0x5633770f3a90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42dc011650 tx=0x7f42dc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c10f769cb86769 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42caffd700 1 -- 172.21.15.7:0/974881039 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc005ce0 con 0x563377064820 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42caffd700 1 -- 172.21.15.7:0/974881039 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42dc02e070 con 0x563377064820 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.512+0000 7f42caffd700 1 -- 172.21.15.7:0/974881039 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc02a250 con 0x563377064820 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 -- 172.21.15.7:0/974881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 msgr2=0x5633770f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 0x5633770f3a90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42dc011650 tx=0x7f42dc000bc0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 -- 172.21.15.7:0/974881039 shutdown_connections 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5633770f0480 0x5633770f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563377064820 0x5633770f3a90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/974881039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633771003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 -- 172.21.15.7:0/974881039 >> 172.21.15.7:0/974881039 conn(0x5633770ea190 msgr2=0x5633770f7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 -- 172.21.15.7:0/974881039 shutdown_connections 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.513+0000 7f42fb37bb80 1 -- 172.21.15.7:0/974881039 wait complete. 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 -- start start 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 Processor -- start 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 0x5633770feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563377100770 con 0x5633770ef5f0 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56337715b180 con 0x5633770f0480 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56337715b2f0 con 0x563377064820 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.514+0000 7f42fb37bb80 10 client.0 _ll_register_callbacks cb 0x563376fe37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 0x5633770feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 0x5633770feaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 -- 172.21.15.7:0/2237230903 learned_addr learned my addr 172.21.15.7:0/2237230903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e4b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e5b9c700 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 -- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 msgr2=0x5633770ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 -- 172.21.15.7:0/2237230903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 msgr2=0x5633770ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770ff030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.515+0000 7f42e539b700 1 -- 172.21.15.7:0/2237230903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563376eb7430 con 0x563377064820 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42e539b700 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 0x5633770feaf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01b040 tx=0x7f42dc004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ec2c0faed25ed9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42c8ff9700 1 -- 172.21.15.7:0/2237230903 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0046e0 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42dc0054e0 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42c8ff9700 1 -- 172.21.15.7:0/2237230903 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f42dc004a30 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42c8ff9700 1 -- 172.21.15.7:0/2237230903 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42dc004bf0 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42fb37bb80 10 client.25552 fetch_fsmap learned FSMap version 23 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.516+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563376ebbd50 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42c8ff9700 1 -- 172.21.15.7:0/2237230903 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42dc0054e0 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42c8ff9700 1 -- 172.21.15.7:0/2237230903 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f42dc011680 con 0x563377064820 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42fb37bb80 10 client.25552 fetch_fsmap finished waiting for FSMap version 23 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42fb37bb80 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42fb37bb80 20 client.25552 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5633770f0e00 0x56337715ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42fb37bb80 4 client.25552 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-01-11T17:58:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.517+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56337703b960 con 0x5633770f0e00 2024-01-11T17:58:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.518+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5633770f0e00 0x56337715ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.518+0000 7f42e4b9a700 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5633770f0e00 0x56337715ec30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cddd6a83cf85369 server_cookie=71297061ea7e5ea6 in_seq=0 out_seq=0 2024-01-11T17:58:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.518+0000 7f42c8ff9700 10 client.25552 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42c8ff9700 1 -- 172.21.15.7:0/2237230903 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56337703b960 con 0x5633770f0e00 2024-01-11T17:58:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42c8ff9700 10 client.25552 handle_command_reply: tid=0 2024-01-11T17:58:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 client.25552 shutdown 2024-01-11T17:58:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 msgr2=0x5633770feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 0x5633770feaf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01b040 tx=0x7f42dc004e60 comp rx=0 tx=0).stop 2024-01-11T17:58:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 shutdown_connections 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5633770f0e00 0x56337715ec30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563377064820 0x5633770feaf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633770f0480 0x5633770ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42c8ff9700 0 client.25552 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 --2- 172.21.15.7:0/2237230903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633770ef5f0 0x5633770ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 >> 172.21.15.7:0/2237230903 conn(0x5633770ea190 msgr2=0x5633770f6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 shutdown_connections 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 1 -- 172.21.15.7:0/2237230903 wait complete. 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.520+0000 7f42fb37bb80 20 client.25552 trim_cache size 0 max 16384 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:58:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-01-11T17:58:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7eb683f7-d2e5-4da3-a588-694dc4079bbe", 2024-01-11T17:58:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-01-11T17:58:07.525 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:58:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f42cc0a1f60 msgr2=0x7f42cc0a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f42cc0a1f60 0x7f42cc0a4420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009fd0 tx=0x7f42d8009420 comp rx=0 tx=0).stop 2024-01-11T17:58:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01027a0 msgr2=0x7f42e0199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01027a0 0x7f42e0199d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42d000e9f0 tx=0x7f42d000ed00 comp rx=0 tx=0).stop 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 shutdown_connections 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f42cc0a1f60 0x7f42cc0a4420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42e0103140 0x7f42e01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e01015b0 0x7f42e01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 --2- 172.21.15.7:0/1604659029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e01027a0 0x7f42e0199d00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 >> 172.21.15.7:0/1604659029 conn(0x7f42e00fcb00 msgr2=0x7f42e0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 shutdown_connections 2024-01-11T17:58:07.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.521+0000 7f42e75ff700 1 -- 172.21.15.7:0/1604659029 wait complete. 2024-01-11T17:58:07.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-01-11T17:58:07.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-01-11T17:58:07.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-01-11T17:58:07.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:07 smithi016 ceph-mon[43195]: pgmap v309: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-01-11T17:58:07.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1224020196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:07.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:07 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1325995101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 -- 172.21.15.7:0/2872325117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1027f0 msgr2=0x7f5dac102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 --2- 172.21.15.7:0/2872325117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1027f0 0x7f5dac102c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0098f0 tx=0x7f5d9c009c00 comp rx=0 tx=0).stop 2024-01-11T17:58:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 -- 172.21.15.7:0/2872325117 shutdown_connections 2024-01-11T17:58:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 --2- 172.21.15.7:0/2872325117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac103190 0x7f5dac109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 --2- 172.21.15.7:0/2872325117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dac101600 0x7f5dac101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 --2- 172.21.15.7:0/2872325117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac1027f0 0x7f5dac102c50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 -- 172.21.15.7:0/2872325117 >> 172.21.15.7:0/2872325117 conn(0x7f5dac0fcb50 msgr2=0x7f5dac0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.759+0000 7f5db3bff700 1 -- 172.21.15.7:0/2872325117 shutdown_connections 2024-01-11T17:58:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.760+0000 7f5db3bff700 1 -- 172.21.15.7:0/2872325117 wait complete. 2024-01-11T17:58:07.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.760+0000 7f5db3bff700 1 Processor -- start 2024-01-11T17:58:07.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 -- start start 2024-01-11T17:58:07.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 0x7f5dac1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dac1027f0 0x7f5dac1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac103190 0x7f5dac19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac19f030 con 0x7f5dac101600 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac19f170 con 0x7f5dac1027f0 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db3bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac19f300 con 0x7f5dac103190 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 0x7f5dac1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 0x7f5dac1974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46946/0 (socket says 172.21.15.7:46946) 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db199b700 1 -- 172.21.15.7:0/3593247221 learned_addr learned my addr 172.21.15.7:0/3593247221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db119a700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dac1027f0 0x7f5dac1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.761+0000 7f5db219c700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac103190 0x7f5dac19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db199b700 1 -- 172.21.15.7:0/3593247221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac103190 msgr2=0x7f5dac19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db199b700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac103190 0x7f5dac19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db199b700 1 -- 172.21.15.7:0/3593247221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dac1027f0 msgr2=0x7f5dac1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db199b700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dac1027f0 0x7f5dac1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db199b700 1 -- 172.21.15.7:0/3593247221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da80095f0 con 0x7f5dac101600 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db199b700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 0x7f5dac1974b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5da800ebf0 tx=0x7f5da800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5da2ffd700 1 -- 172.21.15.7:0/3593247221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8009a10 con 0x7f5dac101600 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c009520 con 0x7f5dac101600 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5da2ffd700 1 -- 172.21.15.7:0/3593247221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da800cc90 con 0x7f5dac101600 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5da2ffd700 1 -- 172.21.15.7:0/3593247221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8018750 con 0x7f5dac101600 2024-01-11T17:58:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.762+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac19f960 con 0x7f5dac101600 2024-01-11T17:58:07.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.766+0000 7f5da2ffd700 1 -- 172.21.15.7:0/3593247221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f5da80188b0 con 0x7f5dac101600 2024-01-11T17:58:07.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.767+0000 7f5dc7933b80 1 Processor -- start 2024-01-11T17:58:07.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 -- start start 2024-01-11T17:58:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc859f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc859df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 0x556dc859e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc8495090 con 0x556dc859e570 2024-01-11T17:58:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc84f8b20 con 0x556dc859fd80 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc84f6440 con 0x556dc859f400 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.768+0000 7f5da2ffd700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5d980a1ee0 0x7f5d980a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc859df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 0x556dc859e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db219c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc859f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 0x556dc859e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46954/0 (socket says 172.21.15.7:46954) 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc859df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49432/0 (socket says 172.21.15.7:49432) 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db119a700 1 -- 172.21.15.7:0/2057035479 learned_addr learned my addr 172.21.15.7:0/2057035479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5da2ffd700 1 -- 172.21.15.7:0/3593247221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5da8014070 con 0x7f5dac101600 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db219c700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5d980a1ee0 0x7f5d980a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 -- 172.21.15.7:0/2057035479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 msgr2=0x556dc859f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc859f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 -- 172.21.15.7:0/2057035479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 msgr2=0x556dc859e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 0x556dc859e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 -- 172.21.15.7:0/2057035479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dc83663b0 con 0x556dc859fd80 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db119a700 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 0x556dc859e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.769+0000 7f5db199b700 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc859df90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005950 tx=0x7f5da8090260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab9eed97ef9067f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.770+0000 7f5d97fff700 1 -- 172.21.15.7:0/2057035479 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da800ce00 con 0x556dc859fd80 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.770+0000 7f5db219c700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5d980a1ee0 0x7f5d980a43a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009ec0 tx=0x7f5da400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.770+0000 7f5d97fff700 1 -- 172.21.15.7:0/2057035479 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da809b3e0 con 0x556dc859fd80 2024-01-11T17:58:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.770+0000 7f5d97fff700 1 -- 172.21.15.7:0/2057035479 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80cbb90 con 0x556dc859fd80 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.770+0000 7f5dc7933b80 1 -- 172.21.15.7:0/2057035479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 msgr2=0x556dc859df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.770+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc859df90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005950 tx=0x7f5da8090260 comp rx=0 tx=0).stop 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 -- 172.21.15.7:0/2057035479 shutdown_connections 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc859f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859e570 0x556dc859e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/2057035479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc859df90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 -- 172.21.15.7:0/2057035479 >> 172.21.15.7:0/2057035479 conn(0x556dc85137a0 msgr2=0x556dc8598de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 -- 172.21.15.7:0/2057035479 shutdown_connections 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 -- 172.21.15.7:0/2057035479 wait complete. 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 -- start start 2024-01-11T17:58:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.771+0000 7f5dc7933b80 1 Processor -- start 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859e570 0x556dc85a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc85a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc84f1d30 con 0x556dc859fd80 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc84603a0 con 0x556dc859e570 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc8460040 con 0x556dc859f400 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5dc7933b80 10 client.0 _ll_register_callbacks cb 0x556dc8492720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5db119a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc85a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5db219c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5db119a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc85a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39948/0 (socket says 172.21.15.7:39948) 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5db119a700 1 -- 172.21.15.7:0/1212793461 learned_addr learned my addr 172.21.15.7:0/1212793461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.772+0000 7f5db199b700 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859e570 0x556dc85a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5db119a700 1 -- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859e570 msgr2=0x556dc85a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5db119a700 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859e570 0x556dc85a7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5db119a700 1 -- 172.21.15.7:0/1212793461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 msgr2=0x556dc85af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5db119a700 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5db119a700 1 -- 172.21.15.7:0/1212793461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac10a390 con 0x556dc859f400 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5db119a700 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc85a8400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00fa70 tx=0x7f5d9c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=446e8788575225f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5d957fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5d957fa700 1 -- 172.21.15.7:0/1212793461 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c005040 con 0x556dc859f400 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556dc8591010 con 0x556dc859f400 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5d957fa700 1 -- 172.21.15.7:0/1212793461 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5d9c0051a0 con 0x556dc859f400 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5d957fa700 1 -- 172.21.15.7:0/1212793461 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9c005360 con 0x556dc859f400 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5dc7933b80 10 client.25564 fetch_fsmap learned FSMap version 23 2024-01-11T17:58:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.773+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f5dac19f960 con 0x556dc859f400 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5d957fa700 1 -- 172.21.15.7:0/1212793461 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556dc8591010 con 0x556dc859f400 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5d957fa700 1 -- 172.21.15.7:0/1212793461 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d9c015020 con 0x556dc859f400 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5dc7933b80 10 client.25564 fetch_fsmap finished waiting for FSMap version 23 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5dc7933b80 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5dc7933b80 20 client.25564 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc85b3dc0 0x556dc85b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5dc7933b80 4 client.25564 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dc84eabc0 con 0x556dc85b3dc0 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.774+0000 7f5db199b700 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc85b3dc0 0x556dc85b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.775+0000 7f5db199b700 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc85b3dc0 0x556dc85b61e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f11dd6ddf7b1662 server_cookie=c7e96251b6c97d9b in_seq=0 out_seq=0 2024-01-11T17:58:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.775+0000 7f5d957fa700 10 client.25564 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.781 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:07 smithi038 ceph-mon[43582]: pgmap v309: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-01-11T17:58:07.781 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1224020196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:07.781 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:07 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1325995101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T17:58:07.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5d957fa700 1 -- 172.21.15.7:0/1212793461 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556dc84eabc0 con 0x556dc85b3dc0 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5d957fa700 10 client.25564 handle_command_reply: tid=0 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 client.25564 shutdown 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 msgr2=0x556dc85a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc85a8400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00fa70 tx=0x7f5d9c00b790 comp rx=0 tx=0).stop 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 shutdown_connections 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc85b3dc0 0x556dc85b61e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859e570 0x556dc85a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5d957fa700 0 client.25564 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/1212793461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859f400 0x556dc85a8400 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 >> 172.21.15.7:0/1212793461 conn(0x556dc85137a0 msgr2=0x556dc859ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.776+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 shutdown_connections 2024-01-11T17:58:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.777+0000 7f5dc7933b80 1 -- 172.21.15.7:0/1212793461 wait complete. 2024-01-11T17:58:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.777+0000 7f5dc7933b80 20 client.25564 trim_cache size 0 max 16384 2024-01-11T17:58:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.796+0000 7f5dc7933b80 1 Processor -- start 2024-01-11T17:58:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.796+0000 7f5dc7933b80 1 -- start start 2024-01-11T17:58:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc859df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 0x556dc85a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc83686a0 con 0x556dc859fd80 2024-01-11T17:58:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc8599ce0 con 0x556dc859f400 2024-01-11T17:58:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc85b05c0 con 0x556dc859e570 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5db119a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 0x556dc85a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5db119a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 0x556dc85a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49454/0 (socket says 172.21.15.7:49454) 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5db119a700 1 -- 172.21.15.7:0/3517397088 learned_addr learned my addr 172.21.15.7:0/3517397088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5db219c700 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.797+0000 7f5db199b700 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc859df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5db119a700 1 -- 172.21.15.7:0/3517397088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 msgr2=0x556dc859df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5db119a700 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc859df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5db119a700 1 -- 172.21.15.7:0/3517397088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 msgr2=0x556dc85af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5db119a700 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5db119a700 1 -- 172.21.15.7:0/3517397088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dc8312b60 con 0x556dc859f400 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5db119a700 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 0x556dc85a5e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00b3e0 tx=0x7f5d9c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ab346e0752d45f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5d977fe700 1 -- 172.21.15.7:0/3517397088 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0058e0 con 0x556dc859f400 2024-01-11T17:58:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5d977fe700 1 -- 172.21.15.7:0/3517397088 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d9c01d070 con 0x556dc859f400 2024-01-11T17:58:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.798+0000 7f5d977fe700 1 -- 172.21.15.7:0/3517397088 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c003970 con 0x556dc859f400 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3517397088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 msgr2=0x556dc85a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 0x556dc85a5e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00b3e0 tx=0x7f5d9c005dc0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3517397088 shutdown_connections 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc859df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859fd80 0x556dc85af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3517397088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859f400 0x556dc85a5e50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3517397088 >> 172.21.15.7:0/3517397088 conn(0x556dc85137a0 msgr2=0x556dc8598b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3517397088 shutdown_connections 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.799+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3517397088 wait complete. 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 -- start start 2024-01-11T17:58:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 Processor -- start 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc85ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 0x556dc85acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc85add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc85ae300 con 0x556dc859f400 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc85ae470 con 0x556dc859fd80 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc85ae630 con 0x556dc859e570 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5dc7933b80 10 client.0 _ll_register_callbacks cb 0x556dc8492720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5db119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 0x556dc85acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.800+0000 7f5db219c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc85add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db219c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc85add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49466/0 (socket says 172.21.15.7:49466) 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 0x556dc85acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46980/0 (socket says 172.21.15.7:46980) 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db219c700 1 -- 172.21.15.7:0/3504761635 learned_addr learned my addr 172.21.15.7:0/3504761635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db199b700 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc85ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 -- 172.21.15.7:0/3504761635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 msgr2=0x556dc85ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc85ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 -- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 msgr2=0x556dc85add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc85add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 -- 172.21.15.7:0/3504761635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac19f960 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5db119a700 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 0x556dc85acf10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009c40 tx=0x7f5d9c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6aa983d793a326d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.801+0000 7f5d957fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5d957fa700 1 -- 172.21.15.7:0/3504761635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0052c0 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5da809b3e0 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5d957fa700 1 -- 172.21.15.7:0/3504761635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5d9c000e40 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5d957fa700 1 -- 172.21.15.7:0/3504761635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9c005420 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5dc7933b80 10 client.15573 fetch_fsmap learned FSMap version 23 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556dc84b3230 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5d957fa700 1 -- 172.21.15.7:0/3504761635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da809b3e0 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5d957fa700 1 -- 172.21.15.7:0/3504761635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d9c03a070 con 0x556dc859f400 2024-01-11T17:58:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5dc7933b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 23 2024-01-11T17:58:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5dc7933b80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.802+0000 7f5dc7933b80 20 client.15573 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.803+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc860a480 0x556dc860c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.803+0000 7f5dc7933b80 4 client.15573 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-01-11T17:58:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.803+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556dc84ea8e0 con 0x556dc860a480 2024-01-11T17:58:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.803+0000 7f5db199b700 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc860a480 0x556dc860c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5db199b700 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc860a480 0x556dc860c8a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56fc83595c362432 server_cookie=15c9e0b31a56e4be in_seq=0 out_seq=0 2024-01-11T17:58:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5d957fa700 10 client.15573 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5d957fa700 1 -- 172.21.15.7:0/3504761635 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556dc84ea8e0 con 0x556dc860a480 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5d957fa700 10 client.15573 handle_command_reply: tid=0 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 client.15573 shutdown 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 msgr2=0x556dc85acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 0x556dc85acf10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009c40 tx=0x7f5d9c005be0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 shutdown_connections 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556dc860a480 0x556dc860c8a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556dc859e570 0x556dc85ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556dc859fd80 0x556dc85add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5d957fa700 0 client.15573 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 --2- 172.21.15.7:0/3504761635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556dc859f400 0x556dc85acf10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 >> 172.21.15.7:0/3504761635 conn(0x556dc85137a0 msgr2=0x556dc85bab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 shutdown_connections 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.804+0000 7f5dc7933b80 1 -- 172.21.15.7:0/3504761635 wait complete. 2024-01-11T17:58:07.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.805+0000 7f5dc7933b80 20 client.15573 trim_cache size 0 max 16384 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: "7eb683f7-d2e5-4da3-a588-694dc4079bbe": { 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7eb683f7-d2e5-4da3-a588-694dc4079bbe", 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:58:07.811 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:58:07.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.805+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5d980a1ee0 msgr2=0x7f5d980a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.805+0000 7f5db3bff700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5d980a1ee0 0x7f5d980a43a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009ec0 tx=0x7f5da400b040 comp rx=0 tx=0).stop 2024-01-11T17:58:07.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.805+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 msgr2=0x7f5dac1974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:07.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.805+0000 7f5db3bff700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 0x7f5dac1974b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5da800ebf0 tx=0x7f5da800ef00 comp rx=0 tx=0).stop 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 shutdown_connections 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5d980a1ee0 0x7f5d980a43a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dac103190 0x7f5dac19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dac101600 0x7f5dac1974b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 --2- 172.21.15.7:0/3593247221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dac1027f0 0x7f5dac1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 >> 172.21.15.7:0/3593247221 conn(0x7f5dac0fcb50 msgr2=0x7f5dac105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 shutdown_connections 2024-01-11T17:58:07.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:07.806+0000 7f5db3bff700 1 -- 172.21.15.7:0/3593247221 wait complete. 2024-01-11T17:58:07.827 INFO:tasks.cephfs.filesystem:scrub status for tag:7eb683f7-d2e5-4da3-a588-694dc4079bbe - {'path': '/', 'tag': '7eb683f7-d2e5-4da3-a588-694dc4079bbe', 'options': 'recursive,force'} 2024-01-11T17:58:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:08 smithi016 ceph-mon[43195]: scrub queued for path: / 2024-01-11T17:58:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:08 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:08 smithi016 ceph-mon[43195]: scrub summary: active paths [/] 2024-01-11T17:58:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:08 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:08.780 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:08 smithi038 ceph-mon[43582]: scrub queued for path: / 2024-01-11T17:58:08.780 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:08 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:08.780 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:08 smithi038 ceph-mon[43582]: scrub summary: active paths [/] 2024-01-11T17:58:08.780 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:08 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:08 smithi007 ceph-mon[33361]: scrub queued for path: / 2024-01-11T17:58:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:08 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:08 smithi007 ceph-mon[33361]: scrub summary: active paths [/] 2024-01-11T17:58:08.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:08 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:09.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:09 smithi016 ceph-mon[43195]: pgmap v310: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-01-11T17:58:09.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:09 smithi038 ceph-mon[43582]: pgmap v310: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-01-11T17:58:09.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:09 smithi007 ceph-mon[33361]: pgmap v310: 129 pgs: 129 active+clean; 649 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-01-11T17:58:11.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-01-11T17:58:11.240 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-01-11T17:58:11.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-01-11T17:58:11.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-01-11T17:58:11.444 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 .. 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 data 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:11.445 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 .. 2024-01-11T17:58:11.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-01-11T17:58:11.574 INFO:teuthology.orchestra.run.smithi007.stdout:[Service] 2024-01-11T17:58:11.574 INFO:teuthology.orchestra.run.smithi007.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-01-11T17:58:11.574 INFO:teuthology.orchestra.run.smithi007.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-01-11T17:58:11.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-01-11T17:58:11.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:11 smithi007 ceph-mon[33361]: pgmap v311: 129 pgs: 129 active+clean; 659 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-01-11T17:58:11.768 INFO:teuthology.orchestra.run.smithi007.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-01-11T17:58:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:11 smithi016 ceph-mon[43195]: pgmap v311: 129 pgs: 129 active+clean; 659 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-01-11T17:58:12.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:11 smithi038 ceph-mon[43582]: pgmap v311: 129 pgs: 129 active+clean; 659 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-01-11T17:58:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:13 smithi016 ceph-mon[43195]: pgmap v312: 129 pgs: 129 active+clean; 659 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-11T17:58:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:13 smithi038 ceph-mon[43582]: pgmap v312: 129 pgs: 129 active+clean; 659 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-11T17:58:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:13 smithi007 ceph-mon[33361]: pgmap v312: 129 pgs: 129 active+clean; 659 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-11T17:58:15.608 INFO:teuthology.orchestra.run.smithi007.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-01-11T17:58:15.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:15 smithi007 ceph-mon[33361]: pgmap v313: 129 pgs: 129 active+clean; 9.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 764 KiB/s wr, 8 op/s 2024-01-11T17:58:15.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 .. 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 data 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-01-11T17:58:15.729 INFO:teuthology.orchestra.run.smithi007.stdout:total 30 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 . 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 .. 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 base 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jan 11 17:58 global 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 log 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_commit_ts 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_dynshmem 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jan 11 17:58 pg_hba.conf 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jan 11 17:58 pg_ident.conf 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 pg_logical 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 pg_multixact 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 pg_notify 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_replslot 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_serial 2024-01-11T17:58:15.730 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_snapshots 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_stat 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_stat_tmp 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 pg_subtrans 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_tblspc 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_twophase 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 pg_wal 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 pg_xact 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jan 11 17:58 postgresql.auto.conf 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jan 11 17:58 postgresql.conf 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base: 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 . 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 1 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 12346 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 12347 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/1: 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:total 7209 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 . 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 .. 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 112 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 113 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jan 11 17:58 12182 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12182_fsm 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12182_vm 2024-01-11T17:58:15.732 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12184 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12186 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12187_fsm 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187_vm 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12189 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12191 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12192_fsm 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192_vm 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12194 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12196 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12197_fsm 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197_vm 2024-01-11T17:58:15.733 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12199 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12201 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12202_fsm 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202_vm 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12204 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12206 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12207_fsm 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207_vm 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12209 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12211 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12212 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12214 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12216 2024-01-11T17:58:15.734 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jan 11 17:58 1247 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1247_fsm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1247_vm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jan 11 17:58 1249 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1249_fsm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1249_vm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jan 11 17:58 1255 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1255_fsm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1255_vm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jan 11 17:58 1259 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1259_fsm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1259_vm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417_vm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418_vm 2024-01-11T17:58:15.735 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 174 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 175 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2187 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224_vm 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328_vm 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336_vm 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2337 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2600 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2600_fsm 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2600_vm 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601 2024-01-11T17:58:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2601_fsm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601_vm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2602 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2602_fsm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2602_vm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2603 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2603_fsm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2603_vm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604_vm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2605 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2605_fsm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2605_vm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2606_fsm 2024-01-11T17:58:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606_vm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607_fsm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2607_vm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jan 11 17:58 2608 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2608_fsm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2608_vm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jan 11 17:58 2609 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2609_fsm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2609_vm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610_fsm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2610_vm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611_vm 2024-01-11T17:58:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2612_fsm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612_vm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613_vm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2615_fsm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615_vm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616_fsm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2616_vm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jan 11 17:58 2617 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2617_fsm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2617_vm 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jan 11 17:58 2618 2024-01-11T17:58:15.739 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2618_fsm 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2618_vm 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jan 11 17:58 2619 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2619_fsm 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2619_vm 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620_vm 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2650 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2651 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2652 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2653 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2654 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2655 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2656 2024-01-11T17:58:15.740 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2657 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jan 11 17:58 2658 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2659 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2660 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2661 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2662 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2663 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2664 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2665 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2666 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2667 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2668 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2669 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2670 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jan 11 17:58 2673 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jan 11 17:58 2674 2024-01-11T17:58:15.741 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jan 11 17:58 2675 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2678 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2679 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2680 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2681 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2682 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2683 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2684 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2685 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2686 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2687 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2688 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2689 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2690 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jan 11 17:58 2691 2024-01-11T17:58:15.742 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2692 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2693 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2696 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2699 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2701 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2702 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2703 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2704 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2753 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2753_fsm 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2753_vm 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2754 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2755 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2756 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2757 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830 2024-01-11T17:58:15.743 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830_vm 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2831 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832_vm 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2833 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834_vm 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2835 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836_vm 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2837 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jan 11 17:58 2838 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2838_fsm 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2838_vm 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2839 2024-01-11T17:58:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840_fsm 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2840_vm 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2841 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995_vm 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2996 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3079_fsm 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079_vm 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3080 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3081 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3085 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118_vm 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3119 2024-01-11T17:58:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3164 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256_vm 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3257 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3258 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350_vm 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3351 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3379 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3380 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381_vm 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3394 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3394_fsm 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3394_vm 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3395 2024-01-11T17:58:15.746 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439_vm 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3440 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 3455 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3456 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3456_fsm 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3456_vm 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466_vm 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3467 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3468 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501_vm 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3502 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3503 2024-01-11T17:58:15.747 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3534 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3541_fsm 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541_vm 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3542 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3574 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3575 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576_vm 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596_vm 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3597 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598_vm 2024-01-11T17:58:15.748 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3599 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3600_fsm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600_vm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3601_fsm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601_vm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3602_fsm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602_vm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3603 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3603_fsm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3603_vm 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3604 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3605 2024-01-11T17:58:15.749 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3606 2024-01-11T17:58:15.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:15 smithi016 ceph-mon[43195]: pgmap v313: 129 pgs: 129 active+clean; 9.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 764 KiB/s wr, 8 op/s 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3607 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3608 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 3609 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3712 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3764_fsm 2024-01-11T17:58:15.751 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764_vm 2024-01-11T17:58:15.752 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3766 2024-01-11T17:58:15.752 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3767 2024-01-11T17:58:15.752 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3997 2024-01-11T17:58:15.752 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 5002 2024-01-11T17:58:15.752 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 548 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 549 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102_vm 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104_vm 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106_vm 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6110 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6111 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6112 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6113 2024-01-11T17:58:15.756 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6117 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826_vm 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 827 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 828 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/12346: 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:total 7209 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 . 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 .. 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 112 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 113 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jan 11 17:58 12182 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12182_fsm 2024-01-11T17:58:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12182_vm 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12184 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12186 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12187_fsm 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187_vm 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12189 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12191 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12192_fsm 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192_vm 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12194 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12196 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12197_fsm 2024-01-11T17:58:15.758 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197_vm 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12199 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12201 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12202_fsm 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202_vm 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12204 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12206 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12207_fsm 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207_vm 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12209 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12211 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12212 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12214 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12216 2024-01-11T17:58:15.759 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jan 11 17:58 1247 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1247_fsm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1247_vm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jan 11 17:58 1249 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1249_fsm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1249_vm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jan 11 17:58 1255 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1255_fsm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1255_vm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jan 11 17:58 1259 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1259_fsm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1259_vm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417_vm 2024-01-11T17:58:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418_vm 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 174 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 175 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2187 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224_vm 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328_vm 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336_vm 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2337 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2600 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2600_fsm 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2600_vm 2024-01-11T17:58:15.761 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2601_fsm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601_vm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2602 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2602_fsm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2602_vm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2603 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2603_fsm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2603_vm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604_vm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2605 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2605_fsm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2605_vm 2024-01-11T17:58:15.762 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2606_fsm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606_vm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607_fsm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2607_vm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jan 11 17:58 2608 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2608_fsm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2608_vm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jan 11 17:58 2609 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2609_fsm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2609_vm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610_fsm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2610_vm 2024-01-11T17:58:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611_vm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2612_fsm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612_vm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613_vm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2615_fsm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615_vm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616_fsm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2616_vm 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jan 11 17:58 2617 2024-01-11T17:58:15.764 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2617_fsm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2617_vm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jan 11 17:58 2618 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2618_fsm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2618_vm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jan 11 17:58 2619 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2619_fsm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2619_vm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620_vm 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2650 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2651 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2652 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2653 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2654 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2655 2024-01-11T17:58:15.765 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2656 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2657 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jan 11 17:58 2658 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2659 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2660 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2661 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2662 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2663 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2664 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2665 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2666 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2667 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2668 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2669 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2670 2024-01-11T17:58:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jan 11 17:58 2673 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jan 11 17:58 2674 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jan 11 17:58 2675 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2678 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2679 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2680 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2681 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2682 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2683 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2684 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2685 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2686 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2687 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2688 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2689 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2690 2024-01-11T17:58:15.767 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jan 11 17:58 2691 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2692 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2693 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2696 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2699 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2701 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2702 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2703 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2704 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2753 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2753_fsm 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2753_vm 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2754 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2755 2024-01-11T17:58:15.768 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2756 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2757 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830_vm 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2831 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832_vm 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2833 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834_vm 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2835 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836_vm 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2837 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jan 11 17:58 2838 2024-01-11T17:58:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2838_fsm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2838_vm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2839 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840_fsm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2840_vm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2841 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995_vm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2996 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3079_fsm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079_vm 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3080 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3081 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3085 2024-01-11T17:58:15.770 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118_vm 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3119 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3164 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256_vm 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3257 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3258 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350_vm 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3351 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3379 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3380 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381_vm 2024-01-11T17:58:15.771 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3394 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3394_fsm 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3394_vm 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3395 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439_vm 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3440 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 3455 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3456 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3456_fsm 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3456_vm 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466_vm 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3467 2024-01-11T17:58:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3468 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501_vm 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3502 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3503 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3534 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3541_fsm 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541_vm 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3542 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3574 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3575 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576_vm 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596_vm 2024-01-11T17:58:15.773 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3597 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598_vm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3599 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3600_fsm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600_vm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3601_fsm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601_vm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3602_fsm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602_vm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3603 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3603_fsm 2024-01-11T17:58:15.774 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3603_vm 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3604 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3605 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3606 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3607 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3608 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 3609 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3712 2024-01-11T17:58:15.775 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764 2024-01-11T17:58:15.777 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3764_fsm 2024-01-11T17:58:15.777 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764_vm 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3766 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3767 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3997 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 5002 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 548 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 549 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102_vm 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104_vm 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106_vm 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6110 2024-01-11T17:58:15.778 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6111 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6112 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6113 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6117 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826_vm 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 827 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 828 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/12347: 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:total 7209 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 . 2024-01-11T17:58:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 .. 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 112 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 113 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jan 11 17:58 12182 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12182_fsm 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12182_vm 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12184 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12186 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12187_fsm 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187_vm 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12189 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12191 2024-01-11T17:58:15.780 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12192_fsm 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192_vm 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12194 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12196 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12197_fsm 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197_vm 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12199 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12201 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12202_fsm 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202_vm 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12204 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12206 2024-01-11T17:58:15.781 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12207_fsm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207_vm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12209 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12211 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12212 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12214 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12216 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jan 11 17:58 1247 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1247_fsm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1247_vm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jan 11 17:58 1249 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1249_fsm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1249_vm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jan 11 17:58 1255 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1255_fsm 2024-01-11T17:58:15.782 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1255_vm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jan 11 17:58 1259 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1259_fsm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1259_vm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417_vm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418_vm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 174 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 175 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2187 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224_vm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328_vm 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336 2024-01-11T17:58:15.783 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336_vm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2337 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2600 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2600_fsm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2600_vm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2601_fsm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601_vm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2602 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2602_fsm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2602_vm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2603 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2603_fsm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2603_vm 2024-01-11T17:58:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604_vm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2605 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2605_fsm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2605_vm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2606_fsm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606_vm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607_fsm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2607_vm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jan 11 17:58 2608 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2608_fsm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2608_vm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jan 11 17:58 2609 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2609_fsm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2609_vm 2024-01-11T17:58:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610_fsm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2610_vm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611_vm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2612_fsm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612_vm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613_vm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2615_fsm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615_vm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616_fsm 2024-01-11T17:58:15.786 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2616_vm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jan 11 17:58 2617 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2617_fsm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2617_vm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jan 11 17:58 2618 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2618_fsm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2618_vm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jan 11 17:58 2619 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2619_fsm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2619_vm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620_vm 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2650 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2651 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2652 2024-01-11T17:58:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2653 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2654 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2655 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2656 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2657 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jan 11 17:58 2658 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2659 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2660 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2661 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2662 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2663 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2664 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2665 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2666 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2667 2024-01-11T17:58:15.788 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2668 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2669 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2670 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jan 11 17:58 2673 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jan 11 17:58 2674 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jan 11 17:58 2675 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2678 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2679 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2680 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2681 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2682 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2683 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2684 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2685 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2686 2024-01-11T17:58:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2687 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2688 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2689 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2690 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jan 11 17:58 2691 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2692 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2693 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2696 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2699 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2701 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2702 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2703 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2704 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2753 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2753_fsm 2024-01-11T17:58:15.790 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2753_vm 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2754 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2755 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2756 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2757 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830_vm 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2831 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832_vm 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2833 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834_vm 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2835 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836 2024-01-11T17:58:15.791 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836_vm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2837 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jan 11 17:58 2838 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2838_fsm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2838_vm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2839 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840_fsm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2840_vm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2841 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995_vm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2996 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3079_fsm 2024-01-11T17:58:15.792 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079_vm 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3080 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3081 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3085 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118_vm 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3119 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3164 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256_vm 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3257 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3258 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350_vm 2024-01-11T17:58:15.793 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3351 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3379 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3380 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381_vm 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3394 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3394_fsm 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3394_vm 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3395 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439_vm 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3440 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 3455 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3456 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3456_fsm 2024-01-11T17:58:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3456_vm 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466_vm 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3467 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3468 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501_vm 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3502 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3503 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3534 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3541_fsm 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541_vm 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3542 2024-01-11T17:58:15.795 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3574 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3575 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576_vm 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596_vm 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3597 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598_vm 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3599 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3600_fsm 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600_vm 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3601_fsm 2024-01-11T17:58:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601_vm 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3602_fsm 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602_vm 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3603 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3603_fsm 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3603_vm 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3604 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3605 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3606 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3607 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3608 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 3609 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3712 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764 2024-01-11T17:58:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3764_fsm 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764_vm 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3766 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3767 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3997 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 5002 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 548 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 549 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102_vm 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104_vm 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106_vm 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6110 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6111 2024-01-11T17:58:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6112 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6113 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6117 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826_vm 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 827 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 828 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/global: 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:total 553 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jan 11 17:58 . 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1136 2024-01-11T17:58:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1136_fsm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1136_vm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 1137 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1213 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1213_fsm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1213_vm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1214 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1214_fsm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1214_vm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 1232 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 1233 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1260 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1260_fsm 2024-01-11T17:58:15.800 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1260_vm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1261 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1261_fsm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1261_vm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1262 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1262_fsm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1262_vm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2396 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2396_fsm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2396_vm 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2397 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2671 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2672 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2676 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2677 2024-01-11T17:58:15.801 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2694 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2695 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2697 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2698 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2846 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2846_vm 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2847 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2964 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2964_vm 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2965 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2966 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2966_vm 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2967 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3592 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3592_vm 2024-01-11T17:58:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3593 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 4060 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 4060_vm 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 4061 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6000 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6000_vm 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6001 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6002 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6100 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6100_vm 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6114 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6115 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 pg_control 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/log: 2024-01-11T17:58:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.804 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 . 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 mappings 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jan 11 17:58 replorigin_checkpoint 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 snapshots 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 .. 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-01-11T17:58:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 .. 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 . 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 members 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 offsets 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:15.806 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 .. 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 .. 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-01-11T17:58:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-01-11T17:58:15.808 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-01-11T17:58:15.809 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:total 16384 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 . 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 17:58 000000010000000000000001 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 archive_status 2024-01-11T17:58:15.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 .. 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jan 11 17:58 .. 2024-01-11T17:58:15.811 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T17:58:15.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-01-11T17:58:15.812 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:15 smithi038 ceph-mon[43582]: pgmap v313: 129 pgs: 129 active+clean; 9.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 764 KiB/s wr, 8 op/s 2024-01-11T17:58:16.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-01-11T17:58:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-01-11T17:58:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-01-11T17:58:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-01-11T17:58:16.203 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-01-11T17:58:16.231 INFO:teuthology.orchestra.run.smithi007.stderr:creating tables... 2024-01-11T17:58:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.13 s, remaining 4.05 s) 2024-01-11T17:58:16.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:16 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:58:16.690 INFO:teuthology.orchestra.run.smithi007.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.46 s, remaining 6.86 s) 2024-01-11T17:58:16.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:16 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:58:16.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:16 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:58:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.79 s, remaining 7.68 s) 2024-01-11T17:58:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:400000 of 3200000 tuples (12%) done (elapsed 1.00 s, remaining 7.01 s) 2024-01-11T17:58:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.36 s, remaining 7.33 s) 2024-01-11T17:58:17.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:17 smithi016 ceph-mon[43195]: pgmap v314: 129 pgs: 129 active+clean; 32 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.1 MiB/s wr, 55 op/s 2024-01-11T17:58:17.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:17 smithi038 ceph-mon[43582]: pgmap v314: 129 pgs: 129 active+clean; 32 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.1 MiB/s wr, 55 op/s 2024-01-11T17:58:17.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:17 smithi007 ceph-mon[33361]: pgmap v314: 129 pgs: 129 active+clean; 32 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 3.1 MiB/s wr, 55 op/s 2024-01-11T17:58:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.79 s, remaining 7.75 s) 2024-01-11T17:58:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.91 s, remaining 6.82 s) 2024-01-11T17:58:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.33 s, remaining 7.00 s) 2024-01-11T17:58:18.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: scrub summary: idle 2024-01-11T17:58:18.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:18 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: scrub summary: idle 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:18.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:18 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: scrub summary: idle 2024-01-11T17:58:19.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:18 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.80 s, remaining 7.17 s) 2024-01-11T17:58:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.91 s, remaining 6.41 s) 2024-01-11T17:58:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:1100000 of 3200000 tuples (34%) done (elapsed 3.30 s, remaining 6.30 s) 2024-01-11T17:58:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.42 s, remaining 5.70 s) 2024-01-11T17:58:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:19 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:19 smithi007 ceph-mon[33361]: pgmap v315: 129 pgs: 129 active+clean; 37 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.1 MiB/s wr, 57 op/s 2024-01-11T17:58:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:58:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:58:19.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:19.974 INFO:teuthology.orchestra.run.smithi007.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.74 s, remaining 5.47 s) 2024-01-11T17:58:19.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:19 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:19.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:19 smithi016 ceph-mon[43195]: pgmap v315: 129 pgs: 129 active+clean; 37 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.1 MiB/s wr, 57 op/s 2024-01-11T17:58:19.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:58:19.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:58:19.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:20.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:19 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:20.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:19 smithi038 ceph-mon[43582]: pgmap v315: 129 pgs: 129 active+clean; 37 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.1 MiB/s wr, 57 op/s 2024-01-11T17:58:20.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:58:20.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:58:20.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:58:20.351 INFO:teuthology.orchestra.run.smithi007.stderr:1400000 of 3200000 tuples (43%) done (elapsed 4.12 s, remaining 5.29 s) 2024-01-11T17:58:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:1500000 of 3200000 tuples (46%) done (elapsed 4.24 s, remaining 4.81 s) 2024-01-11T17:58:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 -- 172.21.15.7:0/4279268436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 msgr2=0x7f23e40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 --2- 172.21.15.7:0/4279268436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e40744f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009990 tx=0x7f23d8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 -- 172.21.15.7:0/4279268436 shutdown_connections 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 --2- 172.21.15.7:0/4279268436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e410db80 0x7f23e410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 --2- 172.21.15.7:0/4279268436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e4075650 0x7f23e4073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 --2- 172.21.15.7:0/4279268436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e40744f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.505+0000 7f23e89ff700 1 -- 172.21.15.7:0/4279268436 >> 172.21.15.7:0/4279268436 conn(0x7f23e40fcb00 msgr2=0x7f23e40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.506+0000 7f23e89ff700 1 -- 172.21.15.7:0/4279268436 shutdown_connections 2024-01-11T17:58:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.506+0000 7f23e89ff700 1 -- 172.21.15.7:0/4279268436 wait complete. 2024-01-11T17:58:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 Processor -- start 2024-01-11T17:58:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 -- start start 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e4075650 0x7f23e419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e410db80 0x7f23e41a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41a5640 con 0x7f23e4074070 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41a5780 con 0x7f23e410db80 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e89ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41a5910 con 0x7f23e4075650 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e4075650 0x7f23e419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60292/0 (socket says 172.21.15.7:60292) 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e259c700 1 -- 172.21.15.7:0/543594200 learned_addr learned my addr 172.21.15.7:0/543594200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.507+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e410db80 0x7f23e41a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e259c700 1 -- 172.21.15.7:0/543594200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e4075650 msgr2=0x7f23e419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e259c700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e4075650 0x7f23e419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e259c700 1 -- 172.21.15.7:0/543594200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e410db80 msgr2=0x7f23e41a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e259c700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e410db80 0x7f23e41a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e259c700 1 -- 172.21.15.7:0/543594200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80095f0 con 0x7f23e4074070 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e410db80 0x7f23e41a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e259c700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e419dac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f23d400e9a0 tx=0x7f23d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23d37fe700 1 -- 172.21.15.7:0/543594200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4009790 con 0x7f23e4074070 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23d37fe700 1 -- 172.21.15.7:0/543594200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d4010430 con 0x7f23e4074070 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e41a5bf0 con 0x7f23e4074070 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.508+0000 7f23d37fe700 1 -- 172.21.15.7:0/543594200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4018680 con 0x7f23e4074070 2024-01-11T17:58:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.509+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e41a6100 con 0x7f23e4074070 2024-01-11T17:58:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.512+0000 7f23d37fe700 1 -- 172.21.15.7:0/543594200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f23d40105a0 con 0x7f23e4074070 2024-01-11T17:58:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 Processor -- start 2024-01-11T17:58:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 -- start start 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23d37fe700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23cc0a1ed0 0x7f23cc0a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e8a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e8b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e91e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23d37fe700 1 -- 172.21.15.7:0/543594200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23d4014070 con 0x7f23e4074070 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3d81030 con 0x561ee3e8b5d0 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3de4ac0 con 0x561ee3e8ac10 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.514+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3de23e0 con 0x561ee3e89f20 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e8b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e8b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48584/0 (socket says 172.21.15.7:48584) 2024-01-11T17:58:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3949720368 learned_addr learned my addr 172.21.15.7:0/3949720368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3949720368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 msgr2=0x561ee3e8a340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:58:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e8a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23cc0a1ed0 0x7f23cc0a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3949720368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 msgr2=0x561ee3e91e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e91e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.515+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3949720368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ee3c52350 con 0x561ee3e8ac10 2024-01-11T17:58:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.516+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23cc0a1ed0 0x7f23cc0a4390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009fd0 tx=0x7f23dc009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.517+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e8b090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009990 tx=0x7f23d8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3fff8b4a444d65 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.517+0000 7f23d0ff9700 1 -- 172.21.15.7:0/3949720368 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d801d070 con 0x561ee3e8ac10 2024-01-11T17:58:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.517+0000 7f23d0ff9700 1 -- 172.21.15.7:0/3949720368 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d801e040 con 0x561ee3e8ac10 2024-01-11T17:58:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.517+0000 7f23d0ff9700 1 -- 172.21.15.7:0/3949720368 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004a60 con 0x561ee3e8ac10 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3949720368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 msgr2=0x561ee3e8b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e8b090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009990 tx=0x7f23d8005e80 comp rx=0 tx=0).stop 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3949720368 shutdown_connections 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e8a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e91e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3949720368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e8b090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3949720368 >> 172.21.15.7:0/3949720368 conn(0x561ee3dff740 msgr2=0x561ee3e84950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3949720368 shutdown_connections 2024-01-11T17:58:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.518+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3949720368 wait complete. 2024-01-11T17:58:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 -- start start 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 Processor -- start 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e93950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e93e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e9aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3dddcd0 con 0x561ee3e89f20 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3d4c340 con 0x561ee3e8b5d0 2024-01-11T17:58:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3d4bfe0 con 0x561ee3e8ac10 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.519+0000 7f23fc7b7b80 10 client.0 _ll_register_callbacks cb 0x561ee3d7e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e93e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e93e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:49104/0 (socket says 172.21.15.7:49104) 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e9aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e1d9b700 1 -- 172.21.15.7:0/329080505 learned_addr learned my addr 172.21.15.7:0/329080505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 -- 172.21.15.7:0/329080505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 msgr2=0x561ee3e93e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e93e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 -- 172.21.15.7:0/329080505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 msgr2=0x561ee3e93950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e93950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 -- 172.21.15.7:0/329080505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e4110920 con 0x561ee3e8b5d0 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e9aed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00e770 tx=0x7f23dc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73e7057eb8d512f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.520+0000 7f23c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23c9ffb700 1 -- 172.21.15.7:0/329080505 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc005040 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ee3e7cd60 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23c9ffb700 1 -- 172.21.15.7:0/329080505 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23dc0051a0 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23c9ffb700 1 -- 172.21.15.7:0/329080505 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23dc005360 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23fc7b7b80 10 client.25562 fetch_fsmap learned FSMap version 24 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561ee3bfeb00 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.521+0000 7f23c9ffb700 1 -- 172.21.15.7:0/329080505 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ee3e7cd60 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23c9ffb700 1 -- 172.21.15.7:0/329080505 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f23dc019a30 con 0x561ee3e8b5d0 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23fc7b7b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 24 2024-01-11T17:58:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23fc7b7b80 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23fc7b7b80 20 client.25562 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3e9f8e0 0x561ee3ea1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23fc7b7b80 4 client.25562 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ee3dd6b60 con 0x561ee3e9f8e0 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.522+0000 7f23e259c700 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3e9f8e0 0x561ee3ea1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.523+0000 7f23e259c700 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3e9f8e0 0x561ee3ea1d00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53a086735167d91 server_cookie=5bdf3eb6cba12abc in_seq=0 out_seq=0 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.523+0000 7f23c9ffb700 10 client.25562 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.523+0000 7f23c9ffb700 1 -- 172.21.15.7:0/329080505 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ee3dd6b60 con 0x561ee3e9f8e0 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.523+0000 7f23c9ffb700 10 client.25562 handle_command_reply: tid=0 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 client.25562 shutdown 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 msgr2=0x561ee3e9aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e9aed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00e770 tx=0x7f23dc004ad0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 shutdown_connections 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3e9f8e0 0x561ee3ea1d00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e9aed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e93950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23c9ffb700 0 client.25562 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/329080505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e93e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 >> 172.21.15.7:0/329080505 conn(0x561ee3dff740 msgr2=0x561ee3e83f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 shutdown_connections 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/329080505 wait complete. 2024-01-11T17:58:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.524+0000 7f23fc7b7b80 20 client.25562 trim_cache size 0 max 16384 2024-01-11T17:58:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.540+0000 7f23fc7b7b80 1 Processor -- start 2024-01-11T17:58:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 -- start start 2024-01-11T17:58:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e9b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e9ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e809b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3c54640 con 0x561ee3e8b5d0 2024-01-11T17:58:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3e81590 con 0x561ee3e8ac10 2024-01-11T17:58:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3e862e0 con 0x561ee3e89f20 2024-01-11T17:58:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.541+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e9ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e9ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48604/0 (socket says 172.21.15.7:48604) 2024-01-11T17:58:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3215681960 learned_addr learned my addr 172.21.15.7:0/3215681960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e259c700 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e9b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3215681960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 msgr2=0x561ee3e9b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e9b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3215681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 msgr2=0x561ee3e809b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e809b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.542+0000 7f23e1d9b700 1 -- 172.21.15.7:0/3215681960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ee3d9f1d0 con 0x561ee3e8ac10 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.543+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e9ae90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005d10 tx=0x7f23d8009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ea38405c7dfcb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.543+0000 7f23cbfff700 1 -- 172.21.15.7:0/3215681960 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d801d040 con 0x561ee3e8ac10 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.543+0000 7f23cbfff700 1 -- 172.21.15.7:0/3215681960 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d800beb0 con 0x561ee3e8ac10 2024-01-11T17:58:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.543+0000 7f23cbfff700 1 -- 172.21.15.7:0/3215681960 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8022660 con 0x561ee3e8ac10 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3215681960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 msgr2=0x561ee3e9ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e9ae90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005d10 tx=0x7f23d8009320 comp rx=0 tx=0).stop 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3215681960 shutdown_connections 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e89f20 0x561ee3e9b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e8b5d0 0x561ee3e809b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/3215681960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8ac10 0x561ee3e9ae90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3215681960 >> 172.21.15.7:0/3215681960 conn(0x561ee3dff740 msgr2=0x561ee3e84920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3215681960 shutdown_connections 2024-01-11T17:58:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.544+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/3215681960 wait complete. 2024-01-11T17:58:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.545+0000 7f23fc7b7b80 1 -- start start 2024-01-11T17:58:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.545+0000 7f23fc7b7b80 1 Processor -- start 2024-01-11T17:58:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e98760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3e99cb0 con 0x561ee3e89f20 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3e99e20 con 0x561ee3e8b5d0 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee3e99fe0 con 0x561ee3e8ac10 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23fc7b7b80 10 client.0 _ll_register_callbacks cb 0x561ee3d7e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e98760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23e2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23e1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e98760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:49120/0 (socket says 172.21.15.7:49120) 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.546+0000 7f23e1d9b700 1 -- 172.21.15.7:0/372199065 learned_addr learned my addr 172.21.15.7:0/372199065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e2d9d700 1 -- 172.21.15.7:0/372199065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 msgr2=0x561ee3e98760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e98760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e2d9d700 1 -- 172.21.15.7:0/372199065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 msgr2=0x561ee3e98220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e259c700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e98220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e2d9d700 1 -- 172.21.15.7:0/372199065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ee3c52350 con 0x561ee3e8b5d0 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e1d9b700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e98760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.547+0000 7f23e259c700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.548+0000 7f23e2d9d700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e99770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0061e0 tx=0x7f23dc019480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cb4e25bfc2726e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.548+0000 7f23c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.548+0000 7f23c9ffb700 1 -- 172.21.15.7:0/372199065 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc03d670 con 0x561ee3e8b5d0 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.548+0000 7f23c9ffb700 1 -- 172.21.15.7:0/372199065 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23dc03d7d0 con 0x561ee3e8b5d0 2024-01-11T17:58:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.548+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23d801e040 con 0x561ee3e8b5d0 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23c9ffb700 1 -- 172.21.15.7:0/372199065 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23dc03d9b0 con 0x561ee3e8b5d0 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 10 client.25568 fetch_fsmap learned FSMap version 24 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561ee3c56c70 con 0x561ee3e8b5d0 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23c9ffb700 1 -- 172.21.15.7:0/372199065 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d801e040 con 0x561ee3e8b5d0 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23c9ffb700 1 -- 172.21.15.7:0/372199065 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f23dc017c10 con 0x561ee3e8b5d0 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 10 client.25568 fetch_fsmap finished waiting for FSMap version 24 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 20 client.25568 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3ef6430 0x561ee3ef8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 4 client.25568 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:58:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.549+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561ee3dd6880 con 0x561ee3ef6430 2024-01-11T17:58:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.550+0000 7f23e259c700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3ef6430 0x561ee3ef8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.550+0000 7f23e259c700 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3ef6430 0x561ee3ef8850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50600e0762f33db6 server_cookie=9a0abb1304d7360b in_seq=0 out_seq=0 2024-01-11T17:58:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.550+0000 7f23c9ffb700 10 client.25568 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23c9ffb700 1 -- 172.21.15.7:0/372199065 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561ee3dd6880 con 0x561ee3ef6430 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23c9ffb700 10 client.25568 handle_command_reply: tid=0 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 client.25568 shutdown 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 msgr2=0x561ee3e99770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e99770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0061e0 tx=0x7f23dc019480 comp rx=0 tx=0).stop 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 shutdown_connections 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x561ee3ef6430 0x561ee3ef8850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ee3e8b5d0 0x561ee3e99770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ee3e89f20 0x561ee3e98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 --2- 172.21.15.7:0/372199065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee3e8ac10 0x561ee3e98760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 >> 172.21.15.7:0/372199065 conn(0x561ee3dff740 msgr2=0x561ee3e84800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.905+0000 7f23c9ffb700 0 client.25568 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.906+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 shutdown_connections 2024-01-11T17:58:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.906+0000 7f23fc7b7b80 1 -- 172.21.15.7:0/372199065 wait complete. 2024-01-11T17:58:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.906+0000 7f23fc7b7b80 20 client.25568 trim_cache size 0 max 16384 2024-01-11T17:58:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:58:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:58:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:58:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23cc0a1ed0 msgr2=0x7f23cc0a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23cc0a1ed0 0x7f23cc0a4390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009fd0 tx=0x7f23dc009920 comp rx=0 tx=0).stop 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 msgr2=0x7f23e419dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e419dac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f23d400e9a0 tx=0x7f23d400ecb0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 shutdown_connections 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23cc0a1ed0 0x7f23cc0a4390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23e410db80 0x7f23e41a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23e4074070 0x7f23e419dac0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 --2- 172.21.15.7:0/543594200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23e4075650 0x7f23e419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 >> 172.21.15.7:0/543594200 conn(0x7f23e40fcb00 msgr2=0x7f23e4100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 shutdown_connections 2024-01-11T17:58:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:20.908+0000 7f23e89ff700 1 -- 172.21.15.7:0/543594200 wait complete. 2024-01-11T17:58:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.81 s, remaining 4.81 s) 2024-01-11T17:58:21.244 INFO:teuthology.orchestra.run.smithi007.stderr:1700000 of 3200000 tuples (53%) done (elapsed 5.01 s, remaining 4.42 s) 2024-01-11T17:58:21.580 INFO:teuthology.orchestra.run.smithi007.stderr:1800000 of 3200000 tuples (56%) done (elapsed 5.35 s, remaining 4.16 s) 2024-01-11T17:58:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.68 s, remaining 3.89 s) 2024-01-11T17:58:21.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:21 smithi016 ceph-mon[43195]: pgmap v316: 129 pgs: 129 active+clean; 87 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 12 MiB/s wr, 115 op/s 2024-01-11T17:58:22.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:21 smithi038 ceph-mon[43582]: pgmap v316: 129 pgs: 129 active+clean; 87 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 12 MiB/s wr, 115 op/s 2024-01-11T17:58:22.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:21 smithi007 ceph-mon[33361]: pgmap v316: 129 pgs: 129 active+clean; 87 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 12 MiB/s wr, 115 op/s 2024-01-11T17:58:22.104 INFO:teuthology.orchestra.run.smithi007.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.87 s, remaining 3.52 s) 2024-01-11T17:58:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2100000 of 3200000 tuples (65%) done (elapsed 6.15 s, remaining 3.22 s) 2024-01-11T17:58:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2200000 of 3200000 tuples (68%) done (elapsed 6.46 s, remaining 2.94 s) 2024-01-11T17:58:22.863 INFO:teuthology.orchestra.run.smithi007.stderr:2300000 of 3200000 tuples (71%) done (elapsed 6.63 s, remaining 2.59 s) 2024-01-11T17:58:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.92 s, remaining 2.31 s) 2024-01-11T17:58:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2500000 of 3200000 tuples (78%) done (elapsed 7.26 s, remaining 2.03 s) 2024-01-11T17:58:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2600000 of 3200000 tuples (81%) done (elapsed 7.49 s, remaining 1.73 s) 2024-01-11T17:58:23.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:23 smithi016 ceph-mon[43195]: pgmap v317: 129 pgs: 129 active+clean; 163 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 25 MiB/s wr, 197 op/s 2024-01-11T17:58:24.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:23 smithi038 ceph-mon[43582]: pgmap v317: 129 pgs: 129 active+clean; 163 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 25 MiB/s wr, 197 op/s 2024-01-11T17:58:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2700000 of 3200000 tuples (84%) done (elapsed 7.80 s, remaining 1.44 s) 2024-01-11T17:58:24.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:23 smithi007 ceph-mon[33361]: pgmap v317: 129 pgs: 129 active+clean; 163 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 25 MiB/s wr, 197 op/s 2024-01-11T17:58:24.378 INFO:teuthology.orchestra.run.smithi007.stderr:2800000 of 3200000 tuples (87%) done (elapsed 8.15 s, remaining 1.16 s) 2024-01-11T17:58:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2900000 of 3200000 tuples (90%) done (elapsed 8.33 s, remaining 0.86 s) 2024-01-11T17:58:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:3000000 of 3200000 tuples (93%) done (elapsed 8.61 s, remaining 0.57 s) 2024-01-11T17:58:24.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:24 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:25.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:24 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:25.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:24 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:3100000 of 3200000 tuples (96%) done (elapsed 8.88 s, remaining 0.29 s) 2024-01-11T17:58:25.233 INFO:teuthology.orchestra.run.smithi007.stderr:3200000 of 3200000 tuples (100%) done (elapsed 9.00 s, remaining 0.00 s) 2024-01-11T17:58:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:vacuum... 2024-01-11T17:58:25.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:25 smithi016 ceph-mon[43195]: pgmap v318: 129 pgs: 129 active+clean; 255 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 40 MiB/s wr, 298 op/s 2024-01-11T17:58:26.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:25 smithi038 ceph-mon[43582]: pgmap v318: 129 pgs: 129 active+clean; 255 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 40 MiB/s wr, 298 op/s 2024-01-11T17:58:26.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:25 smithi007 ceph-mon[33361]: pgmap v318: 129 pgs: 129 active+clean; 255 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 40 MiB/s wr, 298 op/s 2024-01-11T17:58:26.381 INFO:teuthology.orchestra.run.smithi007.stderr:set primary keys... 2024-01-11T17:58:27.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:27 smithi016 ceph-mon[43195]: pgmap v319: 129 pgs: 129 active+clean; 347 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 54 MiB/s wr, 370 op/s 2024-01-11T17:58:27.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:27 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:27 smithi038 ceph-mon[43582]: pgmap v319: 129 pgs: 129 active+clean; 347 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 54 MiB/s wr, 370 op/s 2024-01-11T17:58:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:27 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:28.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:27 smithi007 ceph-mon[33361]: pgmap v319: 129 pgs: 129 active+clean; 347 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 54 MiB/s wr, 370 op/s 2024-01-11T17:58:28.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:27 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:28.967 INFO:teuthology.orchestra.run.smithi007.stderr:done. 2024-01-11T17:58:28.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-01-11T17:58:29.121 INFO:teuthology.orchestra.run.smithi007.stderr:starting vacuum...end. 2024-01-11T17:58:29.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:29 smithi016 ceph-mon[43195]: pgmap v320: 129 pgs: 129 active+clean; 363 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 55 MiB/s wr, 324 op/s 2024-01-11T17:58:30.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:29 smithi038 ceph-mon[43582]: pgmap v320: 129 pgs: 129 active+clean; 363 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 55 MiB/s wr, 324 op/s 2024-01-11T17:58:30.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:29 smithi007 ceph-mon[33361]: pgmap v320: 129 pgs: 129 active+clean; 363 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 55 MiB/s wr, 324 op/s 2024-01-11T17:58:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:31 smithi016 ceph-mon[43195]: pgmap v321: 129 pgs: 129 active+clean; 427 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 62 MiB/s wr, 334 op/s 2024-01-11T17:58:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:31 smithi038 ceph-mon[43582]: pgmap v321: 129 pgs: 129 active+clean; 427 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 62 MiB/s wr, 334 op/s 2024-01-11T17:58:32.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:31 smithi007 ceph-mon[33361]: pgmap v321: 129 pgs: 129 active+clean; 427 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 62 MiB/s wr, 334 op/s 2024-01-11T17:58:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:33 smithi016 ceph-mon[43195]: pgmap v322: 129 pgs: 129 active+clean; 558 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 65 MiB/s wr, 312 op/s 2024-01-11T17:58:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:33 smithi038 ceph-mon[43582]: pgmap v322: 129 pgs: 129 active+clean; 558 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 65 MiB/s wr, 312 op/s 2024-01-11T17:58:34.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:33 smithi007 ceph-mon[33361]: pgmap v322: 129 pgs: 129 active+clean; 558 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 65 MiB/s wr, 312 op/s 2024-01-11T17:58:34.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 5.0 s, 1581.8 tps, lat 60.075 ms stddev 73.800 2024-01-11T17:58:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 -- 172.21.15.7:0/1867619698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 msgr2=0x7f3d6c103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 --2- 172.21.15.7:0/1867619698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c103c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009990 tx=0x7f3d5c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 -- 172.21.15.7:0/1867619698 shutdown_connections 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 --2- 172.21.15.7:0/1867619698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c103c10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 --2- 172.21.15.7:0/1867619698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d6c105380 0x7f3d6c10bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 --2- 172.21.15.7:0/1867619698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d6c1049e0 0x7f3d6c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.141+0000 7f3d723ff700 1 -- 172.21.15.7:0/1867619698 >> 172.21.15.7:0/1867619698 conn(0x7f3d6c0fed60 msgr2=0x7f3d6c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.142+0000 7f3d723ff700 1 -- 172.21.15.7:0/1867619698 shutdown_connections 2024-01-11T17:58:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.142+0000 7f3d723ff700 1 -- 172.21.15.7:0/1867619698 wait complete. 2024-01-11T17:58:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.143+0000 7f3d723ff700 1 Processor -- start 2024-01-11T17:58:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.146+0000 7f3d723ff700 1 -- start start 2024-01-11T17:58:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.146+0000 7f3d723ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.146+0000 7f3d723ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d6c1049e0 0x7f3d6c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d723ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d6c105380 0x7f3d6c1a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d723ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1a55d0 con 0x7f3d6c105380 2024-01-11T17:58:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d723ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1a5710 con 0x7f3d6c1049e0 2024-01-11T17:58:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d723ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1a58a0 con 0x7f3d6c1037f0 2024-01-11T17:58:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44306/0 (socket says 172.21.15.7:44306) 2024-01-11T17:58:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d6bfff700 1 -- 172.21.15.7:0/191184953 learned_addr learned my addr 172.21.15.7:0/191184953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.147+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d6c1049e0 0x7f3d6c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.148+0000 7f3d6bfff700 1 -- 172.21.15.7:0/191184953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d6c1049e0 msgr2=0x7f3d6c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.148+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d6c1049e0 0x7f3d6c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.148+0000 7f3d6bfff700 1 -- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d6c105380 msgr2=0x7f3d6c1a5000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T17:58:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.149+0000 7f3d7099c700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d6c105380 0x7f3d6c1a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.149+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d6c105380 0x7f3d6c1a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.149+0000 7f3d6bfff700 1 -- 172.21.15.7:0/191184953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0095f0 con 0x7f3d6c1037f0 2024-01-11T17:58:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.150+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c19da50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c003790 tx=0x7f3d5c00f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.150+0000 7f3d697fa700 1 -- 172.21.15.7:0/191184953 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c01d070 con 0x7f3d6c1037f0 2024-01-11T17:58:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.151+0000 7f3d697fa700 1 -- 172.21.15.7:0/191184953 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5c00fd00 con 0x7f3d6c1037f0 2024-01-11T17:58:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.151+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c1a5b20 con 0x7f3d6c1037f0 2024-01-11T17:58:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.151+0000 7f3d697fa700 1 -- 172.21.15.7:0/191184953 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0176e0 con 0x7f3d6c1037f0 2024-01-11T17:58:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.151+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c1a5f40 con 0x7f3d6c1037f0 2024-01-11T17:58:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.159+0000 7f3d697fa700 1 -- 172.21.15.7:0/191184953 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f3d5c0229c0 con 0x7f3d6c1037f0 2024-01-11T17:58:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.161+0000 7f3d697fa700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d540a2020 0x7f3d540a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.162+0000 7f3d697fa700 1 -- 172.21.15.7:0/191184953 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3d5c0ceff0 con 0x7f3d6c1037f0 2024-01-11T17:58:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.162+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d540a2020 0x7f3d540a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.163+0000 7f3d86111b80 1 Processor -- start 2024-01-11T17:58:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 -- start start 2024-01-11T17:58:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e696f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e696f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e696fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e695ed030 con 0x555e696f6c10 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69650ac0 con 0x555e696f5f20 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.164+0000 7f3d86111b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6964e3e0 con 0x555e696f75d0 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d540a2020 0x7f3d540a44e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005fd0 tx=0x7f3d6000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e696f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e696f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48146/0 (socket says 172.21.15.7:48146) 2024-01-11T17:58:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 -- 172.21.15.7:0/1962445448 learned_addr learned my addr 172.21.15.7:0/1962445448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 -- 172.21.15.7:0/1962445448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 msgr2=0x555e696fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e696fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 -- 172.21.15.7:0/1962445448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 msgr2=0x555e696f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e696f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.165+0000 7f3d6bfff700 1 -- 172.21.15.7:0/1962445448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e694be350 con 0x555e696f5f20 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.166+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e696f6340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0a3980 tx=0x7f3d5c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b040d1769deef1bc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.166+0000 7f3d527fc700 1 -- 172.21.15.7:0/1962445448 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c09bbb0 con 0x555e696f5f20 2024-01-11T17:58:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.166+0000 7f3d527fc700 1 -- 172.21.15.7:0/1962445448 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5c09fd00 con 0x555e696f5f20 2024-01-11T17:58:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.166+0000 7f3d527fc700 1 -- 172.21.15.7:0/1962445448 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0a3a20 con 0x555e696f5f20 2024-01-11T17:58:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.167+0000 7f3d86111b80 1 -- 172.21.15.7:0/1962445448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 msgr2=0x555e696f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.167+0000 7f3d86111b80 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e696f6340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0a3980 tx=0x7f3d5c003680 comp rx=0 tx=0).stop 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.168+0000 7f3d86111b80 1 -- 172.21.15.7:0/1962445448 shutdown_connections 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.168+0000 7f3d86111b80 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e696fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.168+0000 7f3d86111b80 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e696f6340 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.168+0000 7f3d86111b80 1 --2- 172.21.15.7:0/1962445448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e696f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.168+0000 7f3d86111b80 1 -- 172.21.15.7:0/1962445448 >> 172.21.15.7:0/1962445448 conn(0x555e6966b740 msgr2=0x555e696f0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.169+0000 7f3d86111b80 1 -- 172.21.15.7:0/1962445448 shutdown_connections 2024-01-11T17:58:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.169+0000 7f3d86111b80 1 -- 172.21.15.7:0/1962445448 wait complete. 2024-01-11T17:58:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.170+0000 7f3d86111b80 1 -- start start 2024-01-11T17:58:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.170+0000 7f3d86111b80 1 Processor -- start 2024-01-11T17:58:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.171+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.171+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f6c10 0x555e696ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.171+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e69706c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d7099c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e69706c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f6c10 0x555e696ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d7099c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e69706c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44320/0 (socket says 172.21.15.7:44320) 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d7099c700 1 -- 172.21.15.7:0/961518227 learned_addr learned my addr 172.21.15.7:0/961518227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69649cd0 con 0x555e696f5f20 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e695b8340 con 0x555e696f6c10 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e695b7fe0 con 0x555e696f75d0 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d86111b80 10 client.0 _ll_register_callbacks cb 0x555e695ea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d6b7fe700 1 -- 172.21.15.7:0/961518227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 msgr2=0x555e69706c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e69706c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.172+0000 7f3d6b7fe700 1 -- 172.21.15.7:0/961518227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 msgr2=0x555e696ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.173+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696ff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.173+0000 7f3d6b7fe700 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c1a5f40 con 0x555e696f6c10 2024-01-11T17:58:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.173+0000 7f3d7099c700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e69706c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.173+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:58:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.173+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f6c10 0x555e696ffc00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005950 tx=0x7f3d60010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a505905e0c7e8ba2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.173+0000 7f3d4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.174+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/961518227 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000a370 con 0x555e696f6c10 2024-01-11T17:58:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.175+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/961518227 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3d60004a30 con 0x555e696f6c10 2024-01-11T17:58:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.175+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e696e8d60 con 0x555e696f6c10 2024-01-11T17:58:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.176+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/961518227 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d60004bf0 con 0x555e696f6c10 2024-01-11T17:58:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.176+0000 7f3d86111b80 10 client.25577 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.176+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555e6946ab00 con 0x555e696f6c10 2024-01-11T17:58:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.177+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/961518227 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e696e8d60 con 0x555e696f6c10 2024-01-11T17:58:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.177+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/961518227 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3d60005170 con 0x555e696f6c10 2024-01-11T17:58:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.177+0000 7f3d86111b80 10 client.25577 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.177+0000 7f3d86111b80 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.178+0000 7f3d86111b80 20 client.25577 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.178+0000 7f3d86111b80 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e6970b520 0x555e6970d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.178+0000 7f3d86111b80 4 client.25577 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.178+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e69642b60 con 0x555e6970b520 2024-01-11T17:58:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.178+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e6970b520 0x555e6970d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.179+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e6970b520 0x555e6970d940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97025679d8efd3b5 server_cookie=6a464bbc3114cd99 in_seq=0 out_seq=0 2024-01-11T17:58:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.179+0000 7f3d4b7fe700 10 client.25577 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.180+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/961518227 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e69642b60 con 0x555e6970b520 2024-01-11T17:58:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.180+0000 7f3d4b7fe700 10 client.25577 handle_command_reply: tid=0 2024-01-11T17:58:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.181+0000 7f3d86111b80 1 client.25577 shutdown 2024-01-11T17:58:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.181+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f6c10 msgr2=0x555e696ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.181+0000 7f3d86111b80 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f6c10 0x555e696ffc00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005950 tx=0x7f3d60010990 comp rx=0 tx=0).stop 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 shutdown_connections 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d86111b80 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e6970b520 0x555e6970d940 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d86111b80 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e69706c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d86111b80 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d4b7fe700 0 client.25577 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d86111b80 1 --2- 172.21.15.7:0/961518227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f6c10 0x555e696ffc00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.182+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 >> 172.21.15.7:0/961518227 conn(0x555e6966b740 msgr2=0x555e696eff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.183+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 shutdown_connections 2024-01-11T17:58:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.183+0000 7f3d86111b80 1 -- 172.21.15.7:0/961518227 wait complete. 2024-01-11T17:58:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.183+0000 7f3d86111b80 20 client.25577 trim_cache size 0 max 16384 2024-01-11T17:58:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.212+0000 7f3d86111b80 1 Processor -- start 2024-01-11T17:58:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.212+0000 7f3d86111b80 1 -- start start 2024-01-11T17:58:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.212+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f6c10 0x555e69706c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 0x555e696f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d86111b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e694c0640 con 0x555e696f5f20 2024-01-11T17:58:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d86111b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69707dd0 con 0x555e696f75d0 2024-01-11T17:58:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d86111b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e696ff230 con 0x555e696f6c10 2024-01-11T17:58:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d7099c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 0x555e696f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d7099c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 0x555e696f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48162/0 (socket says 172.21.15.7:48162) 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d7099c700 1 -- 172.21.15.7:0/3096491321 learned_addr learned my addr 172.21.15.7:0/3096491321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f6c10 0x555e69706c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.213+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6bfff700 1 -- 172.21.15.7:0/3096491321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f6c10 msgr2=0x555e69706c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f6c10 0x555e69706c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6bfff700 1 -- 172.21.15.7:0/3096491321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 msgr2=0x555e696f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 0x555e696f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6bfff700 1 -- 172.21.15.7:0/3096491321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e6960b1d0 con 0x555e696f5f20 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d7099c700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 0x555e696f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f6c10 0x555e69706c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.214+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696e88a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c09b8b0 tx=0x7f3d5c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c0a2cd7115a65 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.215+0000 7f3d51ffb700 1 -- 172.21.15.7:0/3096491321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c099f50 con 0x555e696f5f20 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.215+0000 7f3d51ffb700 1 -- 172.21.15.7:0/3096491321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5c0b4190 con 0x555e696f5f20 2024-01-11T17:58:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.215+0000 7f3d51ffb700 1 -- 172.21.15.7:0/3096491321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0a3a20 con 0x555e696f5f20 2024-01-11T17:58:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.216+0000 7f3d86111b80 1 -- 172.21.15.7:0/3096491321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 msgr2=0x555e696e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.216+0000 7f3d86111b80 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696e88a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c09b8b0 tx=0x7f3d5c009310 comp rx=0 tx=0).stop 2024-01-11T17:58:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.217+0000 7f3d86111b80 1 -- 172.21.15.7:0/3096491321 shutdown_connections 2024-01-11T17:58:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.217+0000 7f3d86111b80 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f75d0 0x555e696f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.217+0000 7f3d86111b80 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f5f20 0x555e696e88a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.217+0000 7f3d86111b80 1 --2- 172.21.15.7:0/3096491321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f6c10 0x555e69706c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.217+0000 7f3d86111b80 1 -- 172.21.15.7:0/3096491321 >> 172.21.15.7:0/3096491321 conn(0x555e6966b740 msgr2=0x555e696ec9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.217+0000 7f3d86111b80 1 -- 172.21.15.7:0/3096491321 shutdown_connections 2024-01-11T17:58:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.218+0000 7f3d86111b80 1 -- 172.21.15.7:0/3096491321 wait complete. 2024-01-11T17:58:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.219+0000 7f3d86111b80 1 -- start start 2024-01-11T17:58:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.219+0000 7f3d86111b80 1 Processor -- start 2024-01-11T17:58:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.221+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e69703ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.221+0000 7f3d86111b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e69704000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.221+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e69703ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.221+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e69703ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48164/0 (socket says 172.21.15.7:48164) 2024-01-11T17:58:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.221+0000 7f3d6bfff700 1 -- 172.21.15.7:0/4223392774 learned_addr learned my addr 172.21.15.7:0/4223392774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.221+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e6975fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.222+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69704710 con 0x555e696f6c10 2024-01-11T17:58:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.222+0000 7f3d7099c700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e6975fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.222+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6975ff40 con 0x555e696f5f20 2024-01-11T17:58:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.222+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e69760100 con 0x555e696f75d0 2024-01-11T17:58:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.222+0000 7f3d86111b80 10 client.0 _ll_register_callbacks cb 0x555e695ea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.222+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e69704000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6bfff700 1 -- 172.21.15.7:0/4223392774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 msgr2=0x555e6975fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e6975fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6bfff700 1 -- 172.21.15.7:0/4223392774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 msgr2=0x555e69704000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e69704000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6bfff700 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e6946ab00 con 0x555e696f5f20 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d7099c700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e6975fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e69704000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d6bfff700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e69703ac0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c098620 tx=0x7f3d5c097ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8838375c24966bdd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.223+0000 7f3d4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.224+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/4223392774 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c099f50 con 0x555e696f5f20 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.224+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/4223392774 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3d5c09b1f0 con 0x555e696f5f20 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.224+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d5c09fd00 con 0x555e696f5f20 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.224+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/4223392774 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d5c0a3a20 con 0x555e696f5f20 2024-01-11T17:58:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.224+0000 7f3d86111b80 10 client.25583 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.224+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f3d64010340 con 0x555e696f5f20 2024-01-11T17:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/4223392774 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5c09fd00 con 0x555e696f5f20 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/4223392774 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3d5c022600 con 0x555e696f5f20 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d86111b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d86111b80 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d86111b80 20 client.25583 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e697617c0 0x555e69763be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.225+0000 7f3d86111b80 4 client.25583 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.226+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e69642880 con 0x555e697617c0 2024-01-11T17:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.226+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e697617c0 0x555e69763be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.227+0000 7f3d6b7fe700 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e697617c0 0x555e69763be0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df160a2043910437 server_cookie=35f4eb4b6d3460f8 in_seq=0 out_seq=0 2024-01-11T17:58:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.227+0000 7f3d4b7fe700 10 client.25583 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.248+0000 7f3d4b7fe700 1 -- 172.21.15.7:0/4223392774 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e69642880 con 0x555e697617c0 2024-01-11T17:58:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.248+0000 7f3d4b7fe700 10 client.25583 handle_command_reply: tid=0 2024-01-11T17:58:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.248+0000 7f3d86111b80 1 client.25583 shutdown 2024-01-11T17:58:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.249+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 msgr2=0x555e69703ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.249+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e69703ac0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c098620 tx=0x7f3d5c097ef0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 shutdown_connections 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555e697617c0 0x555e69763be0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e696f75d0 0x555e6975fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e696f5f20 0x555e69703ac0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d86111b80 1 --2- 172.21.15.7:0/4223392774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e696f6c10 0x555e69704000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 >> 172.21.15.7:0/4223392774 conn(0x555e6966b740 msgr2=0x555e697122a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.250+0000 7f3d4b7fe700 0 client.25583 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.251+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 shutdown_connections 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.251+0000 7f3d86111b80 1 -- 172.21.15.7:0/4223392774 wait complete. 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.251+0000 7f3d86111b80 20 client.25583 trim_cache size 0 max 16384 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:58:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d540a2020 msgr2=0x7f3d540a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d540a2020 0x7f3d540a44e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005fd0 tx=0x7f3d6000a560 comp rx=0 tx=0).stop 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 msgr2=0x7f3d6c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c19da50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c003790 tx=0x7f3d5c00f720 comp rx=0 tx=0).stop 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 shutdown_connections 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d540a2020 0x7f3d540a44e0 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005fd0 tx=0x7f3d6000a560 comp rx=0 tx=0).stop 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d6c1037f0 0x7f3d6c19da50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d6c105380 0x7f3d6c1a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 --2- 172.21.15.7:0/191184953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d6c1049e0 0x7f3d6c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.252+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 >> 172.21.15.7:0/191184953 conn(0x7f3d6c0fed60 msgr2=0x7f3d6c077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.253+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 shutdown_connections 2024-01-11T17:58:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:34.253+0000 7f3d723ff700 1 -- 172.21.15.7:0/191184953 wait complete. 2024-01-11T17:58:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:35 smithi016 ceph-mon[43195]: pgmap v323: 129 pgs: 129 active+clean; 762 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 60 MiB/s wr, 296 op/s 2024-01-11T17:58:35.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:35 smithi038 ceph-mon[43582]: pgmap v323: 129 pgs: 129 active+clean; 762 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 60 MiB/s wr, 296 op/s 2024-01-11T17:58:35.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:35 smithi007 ceph-mon[33361]: pgmap v323: 129 pgs: 129 active+clean; 762 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 60 MiB/s wr, 296 op/s 2024-01-11T17:58:37.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-01-11T17:58:37.951 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:37 smithi007 ceph-mon[33361]: pgmap v324: 129 pgs: 129 active+clean; 982 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 56 MiB/s wr, 288 op/s 2024-01-11T17:58:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:37 smithi016 ceph-mon[43195]: pgmap v324: 129 pgs: 129 active+clean; 982 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 56 MiB/s wr, 288 op/s 2024-01-11T17:58:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:37 smithi038 ceph-mon[43582]: pgmap v324: 129 pgs: 129 active+clean; 982 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 56 MiB/s wr, 288 op/s 2024-01-11T17:58:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 -- 172.21.15.7:0/1591813838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 msgr2=0x7f4ed8100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 --2- 172.21.15.7:0/1591813838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed8100240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009990 tx=0x7f4ec8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 -- 172.21.15.7:0/1591813838 shutdown_connections 2024-01-11T17:58:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 --2- 172.21.15.7:0/1591813838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed8100240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 --2- 172.21.15.7:0/1591813838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 0x7f4ed81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 --2- 172.21.15.7:0/1591813838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 0x7f4ed8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.196+0000 7f4edddff700 1 -- 172.21.15.7:0/1591813838 >> 172.21.15.7:0/1591813838 conn(0x7f4ed80fba00 msgr2=0x7f4ed80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.197+0000 7f4edddff700 1 -- 172.21.15.7:0/1591813838 shutdown_connections 2024-01-11T17:58:38.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.197+0000 7f4edddff700 1 -- 172.21.15.7:0/1591813838 wait complete. 2024-01-11T17:58:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 Processor -- start 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 -- start start 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed81059a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 0x7f4ed8105ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 0x7f4ed8102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8102cd0 con 0x7f4ed8100780 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8102e40 con 0x7f4ed8101140 2024-01-11T17:58:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4edddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8103000 con 0x7f4ed80ffe20 2024-01-11T17:58:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.199+0000 7f4ed77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed81059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.200+0000 7f4ed77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed81059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44348/0 (socket says 172.21.15.7:44348) 2024-01-11T17:58:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.200+0000 7f4ed77fe700 1 -- 172.21.15.7:0/2487807871 learned_addr learned my addr 172.21.15.7:0/2487807871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.200+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 0x7f4ed8102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed6ffd700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 0x7f4ed8105ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed77fe700 1 -- 172.21.15.7:0/2487807871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 msgr2=0x7f4ed8102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 0x7f4ed8102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed77fe700 1 -- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 msgr2=0x7f4ed8105ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 0x7f4ed8105ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed77fe700 1 -- 172.21.15.7:0/2487807871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80095f0 con 0x7f4ed80ffe20 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed6ffd700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 0x7f4ed8105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.201+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 0x7f4ed8102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:38.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.202+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed81059a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800b680 tx=0x7f4ec8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.202+0000 7f4ed4ff9700 1 -- 172.21.15.7:0/2487807871 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec801d070 con 0x7f4ed80ffe20 2024-01-11T17:58:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.202+0000 7f4ed4ff9700 1 -- 172.21.15.7:0/2487807871 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec8022470 con 0x7f4ed80ffe20 2024-01-11T17:58:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.202+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed81a5bb0 con 0x7f4ed80ffe20 2024-01-11T17:58:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.203+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed81a5fd0 con 0x7f4ed80ffe20 2024-01-11T17:58:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.203+0000 7f4ed4ff9700 1 -- 172.21.15.7:0/2487807871 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8017650 con 0x7f4ed80ffe20 2024-01-11T17:58:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.208+0000 7f4ed4ff9700 1 -- 172.21.15.7:0/2487807871 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4ec8022870 con 0x7f4ed80ffe20 2024-01-11T17:58:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.211+0000 7f4ef1af7b80 1 Processor -- start 2024-01-11T17:58:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.212+0000 7f4ef1af7b80 1 -- start start 2024-01-11T17:58:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.213+0000 7f4ed4ff9700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec00a1ee0 0x7f4ec00a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad4168820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4166f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad41679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad405e090 con 0x563ad4168400 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad40c1b20 con 0x563ad4168d80 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad40bf440 con 0x563ad4167570 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ed4ff9700 1 -- 172.21.15.7:0/2487807871 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ec80cf000 con 0x7f4ed80ffe20 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ed7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad41679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ed7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad41679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44356/0 (socket says 172.21.15.7:44356) 2024-01-11T17:58:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ed7fff700 1 -- 172.21.15.7:0/1100505389 learned_addr learned my addr 172.21.15.7:0/1100505389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec00a1ee0 0x7f4ec00a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.214+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad4168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 -- 172.21.15.7:0/1100505389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 msgr2=0x563ad4166f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4166f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 -- 172.21.15.7:0/1100505389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 msgr2=0x563ad4168820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad4168820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 -- 172.21.15.7:0/1100505389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ad3f2f3b0 con 0x563ad4167570 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad41679f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00060b0 tx=0x7f4ed0008dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d33bd990b0e89e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec00a1ee0 0x7f4ec00a43a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0011000 tx=0x7f4ed001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ebdffb700 1 -- 172.21.15.7:0/1100505389 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed001a930 con 0x563ad4167570 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.215+0000 7f4ebdffb700 1 -- 172.21.15.7:0/1100505389 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed000f040 con 0x563ad4167570 2024-01-11T17:58:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.216+0000 7f4ebdffb700 1 -- 172.21.15.7:0/1100505389 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0022d50 con 0x563ad4167570 2024-01-11T17:58:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.216+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/1100505389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 msgr2=0x563ad41679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.216+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad41679f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00060b0 tx=0x7f4ed0008dd0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.217+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/1100505389 shutdown_connections 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.217+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad41679f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.217+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad4168820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.217+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/1100505389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4166f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.217+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/1100505389 >> 172.21.15.7:0/1100505389 conn(0x563ad40dc7a0 msgr2=0x563ad4161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.218+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/1100505389 shutdown_connections 2024-01-11T17:58:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.218+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/1100505389 wait complete. 2024-01-11T17:58:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.219+0000 7f4ef1af7b80 1 -- start start 2024-01-11T17:58:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.220+0000 7f4ef1af7b80 1 Processor -- start 2024-01-11T17:58:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 0x563ad4170ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168400 0x563ad4171430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168d80 0x563ad4178400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad40bad30 con 0x563ad4167570 2024-01-11T17:58:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad40293a0 con 0x563ad4168400 2024-01-11T17:58:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad4029040 con 0x563ad4168d80 2024-01-11T17:58:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ef1af7b80 10 client.0 _ll_register_callbacks cb 0x563ad405b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ed77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 0x563ad4170ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ed7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168d80 0x563ad4178400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ed77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 0x563ad4170ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53064/0 (socket says 172.21.15.7:53064) 2024-01-11T17:58:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.223+0000 7f4ed77fe700 1 -- 172.21.15.7:0/595089323 learned_addr learned my addr 172.21.15.7:0/595089323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed6ffd700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168400 0x563ad4171430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed7fff700 1 -- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168400 msgr2=0x563ad4171430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168400 0x563ad4171430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed7fff700 1 -- 172.21.15.7:0/595089323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 msgr2=0x563ad4170ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 0x563ad4170ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed7fff700 1 -- 172.21.15.7:0/595089323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed00086d0 con 0x563ad4168d80 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.224+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 0x563ad4170ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168d80 0x563ad4178400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000e7d0 tx=0x7f4ed0022f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea6ae83c77ebe7f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4eb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/595089323 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed001ad40 con 0x563ad4168d80 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/595089323 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ed002b430 con 0x563ad4168d80 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ad415a010 con 0x563ad4168d80 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/595089323 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed0009550 con 0x563ad4168d80 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4ef1af7b80 10 client.25612 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:38.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.225+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563ad3edbb60 con 0x563ad4168d80 2024-01-11T17:58:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.226+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/595089323 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ad415a010 con 0x563ad4168d80 2024-01-11T17:58:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.226+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/595089323 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4ed0031020 con 0x563ad4168d80 2024-01-11T17:58:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ef1af7b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ef1af7b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ef1af7b80 20 client.25612 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad417cc70 0x563ad417f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ef1af7b80 4 client.25612 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ad40b3bc0 con 0x563ad417cc70 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.227+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad417cc70 0x563ad417f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.228+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad417cc70 0x563ad417f0b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684fb99b7307ecb1 server_cookie=17277756637a1be8 in_seq=0 out_seq=0 2024-01-11T17:58:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.228+0000 7f4eb6ffd700 10 client.25612 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.229+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/595089323 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ad40b3bc0 con 0x563ad417cc70 2024-01-11T17:58:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.229+0000 7f4eb6ffd700 10 client.25612 handle_command_reply: tid=0 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.229+0000 7f4ef1af7b80 1 client.25612 shutdown 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168d80 msgr2=0x563ad4178400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168d80 0x563ad4178400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000e7d0 tx=0x7f4ed0022f70 comp rx=0 tx=0).stop 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 shutdown_connections 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad417cc70 0x563ad417f0b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168d80 0x563ad4178400 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4167570 0x563ad4170ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/595089323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168400 0x563ad4171430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4eb6ffd700 0 client.25612 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 >> 172.21.15.7:0/595089323 conn(0x563ad40dc7a0 msgr2=0x563ad4165e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 shutdown_connections 2024-01-11T17:58:38.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/595089323 wait complete. 2024-01-11T17:58:38.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.230+0000 7f4ef1af7b80 20 client.25612 trim_cache size 0 max 16384 2024-01-11T17:58:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.270+0000 7f4ef1af7b80 1 Processor -- start 2024-01-11T17:58:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.270+0000 7f4ef1af7b80 1 -- start start 2024-01-11T17:58:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4167570 0x563ad4166eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168400 0x563ad41783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168d80 0x563ad416eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad3f316a0 con 0x563ad4168d80 2024-01-11T17:58:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad41680a0 con 0x563ad4167570 2024-01-11T17:58:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad41673f0 con 0x563ad4168400 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ed6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168400 0x563ad41783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168400 0x563ad41783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44372/0 (socket says 172.21.15.7:44372) 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed6ffd700 1 -- 172.21.15.7:0/2804955885 learned_addr learned my addr 172.21.15.7:0/2804955885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.271+0000 7f4ed7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168d80 0x563ad416eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed7fff700 1 -- 172.21.15.7:0/2804955885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168400 msgr2=0x563ad41783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168400 0x563ad41783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed7fff700 1 -- 172.21.15.7:0/2804955885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4167570 msgr2=0x563ad4166eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4167570 0x563ad4166eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.272+0000 7f4ed7fff700 1 -- 172.21.15.7:0/2804955885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ad407c230 con 0x563ad4168d80 2024-01-11T17:58:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.273+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168d80 0x563ad416eea0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00089c0 tx=0x7f4ed0008dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbc740393db1774 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.273+0000 7f4ebd7fa700 1 -- 172.21.15.7:0/2804955885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed001ad40 con 0x563ad4168d80 2024-01-11T17:58:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.273+0000 7f4ebd7fa700 1 -- 172.21.15.7:0/2804955885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed0009550 con 0x563ad4168d80 2024-01-11T17:58:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.273+0000 7f4ebd7fa700 1 -- 172.21.15.7:0/2804955885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0033e50 con 0x563ad4168d80 2024-01-11T17:58:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.274+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/2804955885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168d80 msgr2=0x563ad416eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.274+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168d80 0x563ad416eea0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00089c0 tx=0x7f4ed0008dd0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.275+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/2804955885 shutdown_connections 2024-01-11T17:58:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.275+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168d80 0x563ad416eea0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.275+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4167570 0x563ad4166eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.275+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/2804955885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4168400 0x563ad41783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.275+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/2804955885 >> 172.21.15.7:0/2804955885 conn(0x563ad40dc7a0 msgr2=0x563ad4183a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.276+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/2804955885 shutdown_connections 2024-01-11T17:58:38.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.276+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/2804955885 wait complete. 2024-01-11T17:58:38.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.277+0000 7f4ef1af7b80 1 -- start start 2024-01-11T17:58:38.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.278+0000 7f4ef1af7b80 1 Processor -- start 2024-01-11T17:58:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.278+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad4175f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad41764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ef1af7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4177380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad41778c0 con 0x563ad4168400 2024-01-11T17:58:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad4177a30 con 0x563ad4168d80 2024-01-11T17:58:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ef1af7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ad4177bf0 con 0x563ad4167570 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ef1af7b80 10 client.0 _ll_register_callbacks cb 0x563ad405b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ed7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4177380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ed7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4177380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48202/0 (socket says 172.21.15.7:48202) 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.279+0000 7f4ed7fff700 1 -- 172.21.15.7:0/3591078747 learned_addr learned my addr 172.21.15.7:0/3591078747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad4175f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed6ffd700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad41764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed7fff700 1 -- 172.21.15.7:0/3591078747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 msgr2=0x563ad4175f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad4175f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed7fff700 1 -- 172.21.15.7:0/3591078747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 msgr2=0x563ad41764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:38.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad41764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed7fff700 1 -- 172.21.15.7:0/3591078747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ad3edbb60 con 0x563ad4168d80 2024-01-11T17:58:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed6ffd700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad41764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4ed7fff700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4177380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00086a0 tx=0x7f4ed00338b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde2a2cb1cd463d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4eb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.280+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/3591078747 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00227b0 con 0x563ad4168d80 2024-01-11T17:58:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.281+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ed000f040 con 0x563ad4168d80 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.281+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/3591078747 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ed0022910 con 0x563ad4168d80 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.281+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/3591078747 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed001ae10 con 0x563ad4168d80 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.281+0000 7f4ef1af7b80 10 client.25592 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.281+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563ad3f33cd0 con 0x563ad4168d80 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/3591078747 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed000f040 con 0x563ad4168d80 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/3591078747 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4ed000e4f0 con 0x563ad4168d80 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4ef1af7b80 10 client.25592 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4ef1af7b80 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:38.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4ef1af7b80 20 client.25592 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:38.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad41d3a90 0x563ad41d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4ef1af7b80 4 client.25592 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-01-11T17:58:38.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.282+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563ad40b38e0 con 0x563ad41d3a90 2024-01-11T17:58:38.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.283+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad41d3a90 0x563ad41d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.285+0000 7f4ed77fe700 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad41d3a90 0x563ad41d5eb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0df8f6b8732976 server_cookie=e5526fe9f91cd501 in_seq=0 out_seq=0 2024-01-11T17:58:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.286+0000 7f4eb6ffd700 10 client.25592 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.286+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/3591078747 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563ad40b38e0 con 0x563ad41d3a90 2024-01-11T17:58:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.286+0000 7f4eb6ffd700 10 client.25592 handle_command_reply: tid=0 2024-01-11T17:58:38.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.286+0000 7f4ef1af7b80 1 client.25592 shutdown 2024-01-11T17:58:38.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 msgr2=0x563ad4177380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4177380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00086a0 tx=0x7f4ed00338b0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 shutdown_connections 2024-01-11T17:58:38.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563ad41d3a90 0x563ad41d5eb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563ad4167570 0x563ad4175f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ad4168d80 0x563ad4177380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4eb6ffd700 0 client.25592 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 --2- 172.21.15.7:0/3591078747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ad4168400 0x563ad41764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.287+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 >> 172.21.15.7:0/3591078747 conn(0x563ad40dc7a0 msgr2=0x563ad4183a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.288+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 shutdown_connections 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.288+0000 7f4ef1af7b80 1 -- 172.21.15.7:0/3591078747 wait complete. 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.288+0000 7f4ef1af7b80 20 client.25592 trim_cache size 0 max 16384 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-01-11T17:58:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec00a1ee0 msgr2=0x7f4ec00a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec00a1ee0 0x7f4ec00a43a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0011000 tx=0x7f4ed001e000 comp rx=0 tx=0).stop 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 msgr2=0x7f4ed81059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed81059a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800b680 tx=0x7f4ec8009580 comp rx=0 tx=0).stop 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 shutdown_connections 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f4ec00a1ee0 0x7f4ec00a43a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ed80ffe20 0x7f4ed81059a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed8101140 0x7f4ed8102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 --2- 172.21.15.7:0/2487807871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ed8100780 0x7f4ed8105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.289+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 >> 172.21.15.7:0/2487807871 conn(0x7f4ed80fba00 msgr2=0x7f4ed80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.290+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 shutdown_connections 2024-01-11T17:58:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.290+0000 7f4edddff700 1 -- 172.21.15.7:0/2487807871 wait complete. 2024-01-11T17:58:38.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-01-11T17:58:38.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:58:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.691+0000 7fc53f3ff700 1 -- 172.21.15.7:0/776783464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 msgr2=0x7fc538109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.691+0000 7fc53f3ff700 1 --2- 172.21.15.7:0/776783464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc538109740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc530009f90 tx=0x7fc53000b330 comp rx=0 tx=0).stop 2024-01-11T17:58:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.692+0000 7fc53f3ff700 1 -- 172.21.15.7:0/776783464 shutdown_connections 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.692+0000 7fc53f3ff700 1 --2- 172.21.15.7:0/776783464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc538109740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.692+0000 7fc53f3ff700 1 --2- 172.21.15.7:0/776783464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc538102070 0x7fc538104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.692+0000 7fc53f3ff700 1 --2- 172.21.15.7:0/776783464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5381049a0 0x7fc538106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.692+0000 7fc53f3ff700 1 -- 172.21.15.7:0/776783464 >> 172.21.15.7:0/776783464 conn(0x7fc5380fba60 msgr2=0x7fc5380fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.693+0000 7fc53f3ff700 1 -- 172.21.15.7:0/776783464 shutdown_connections 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.693+0000 7fc53f3ff700 1 -- 172.21.15.7:0/776783464 wait complete. 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.694+0000 7fc53f3ff700 1 Processor -- start 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.694+0000 7fc53f3ff700 1 -- start start 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53f3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc538102070 0x7fc538197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53f3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5381049a0 0x7fc5381978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53f3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc53819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53f3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819ef00 con 0x7fc538102070 2024-01-11T17:58:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53f3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819f040 con 0x7fc5381049a0 2024-01-11T17:58:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53f3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819f1d0 con 0x7fc5381072d0 2024-01-11T17:58:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc53819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc53819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44408/0 (socket says 172.21.15.7:44408) 2024-01-11T17:58:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53d99c700 1 -- 172.21.15.7:0/2175442785 learned_addr learned my addr 172.21.15.7:0/2175442785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.695+0000 7fc53c99a700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5381049a0 0x7fc5381978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.696+0000 7fc53d99c700 1 -- 172.21.15.7:0/2175442785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5381049a0 msgr2=0x7fc5381978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.696+0000 7fc53d99c700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5381049a0 0x7fc5381978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.696+0000 7fc53d99c700 1 -- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc538102070 msgr2=0x7fc538197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.696+0000 7fc53d99c700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc538102070 0x7fc538197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.696+0000 7fc53d99c700 1 -- 172.21.15.7:0/2175442785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc530009c40 con 0x7fc5381072d0 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.696+0000 7fc53d99c700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc53819e930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc530009790 tx=0x7fc530004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.697+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53000e040 con 0x7fc5381072d0 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.697+0000 7fc53f3ff700 1 -- 172.21.15.7:0/2175442785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53819f450 con 0x7fc5381072d0 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.697+0000 7fc53f3ff700 1 -- 172.21.15.7:0/2175442785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53819f920 con 0x7fc5381072d0 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.697+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53000be00 con 0x7fc5381072d0 2024-01-11T17:58:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.697+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530005330 con 0x7fc5381072d0 2024-01-11T17:58:38.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.702+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518005330 con 0x7fc5381072d0 2024-01-11T17:58:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.704+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc530005490 con 0x7fc5381072d0 2024-01-11T17:58:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.706+0000 7fc52e7fc700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5240a1fe0 0x7fc5240a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.706+0000 7fc53d19b700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5240a1fe0 0x7fc5240a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.708+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc5300cf3b0 con 0x7fc5381072d0 2024-01-11T17:58:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.709+0000 7fc53d19b700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5240a1fe0 0x7fc5240a44a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc534005fd0 tx=0x7fc534005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:38.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:38.712+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc530098150 con 0x7fc5381072d0 2024-01-11T17:58:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 10.0 s, 1668.9 tps, lat 59.213 ms stddev 77.262 2024-01-11T17:58:39.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.166+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc518005b60 con 0x7fc5381072d0 2024-01-11T17:58:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.169+0000 7fc52e7fc700 1 -- 172.21.15.7:0/2175442785 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11313 (secure 0 0 0) 0x7fc53009b770 con 0x7fc5381072d0 2024-01-11T17:58:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:58:39.172 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:58:26.312498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[0],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5240a1fe0 msgr2=0x7fc5240a44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5240a1fe0 0x7fc5240a44a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc534005fd0 tx=0x7fc534005d00 comp rx=0 tx=0).stop 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 msgr2=0x7fc53819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc53819e930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc530009790 tx=0x7fc530004d30 comp rx=0 tx=0).stop 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 shutdown_connections 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc5240a1fe0 0x7fc5240a44a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5381072d0 0x7fc53819e930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc538102070 0x7fc538197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 --2- 172.21.15.7:0/2175442785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5381049a0 0x7fc5381978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 >> 172.21.15.7:0/2175442785 conn(0x7fc5380fba60 msgr2=0x7fc538102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 shutdown_connections 2024-01-11T17:58:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.174+0000 7fc523fff700 1 -- 172.21.15.7:0/2175442785 wait complete. 2024-01-11T17:58:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-11T17:58:39.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-01-11T17:58:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.566+0000 7f23c5fff700 1 -- 172.21.15.7:0/3099735501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 msgr2=0x7f23c00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.566+0000 7f23c5fff700 1 --2- 172.21.15.7:0/3099735501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c00744f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23b40098f0 tx=0x7f23b4009c00 comp rx=0 tx=0).stop 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 -- 172.21.15.7:0/3099735501 shutdown_connections 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 --2- 172.21.15.7:0/3099735501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c010db80 0x7f23c010ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 --2- 172.21.15.7:0/3099735501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0075650 0x7f23c0073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 --2- 172.21.15.7:0/3099735501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c00744f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 -- 172.21.15.7:0/3099735501 >> 172.21.15.7:0/3099735501 conn(0x7f23c00fcb00 msgr2=0x7f23c00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 -- 172.21.15.7:0/3099735501 shutdown_connections 2024-01-11T17:58:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.567+0000 7f23c5fff700 1 -- 172.21.15.7:0/3099735501 wait complete. 2024-01-11T17:58:39.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.569+0000 7f23c5fff700 1 Processor -- start 2024-01-11T17:58:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.569+0000 7f23c5fff700 1 -- start start 2024-01-11T17:58:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23c5fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23c5fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0075650 0x7f23c019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23c5fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c010db80 0x7f23c01a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23c5fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c01a56a0 con 0x7f23c0075650 2024-01-11T17:58:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23c5fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c01a57e0 con 0x7f23c010db80 2024-01-11T17:58:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23c5fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c01a5970 con 0x7f23c0074070 2024-01-11T17:58:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44434/0 (socket says 172.21.15.7:44434) 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23bf7fe700 1 -- 172.21.15.7:0/4000338097 learned_addr learned my addr 172.21.15.7:0/4000338097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.570+0000 7f23bffff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c010db80 0x7f23c01a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23beffd700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0075650 0x7f23c019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23bffff700 1 -- 172.21.15.7:0/4000338097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 msgr2=0x7f23c019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23bffff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23bffff700 1 -- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0075650 msgr2=0x7f23c019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23bffff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0075650 0x7f23c019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23bffff700 1 -- 172.21.15.7:0/4000338097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00095f0 con 0x7f23c010db80 2024-01-11T17:58:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.571+0000 7f23beffd700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0075650 0x7f23c019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.572+0000 7f23bffff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c010db80 0x7f23c01a50d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f23b800efb0 tx=0x7f23b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.572+0000 7f23bcff9700 1 -- 172.21.15.7:0/4000338097 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b800ca50 con 0x7f23c010db80 2024-01-11T17:58:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.572+0000 7f23bcff9700 1 -- 172.21.15.7:0/4000338097 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b800f040 con 0x7f23c010db80 2024-01-11T17:58:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.572+0000 7f23bcff9700 1 -- 172.21.15.7:0/4000338097 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8004620 con 0x7f23c010db80 2024-01-11T17:58:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.572+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b8009c40 con 0x7f23c010db80 2024-01-11T17:58:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.573+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b4009520 con 0x7f23c010db80 2024-01-11T17:58:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.578+0000 7f23bcff9700 1 -- 172.21.15.7:0/4000338097 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f23b8004780 con 0x7f23c010db80 2024-01-11T17:58:39.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.581+0000 7f23bcff9700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23a80a1f60 0x7f23a80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.581+0000 7f23bcff9700 1 -- 172.21.15.7:0/4000338097 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23b80cd8b0 con 0x7f23c010db80 2024-01-11T17:58:39.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.582+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23a80a1f60 0x7f23a80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.583+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23a80a1f60 0x7f23a80a4420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005d10 tx=0x7f23b0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:39.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.583+0000 7f23d9d4cb80 1 Processor -- start 2024-01-11T17:58:39.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.584+0000 7f23d9d4cb80 1 -- start start 2024-01-11T17:58:39.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3400 0x56181d6d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d5c9090 con 0x56181d6d3400 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d62cb20 con 0x56181d6d3d80 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d62a440 con 0x56181d6d2570 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23bffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.585+0000 7f23beffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48262/0 (socket says 172.21.15.7:48262) 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 -- 172.21.15.7:0/1145018902 learned_addr learned my addr 172.21.15.7:0/1145018902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 -- 172.21.15.7:0/1145018902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 msgr2=0x56181d6d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 -- 172.21.15.7:0/1145018902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3400 msgr2=0x56181d6d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3400 0x56181d6d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.586+0000 7f23beffd700 1 -- 172.21.15.7:0/1145018902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56181d49a3b0 con 0x56181d6d3d80 2024-01-11T17:58:39.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.587+0000 7f23beffd700 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6d1f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009230 tx=0x7f23b4009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=712feba6bd724f9f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:39.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.587+0000 7f23a5ffb700 1 -- 172.21.15.7:0/1145018902 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401c070 con 0x56181d6d3d80 2024-01-11T17:58:39.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.587+0000 7f23a5ffb700 1 -- 172.21.15.7:0/1145018902 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b400b560 con 0x56181d6d3d80 2024-01-11T17:58:39.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.587+0000 7f23a5ffb700 1 -- 172.21.15.7:0/1145018902 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4005460 con 0x56181d6d3d80 2024-01-11T17:58:39.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.588+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1145018902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 msgr2=0x56181d6d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.588+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6d1f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009230 tx=0x7f23b4009420 comp rx=0 tx=0).stop 2024-01-11T17:58:39.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1145018902 shutdown_connections 2024-01-11T17:58:39.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3400 0x56181d6d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1145018902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6d1f90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1145018902 >> 172.21.15.7:0/1145018902 conn(0x56181d6477a0 msgr2=0x56181d6ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1145018902 shutdown_connections 2024-01-11T17:58:39.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.589+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1145018902 wait complete. 2024-01-11T17:58:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.590+0000 7f23d9d4cb80 1 -- start start 2024-01-11T17:58:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.591+0000 7f23d9d4cb80 1 Processor -- start 2024-01-11T17:58:39.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.592+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.593+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3400 0x56181d6dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.594+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.594+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d625d30 con 0x56181d6d3d80 2024-01-11T17:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.594+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d5943a0 con 0x56181d6d3400 2024-01-11T17:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.594+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d594040 con 0x56181d6d2570 2024-01-11T17:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.594+0000 7f23d9d4cb80 10 client.0 _ll_register_callbacks cb 0x56181d5c6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.595+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.595+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6dbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44448/0 (socket says 172.21.15.7:44448) 2024-01-11T17:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.595+0000 7f23bf7fe700 1 -- 172.21.15.7:0/1260758799 learned_addr learned my addr 172.21.15.7:0/1260758799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.595+0000 7f23beffd700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3400 0x56181d6dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.596+0000 7f23bffff700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.596+0000 7f23bf7fe700 1 -- 172.21.15.7:0/1260758799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3400 msgr2=0x56181d6dc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.596+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3400 0x56181d6dc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.596+0000 7f23bf7fe700 1 -- 172.21.15.7:0/1260758799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 msgr2=0x56181d6e33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.596+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6e33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.596+0000 7f23bf7fe700 1 -- 172.21.15.7:0/1260758799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c0110920 con 0x56181d6d2570 2024-01-11T17:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.597+0000 7f23bffff700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.597+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6dbe30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23b000ed00 tx=0x7f23b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc7ff9f93d0b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.597+0000 7f239effd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.598+0000 7f239effd700 1 -- 172.21.15.7:0/1260758799 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b001cd40 con 0x56181d6d2570 2024-01-11T17:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.598+0000 7f239effd700 1 -- 172.21.15.7:0/1260758799 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23b0023070 con 0x56181d6d2570 2024-01-11T17:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.598+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56181d6c5010 con 0x56181d6d2570 2024-01-11T17:58:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.598+0000 7f239effd700 1 -- 172.21.15.7:0/1260758799 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b001cea0 con 0x56181d6d2570 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.599+0000 7f23d9d4cb80 10 client.25636 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.599+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56181d446b60 con 0x56181d6d2570 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.599+0000 7f239effd700 1 -- 172.21.15.7:0/1260758799 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56181d6c5010 con 0x56181d6d2570 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.599+0000 7f239effd700 1 -- 172.21.15.7:0/1260758799 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f23b0032700 con 0x56181d6d2570 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23d9d4cb80 10 client.25636 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23d9d4cb80 10 client.25636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23d9d4cb80 20 client.25636 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d6e7c00 0x56181d6ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23d9d4cb80 4 client.25636 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56181d61ebc0 con 0x56181d6e7c00 2024-01-11T17:58:39.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.600+0000 7f23beffd700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d6e7c00 0x56181d6ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.601+0000 7f23beffd700 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d6e7c00 0x56181d6ea020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64dffeabc8130fca server_cookie=179a201bb0fbf5f5 in_seq=0 out_seq=0 2024-01-11T17:58:39.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.602+0000 7f239effd700 10 client.25636 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.603+0000 7f239effd700 1 -- 172.21.15.7:0/1260758799 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56181d61ebc0 con 0x56181d6e7c00 2024-01-11T17:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.603+0000 7f239effd700 10 client.25636 handle_command_reply: tid=0 2024-01-11T17:58:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.603+0000 7f23d9d4cb80 1 client.25636 shutdown 2024-01-11T17:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 msgr2=0x56181d6dbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6dbe30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23b000ed00 tx=0x7f23b0004750 comp rx=0 tx=0).stop 2024-01-11T17:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 shutdown_connections 2024-01-11T17:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d6e7c00 0x56181d6ea020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d2570 0x56181d6dbe30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/1260758799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3400 0x56181d6dc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 >> 172.21.15.7:0/1260758799 conn(0x56181d6477a0 msgr2=0x56181d6d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f239effd700 0 client.25636 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 shutdown_connections 2024-01-11T17:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.605+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/1260758799 wait complete. 2024-01-11T17:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.606+0000 7f23d9d4cb80 20 client.25636 trim_cache size 0 max 16384 2024-01-11T17:58:39.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.644+0000 7f23d9d4cb80 1 Processor -- start 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 -- start start 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d2570 0x56181d6d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d49c6a0 con 0x56181d6d3d80 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d6e4530 con 0x56181d6d2570 2024-01-11T17:58:39.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.645+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d6ce0d0 con 0x56181d6d3400 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d2570 0x56181d6d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6e3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44464/0 (socket says 172.21.15.7:44464) 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 -- 172.21.15.7:0/2184865960 learned_addr learned my addr 172.21.15.7:0/2184865960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 -- 172.21.15.7:0/2184865960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d2570 msgr2=0x56181d6d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d2570 0x56181d6d1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 -- 172.21.15.7:0/2184865960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 msgr2=0x56181d6d9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23bffff700 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6d9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.646+0000 7f23beffd700 1 -- 172.21.15.7:0/2184865960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56181d5e7230 con 0x56181d6d3400 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.647+0000 7f23bffff700 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.647+0000 7f23beffd700 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6e3340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000c00 tx=0x7f23b4009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24098a904caa65e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.647+0000 7f23a57fa700 1 -- 172.21.15.7:0/2184865960 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401c040 con 0x56181d6d3400 2024-01-11T17:58:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.647+0000 7f23a57fa700 1 -- 172.21.15.7:0/2184865960 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b400f640 con 0x56181d6d3400 2024-01-11T17:58:39.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.648+0000 7f23a57fa700 1 -- 172.21.15.7:0/2184865960 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4004930 con 0x56181d6d3400 2024-01-11T17:58:39.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.649+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2184865960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 msgr2=0x56181d6e3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.649+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6e3340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000c00 tx=0x7f23b4009fd0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2184865960 shutdown_connections 2024-01-11T17:58:39.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d3d80 0x56181d6d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d2570 0x56181d6d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2184865960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6e3340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2184865960 >> 172.21.15.7:0/2184865960 conn(0x56181d6477a0 msgr2=0x56181d6db3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2184865960 shutdown_connections 2024-01-11T17:58:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.651+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2184865960 wait complete. 2024-01-11T17:58:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.652+0000 7f23d9d4cb80 1 -- start start 2024-01-11T17:58:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.652+0000 7f23d9d4cb80 1 Processor -- start 2024-01-11T17:58:39.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.653+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d2570 0x56181d6de970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.653+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23d9d4cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d6e2290 con 0x56181d6d2570 2024-01-11T17:58:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d6e2400 con 0x56181d6d3d80 2024-01-11T17:58:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23d9d4cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181d6e25c0 con 0x56181d6d3400 2024-01-11T17:58:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23d9d4cb80 10 client.0 _ll_register_callbacks cb 0x56181d5c6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6deeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44474/0 (socket says 172.21.15.7:44474) 2024-01-11T17:58:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 -- 172.21.15.7:0/2892879594 learned_addr learned my addr 172.21.15.7:0/2892879594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23bffff700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 -- 172.21.15.7:0/2892879594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 msgr2=0x56181d6e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 -- 172.21.15.7:0/2892879594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d2570 msgr2=0x56181d6de970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d2570 0x56181d6de970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.654+0000 7f23beffd700 1 -- 172.21.15.7:0/2892879594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56181d49a3b0 con 0x56181d6d3400 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.655+0000 7f23bffff700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.655+0000 7f23beffd700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6deeb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f23b4032040 tx=0x7f23b400b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=598279d6a4119f90 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.655+0000 7f239effd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.655+0000 7f239effd700 1 -- 172.21.15.7:0/2892879594 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401c040 con 0x56181d6d3400 2024-01-11T17:58:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.656+0000 7f239effd700 1 -- 172.21.15.7:0/2892879594 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23b40055d0 con 0x56181d6d3400 2024-01-11T17:58:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.656+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b400b560 con 0x56181d6d3400 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.656+0000 7f239effd700 1 -- 172.21.15.7:0/2892879594 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b40046d0 con 0x56181d6d3400 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.656+0000 7f23d9d4cb80 10 client.25642 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.656+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56181d49ecd0 con 0x56181d6d3400 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.656+0000 7f239effd700 1 -- 172.21.15.7:0/2892879594 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b400b560 con 0x56181d6d3400 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f239effd700 1 -- 172.21.15.7:0/2892879594 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f23b4009c70 con 0x56181d6d3400 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23d9d4cb80 10 client.25642 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23d9d4cb80 10 client.25642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23d9d4cb80 20 client.25642 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d74f310 0x56181d751730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23d9d4cb80 4 client.25642 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56181d61e8e0 con 0x56181d74f310 2024-01-11T17:58:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.657+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d74f310 0x56181d751730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23bf7fe700 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d74f310 0x56181d751730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b43a4d08ee48a5a9 server_cookie=56d547683d354f9 in_seq=0 out_seq=0 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f239effd700 10 client.25642 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f239effd700 1 -- 172.21.15.7:0/2892879594 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56181d61e8e0 con 0x56181d74f310 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f239effd700 10 client.25642 handle_command_reply: tid=0 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 client.25642 shutdown 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 msgr2=0x56181d6deeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6deeb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f23b4032040 tx=0x7f23b400b780 comp rx=0 tx=0).stop 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 shutdown_connections 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x56181d74f310 0x56181d751730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56181d6d3d80 0x56181d6e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56181d6d2570 0x56181d6de970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 --2- 172.21.15.7:0/2892879594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56181d6d3400 0x56181d6deeb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 >> 172.21.15.7:0/2892879594 conn(0x56181d6477a0 msgr2=0x56181d6d9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.658+0000 7f239effd700 0 client.25642 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:58:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.659+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 shutdown_connections 2024-01-11T17:58:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.659+0000 7f23d9d4cb80 1 -- 172.21.15.7:0/2892879594 wait complete. 2024-01-11T17:58:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.659+0000 7f23d9d4cb80 20 client.25642 trim_cache size 0 max 16384 2024-01-11T17:58:39.661 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23a80a1f60 msgr2=0x7f23a80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23a80a1f60 0x7f23a80a4420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005d10 tx=0x7f23b0009310 comp rx=0 tx=0).stop 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c010db80 msgr2=0x7f23c01a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c010db80 0x7f23c01a50d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f23b800efb0 tx=0x7f23b800c3a0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 shutdown_connections 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f23a80a1f60 0x7f23a80a4420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23c0074070 0x7f23c019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c010db80 0x7f23c01a50d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 --2- 172.21.15.7:0/4000338097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c0075650 0x7f23c019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 >> 172.21.15.7:0/4000338097 conn(0x7f23c00fcb00 msgr2=0x7f23c0100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 shutdown_connections 2024-01-11T17:58:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:39.661+0000 7f23c5fff700 1 -- 172.21.15.7:0/4000338097 wait complete. 2024-01-11T17:58:39.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-01-11T17:58:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:39 smithi016 ceph-mon[43195]: pgmap v325: 129 pgs: 129 active+clean; 996 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 42 MiB/s wr, 233 op/s 2024-01-11T17:58:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:39 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2175442785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:39 smithi038 ceph-mon[43582]: pgmap v325: 129 pgs: 129 active+clean; 996 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 42 MiB/s wr, 233 op/s 2024-01-11T17:58:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:39 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2175442785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.054+0000 7f35505ff700 1 -- 172.21.15.7:0/3047807958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548102020 msgr2=0x7f35481088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.054+0000 7f35505ff700 1 --2- 172.21.15.7:0/3047807958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548102020 0x7f35481088d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3538009990 tx=0x7f3538009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.055+0000 7f35505ff700 1 -- 172.21.15.7:0/3047807958 shutdown_connections 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.055+0000 7f35505ff700 1 --2- 172.21.15.7:0/3047807958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548102020 0x7f35481088d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.055+0000 7f35505ff700 1 --2- 172.21.15.7:0/3047807958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 0x7f35481008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.055+0000 7f35505ff700 1 --2- 172.21.15.7:0/3047807958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3548101680 0x7f3548101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.055+0000 7f35505ff700 1 -- 172.21.15.7:0/3047807958 >> 172.21.15.7:0/3047807958 conn(0x7f35480fba00 msgr2=0x7f35480fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.057+0000 7f35505ff700 1 -- 172.21.15.7:0/3047807958 shutdown_connections 2024-01-11T17:58:40.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.057+0000 7f35505ff700 1 -- 172.21.15.7:0/3047807958 wait complete. 2024-01-11T17:58:40.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.059+0000 7f35505ff700 1 Processor -- start 2024-01-11T17:58:40.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.059+0000 7f35505ff700 1 -- start start 2024-01-11T17:58:40.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.060+0000 7f35505ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 0x7f354806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.060+0000 7f35505ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 0x7f354806c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.060+0000 7f35505ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3548102020 0x7f3548073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.060+0000 7f35505ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481addc0 con 0x7f3548102020 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.060+0000 7f35505ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481adf00 con 0x7f3548100490 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.060+0000 7f35505ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481ae090 con 0x7f3548101680 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.061+0000 7f354eb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3548102020 0x7f3548073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.061+0000 7f354db9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 0x7f354806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.061+0000 7f354db9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 0x7f354806c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44488/0 (socket says 172.21.15.7:44488) 2024-01-11T17:58:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.061+0000 7f354db9a700 1 -- 172.21.15.7:0/2203130172 learned_addr learned my addr 172.21.15.7:0/2203130172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.062+0000 7f354e39b700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 0x7f354806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.062+0000 7f354db9a700 1 -- 172.21.15.7:0/2203130172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 msgr2=0x7f354806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.062+0000 7f354db9a700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 0x7f354806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.062+0000 7f354db9a700 1 -- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3548102020 msgr2=0x7f3548073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.062+0000 7f354db9a700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3548102020 0x7f3548073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.062+0000 7f354db9a700 1 -- 172.21.15.7:0/2203130172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35380095f0 con 0x7f3548101680 2024-01-11T17:58:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.063+0000 7f354e39b700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 0x7f354806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.063+0000 7f354db9a700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 0x7f354806c240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9d0 tx=0x7f354400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.064+0000 7f353f7fe700 1 -- 172.21.15.7:0/2203130172 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544009790 con 0x7f3548101680 2024-01-11T17:58:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.065+0000 7f353f7fe700 1 -- 172.21.15.7:0/2203130172 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f354400cbd0 con 0x7f3548101680 2024-01-11T17:58:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.065+0000 7f353f7fe700 1 -- 172.21.15.7:0/2203130172 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544018680 con 0x7f3548101680 2024-01-11T17:58:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.065+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35481ae370 con 0x7f3548101680 2024-01-11T17:58:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.065+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35481ae830 con 0x7f3548101680 2024-01-11T17:58:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:39 smithi007 ceph-mon[33361]: pgmap v325: 129 pgs: 129 active+clean; 996 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 42 MiB/s wr, 233 op/s 2024-01-11T17:58:40.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:39 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2175442785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:58:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.071+0000 7f353f7fe700 1 -- 172.21.15.7:0/2203130172 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f3544010430 con 0x7f3548101680 2024-01-11T17:58:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.072+0000 7f356423ab80 1 Processor -- start 2024-01-11T17:58:40.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.074+0000 7f356423ab80 1 -- start start 2024-01-11T17:58:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f356423ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc08a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f356423ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc088f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f356423ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc0899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f356423ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbf80090 con 0x5639dc08a400 2024-01-11T17:58:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f356423ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbfe3b20 con 0x5639dc089570 2024-01-11T17:58:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f356423ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbfe1440 con 0x5639dc08ad80 2024-01-11T17:58:40.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f354e39b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc088f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f354e39b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc088f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44502/0 (socket says 172.21.15.7:44502) 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.075+0000 7f354e39b700 1 -- 172.21.15.7:0/1531257153 learned_addr learned my addr 172.21.15.7:0/1531257153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354eb9c700 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc08a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354e39b700 1 -- 172.21.15.7:0/1531257153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 msgr2=0x5639dc0899f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354e39b700 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc0899f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354e39b700 1 -- 172.21.15.7:0/1531257153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 msgr2=0x5639dc08a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354e39b700 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc08a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354e39b700 1 -- 172.21.15.7:0/1531257153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639dbe513b0 con 0x5639dc08ad80 2024-01-11T17:58:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.076+0000 7f354e39b700 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc088f90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3540009400 tx=0x7f3540009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea25b7927bdaca29 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.077+0000 7f353f7fe700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35340a2060 0x7f35340a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.077+0000 7f353cbfe700 1 -- 172.21.15.7:0/1531257153 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540018070 con 0x5639dc08ad80 2024-01-11T17:58:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.077+0000 7f353cbfe700 1 -- 172.21.15.7:0/1531257153 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f354000f050 con 0x5639dc08ad80 2024-01-11T17:58:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.077+0000 7f353cbfe700 1 -- 172.21.15.7:0/1531257153 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540005020 con 0x5639dc08ad80 2024-01-11T17:58:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.077+0000 7f353f7fe700 1 -- 172.21.15.7:0/2203130172 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3544014070 con 0x7f3548101680 2024-01-11T17:58:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.077+0000 7f354eb9c700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35340a2060 0x7f35340a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.079+0000 7f356423ab80 1 -- 172.21.15.7:0/1531257153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 msgr2=0x5639dc088f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.079+0000 7f356423ab80 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc088f90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3540009400 tx=0x7f3540009920 comp rx=0 tx=0).stop 2024-01-11T17:58:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.079+0000 7f354eb9c700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35340a2060 0x7f35340a4520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f353800b2e0 tx=0x7f353800b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.080+0000 7f356423ab80 1 -- 172.21.15.7:0/1531257153 shutdown_connections 2024-01-11T17:58:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.080+0000 7f356423ab80 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc0899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.080+0000 7f356423ab80 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc08a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.080+0000 7f356423ab80 1 --2- 172.21.15.7:0/1531257153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc088f90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.080+0000 7f356423ab80 1 -- 172.21.15.7:0/1531257153 >> 172.21.15.7:0/1531257153 conn(0x5639dbffe7a0 msgr2=0x5639dc083de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.081+0000 7f356423ab80 1 -- 172.21.15.7:0/1531257153 shutdown_connections 2024-01-11T17:58:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.081+0000 7f356423ab80 1 -- 172.21.15.7:0/1531257153 wait complete. 2024-01-11T17:58:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.082+0000 7f356423ab80 1 -- start start 2024-01-11T17:58:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.082+0000 7f356423ab80 1 Processor -- start 2024-01-11T17:58:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.083+0000 7f356423ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc092ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.083+0000 7f356423ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc093400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.083+0000 7f356423ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc09a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f356423ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbfdcd30 con 0x5639dc08a400 2024-01-11T17:58:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f356423ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbf4b3a0 con 0x5639dc089570 2024-01-11T17:58:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f356423ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbf4b040 con 0x5639dc08ad80 2024-01-11T17:58:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f354eb9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc09a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f354eb9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc09a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44516/0 (socket says 172.21.15.7:44516) 2024-01-11T17:58:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f354eb9c700 1 -- 172.21.15.7:0/1932651678 learned_addr learned my addr 172.21.15.7:0/1932651678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f354e39b700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc092ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.084+0000 7f356423ab80 10 client.0 _ll_register_callbacks cb 0x5639dbf7d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.085+0000 7f354eb9c700 1 -- 172.21.15.7:0/1932651678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 msgr2=0x5639dc092ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.085+0000 7f354eb9c700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc092ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.085+0000 7f354eb9c700 1 -- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 msgr2=0x5639dc093400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.085+0000 7f354eb9c700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc093400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.085+0000 7f354eb9c700 1 -- 172.21.15.7:0/1932651678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35400090b0 con 0x5639dc08ad80 2024-01-11T17:58:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.085+0000 7f354e39b700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc092ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.087+0000 7f354eb9c700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc09a440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3538015000 tx=0x7f3538009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0d35af7e4d10e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.087+0000 7f3531ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.087+0000 7f3531ffb700 1 -- 172.21.15.7:0/1932651678 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538004990 con 0x5639dc08ad80 2024-01-11T17:58:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.087+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639dc07c010 con 0x5639dc08ad80 2024-01-11T17:58:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.087+0000 7f3531ffb700 1 -- 172.21.15.7:0/1932651678 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3538004af0 con 0x5639dc08ad80 2024-01-11T17:58:40.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.088+0000 7f3531ffb700 1 -- 172.21.15.7:0/1932651678 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3538004cb0 con 0x5639dc08ad80 2024-01-11T17:58:40.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.088+0000 7f356423ab80 10 client.25654 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:40.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.088+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5639dbdfdb60 con 0x5639dc08ad80 2024-01-11T17:58:40.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f3531ffb700 1 -- 172.21.15.7:0/1932651678 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639dc07c010 con 0x5639dc08ad80 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f3531ffb700 1 -- 172.21.15.7:0/1932651678 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f353800f460 con 0x5639dc08ad80 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f356423ab80 10 client.25654 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f356423ab80 10 client.25654 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f356423ab80 20 client.25654 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f356423ab80 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc0afd40 0x5639dc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f356423ab80 4 client.25654 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.089+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639dbfd5bc0 con 0x5639dc0afd40 2024-01-11T17:58:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.090+0000 7f354e39b700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc0afd40 0x5639dc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.092+0000 7f3531ffb700 10 client.25654 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:58:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.093+0000 7f354e39b700 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc0afd40 0x5639dc0b2160 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aca3a8df26bbabbc server_cookie=25b63afb26dc2789 in_seq=0 out_seq=0 2024-01-11T17:58:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.095+0000 7f3531ffb700 1 -- 172.21.15.7:0/1932651678 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639dbfd5bc0 con 0x5639dc0afd40 2024-01-11T17:58:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.095+0000 7f3531ffb700 10 client.25654 handle_command_reply: tid=0 2024-01-11T17:58:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.095+0000 7f356423ab80 1 client.25654 shutdown 2024-01-11T17:58:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 msgr2=0x5639dc09a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc09a440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3538015000 tx=0x7f3538009310 comp rx=0 tx=0).stop 2024-01-11T17:58:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 shutdown_connections 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc0afd40 0x5639dc0b2160 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc09a440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc092ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f3531ffb700 0 client.25654 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 --2- 172.21.15.7:0/1932651678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc093400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.097+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 >> 172.21.15.7:0/1932651678 conn(0x5639dbffe7a0 msgr2=0x5639dc087e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.098+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 shutdown_connections 2024-01-11T17:58:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.098+0000 7f356423ab80 1 -- 172.21.15.7:0/1932651678 wait complete. 2024-01-11T17:58:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.098+0000 7f356423ab80 20 client.25654 trim_cache size 0 max 16384 2024-01-11T17:58:40.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 Processor -- start 2024-01-11T17:58:40.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 -- start start 2024-01-11T17:58:40.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc07bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc09a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc090e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dbe536a0 con 0x5639dc08a400 2024-01-11T17:58:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc09b5a0 con 0x5639dc089570 2024-01-11T17:58:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.126+0000 7f356423ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc084cb0 con 0x5639dc08ad80 2024-01-11T17:58:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.127+0000 7f354eb9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc090e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.127+0000 7f354eb9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc090e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44526/0 (socket says 172.21.15.7:44526) 2024-01-11T17:58:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.127+0000 7f354eb9c700 1 -- 172.21.15.7:0/194038659 learned_addr learned my addr 172.21.15.7:0/194038659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.127+0000 7f354db9a700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc09a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354eb9c700 1 -- 172.21.15.7:0/194038659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 msgr2=0x5639dc07bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.127+0000 7f354e39b700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc07bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354eb9c700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc07bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354eb9c700 1 -- 172.21.15.7:0/194038659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 msgr2=0x5639dc09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354eb9c700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc09a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354eb9c700 1 -- 172.21.15.7:0/194038659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639dbf9e230 con 0x5639dc08ad80 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354db9a700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc09a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.128+0000 7f354e39b700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc07bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.129+0000 7f354eb9c700 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc090e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f353800f810 tx=0x7f35380094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ede9a7c1594851 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.129+0000 7f3533fff700 1 -- 172.21.15.7:0/194038659 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800fa00 con 0x5639dc08ad80 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.129+0000 7f3533fff700 1 -- 172.21.15.7:0/194038659 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f353801f040 con 0x5639dc08ad80 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.129+0000 7f3533fff700 1 -- 172.21.15.7:0/194038659 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353803b7f0 con 0x5639dc08ad80 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 -- 172.21.15.7:0/194038659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 msgr2=0x5639dc090e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc090e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f353800f810 tx=0x7f35380094b0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 -- 172.21.15.7:0/194038659 shutdown_connections 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc090e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc089570 0x5639dc07bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 --2- 172.21.15.7:0/194038659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc08a400 0x5639dc09a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 -- 172.21.15.7:0/194038659 >> 172.21.15.7:0/194038659 conn(0x5639dbffe7a0 msgr2=0x5639dc083db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 -- 172.21.15.7:0/194038659 shutdown_connections 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.130+0000 7f356423ab80 1 -- 172.21.15.7:0/194038659 wait complete. 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.131+0000 7f356423ab80 1 -- start start 2024-01-11T17:58:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.131+0000 7f356423ab80 1 Processor -- start 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc089570 0x5639dc0975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 0x5639dc097b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc098ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc099020 con 0x5639dc089570 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc099190 con 0x5639dc08a400 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc099350 con 0x5639dc08ad80 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f356423ab80 10 client.0 _ll_register_callbacks cb 0x5639dbf7d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.132+0000 7f354db9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 0x5639dc097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 0x5639dc097b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48352/0 (socket says 172.21.15.7:48352) 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 -- 172.21.15.7:0/3908478219 learned_addr learned my addr 172.21.15.7:0/3908478219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354e39b700 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc089570 0x5639dc0975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 -- 172.21.15.7:0/3908478219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 msgr2=0x5639dc098ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 -- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc089570 msgr2=0x5639dc0975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc089570 0x5639dc0975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.133+0000 7f354db9a700 1 -- 172.21.15.7:0/3908478219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639dbdfdb60 con 0x5639dc08a400 2024-01-11T17:58:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.135+0000 7f354db9a700 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 0x5639dc097b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f354409a600 tx=0x7f35440ba390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615af876b17221dc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.135+0000 7f3531ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.135+0000 7f3531ffb700 1 -- 172.21.15.7:0/3908478219 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35440a2c90 con 0x5639dc08a400 2024-01-11T17:58:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.135+0000 7f3531ffb700 1 -- 172.21.15.7:0/3908478219 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3544098ca0 con 0x5639dc08a400 2024-01-11T17:58:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.135+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f354000f050 con 0x5639dc08a400 2024-01-11T17:58:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.136+0000 7f3531ffb700 1 -- 172.21.15.7:0/3908478219 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3544099df0 con 0x5639dc08a400 2024-01-11T17:58:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.136+0000 7f356423ab80 10 client.25622 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.136+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5639dbe55cd0 con 0x5639dc08a400 2024-01-11T17:58:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.136+0000 7f3531ffb700 1 -- 172.21.15.7:0/3908478219 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f354000f050 con 0x5639dc08a400 2024-01-11T17:58:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f3531ffb700 1 -- 172.21.15.7:0/3908478219 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f35440106e0 con 0x5639dc08a400 2024-01-11T17:58:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f356423ab80 10 client.25622 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f356423ab80 10 client.25622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-01-11T17:58:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f356423ab80 20 client.25622 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f356423ab80 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc1065c0 0x5639dc1089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f356423ab80 4 client.25622 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-01-11T17:58:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.137+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639dbfd58e0 con 0x5639dc1065c0 2024-01-11T17:58:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.138+0000 7f354e39b700 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc1065c0 0x5639dc1089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.139+0000 7f3531ffb700 10 client.25622 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:58:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.139+0000 7f354e39b700 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc1065c0 0x5639dc1089e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=accc4b04fd73fce2 server_cookie=adbc1948f9526980 in_seq=0 out_seq=0 2024-01-11T17:58:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.140+0000 7f3531ffb700 1 -- 172.21.15.7:0/3908478219 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639dbfd58e0 con 0x5639dc1065c0 2024-01-11T17:58:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.140+0000 7f3531ffb700 10 client.25622 handle_command_reply: tid=0 2024-01-11T17:58:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.140+0000 7f356423ab80 1 client.25622 shutdown 2024-01-11T17:58:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.140+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 msgr2=0x5639dc097b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.140+0000 7f356423ab80 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 0x5639dc097b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f354409a600 tx=0x7f35440ba390 comp rx=0 tx=0).stop 2024-01-11T17:58:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 shutdown_connections 2024-01-11T17:58:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f356423ab80 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5639dc1065c0 0x5639dc1089e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f356423ab80 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5639dc08ad80 0x5639dc098ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f356423ab80 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639dc089570 0x5639dc0975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f356423ab80 1 --2- 172.21.15.7:0/3908478219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639dc08a400 0x5639dc097b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 >> 172.21.15.7:0/3908478219 conn(0x5639dbffe7a0 msgr2=0x5639dc083d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.141+0000 7f3531ffb700 0 client.25622 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:58:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.142+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 shutdown_connections 2024-01-11T17:58:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.143+0000 7f356423ab80 1 -- 172.21.15.7:0/3908478219 wait complete. 2024-01-11T17:58:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.143+0000 7f356423ab80 20 client.25622 trim_cache size 0 max 16384 2024-01-11T17:58:40.146 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-01-11T17:58:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35340a2060 msgr2=0x7f35340a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35340a2060 0x7f35340a4520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f353800b2e0 tx=0x7f353800b750 comp rx=0 tx=0).stop 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 msgr2=0x7f354806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 0x7f354806c240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9d0 tx=0x7f354400ed90 comp rx=0 tx=0).stop 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 shutdown_connections 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f35340a2060 0x7f35340a4520 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3548102020 0x7f3548073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3548100490 0x7f354806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 --2- 172.21.15.7:0/2203130172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548101680 0x7f354806c240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 >> 172.21.15.7:0/2203130172 conn(0x7f35480fba00 msgr2=0x7f354810d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.144+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 shutdown_connections 2024-01-11T17:58:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.145+0000 7f35505ff700 1 -- 172.21.15.7:0/2203130172 wait complete. 2024-01-11T17:58:40.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-01-11T17:58:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.538+0000 7f0d1ebff700 1 -- 172.21.15.7:0/3698438806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1810fa80 msgr2=0x7f0d18111f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.538+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/3698438806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1810fa80 0x7f0d18111f40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0098f0 tx=0x7f0d0c009c00 comp rx=0 tx=0).stop 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.539+0000 7f0d1ebff700 1 -- 172.21.15.7:0/3698438806 shutdown_connections 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.539+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/3698438806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1810fa80 0x7f0d18111f40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.539+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/3698438806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d18073430 0x7f0d18108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.539+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/3698438806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d18108c00 0x7f0d18109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.539+0000 7f0d1ebff700 1 -- 172.21.15.7:0/3698438806 >> 172.21.15.7:0/3698438806 conn(0x7f0d1806e530 msgr2=0x7f0d18070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.540+0000 7f0d1ebff700 1 -- 172.21.15.7:0/3698438806 shutdown_connections 2024-01-11T17:58:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.540+0000 7f0d1ebff700 1 -- 172.21.15.7:0/3698438806 wait complete. 2024-01-11T17:58:40.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.541+0000 7f0d1ebff700 1 Processor -- start 2024-01-11T17:58:40.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.542+0000 7f0d1ebff700 1 -- start start 2024-01-11T17:58:40.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.543+0000 7f0d1ebff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 0x7f0d1819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.543+0000 7f0d1ebff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d18108c00 0x7f0d1819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.543+0000 7f0d1c99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 0x7f0d1819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.543+0000 7f0d1c99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 0x7f0d1819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48368/0 (socket says 172.21.15.7:48368) 2024-01-11T17:58:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.543+0000 7f0d1c99b700 1 -- 172.21.15.7:0/627417746 learned_addr learned my addr 172.21.15.7:0/627417746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.543+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1810fa80 0x7f0d181a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181a55d0 con 0x7f0d1810fa80 2024-01-11T17:58:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181a5710 con 0x7f0d18073430 2024-01-11T17:58:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181a58a0 con 0x7f0d18108c00 2024-01-11T17:58:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d17fff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d18108c00 0x7f0d1819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1c99b700 1 -- 172.21.15.7:0/627417746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d18108c00 msgr2=0x7f0d1819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d18108c00 0x7f0d1819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1c99b700 1 -- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1810fa80 msgr2=0x7f0d181a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1810fa80 0x7f0d181a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.544+0000 7f0d1c99b700 1 -- 172.21.15.7:0/627417746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10007600 con 0x7f0d18073430 2024-01-11T17:58:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.545+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 0x7f0d1819da50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800e970 tx=0x7f0d0800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.545+0000 7f0d15ffb700 1 -- 172.21.15.7:0/627417746 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08009790 con 0x7f0d18073430 2024-01-11T17:58:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.545+0000 7f0d15ffb700 1 -- 172.21.15.7:0/627417746 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d08004cf0 con 0x7f0d18073430 2024-01-11T17:58:40.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.545+0000 7f0d15ffb700 1 -- 172.21.15.7:0/627417746 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08005680 con 0x7f0d18073430 2024-01-11T17:58:40.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.547+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c009520 con 0x7f0d18073430 2024-01-11T17:58:40.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.547+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d181a5eb0 con 0x7f0d18073430 2024-01-11T17:58:40.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.560+0000 7f0d15ffb700 1 -- 172.21.15.7:0/627417746 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0d08004750 con 0x7f0d18073430 2024-01-11T17:58:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.562+0000 7f0d32995b80 1 Processor -- start 2024-01-11T17:58:40.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.563+0000 7f0d32995b80 1 -- start start 2024-01-11T17:58:40.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.563+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b47820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d32995b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42a3d090 con 0x557e42b47400 2024-01-11T17:58:40.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d32995b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42aa0b20 con 0x557e42b47d80 2024-01-11T17:58:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d32995b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42a9e440 con 0x557e42b46570 2024-01-11T17:58:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d1d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b47820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d1d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b47820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53234/0 (socket says 172.21.15.7:53234) 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.564+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2071154674 learned_addr learned my addr 172.21.15.7:0/2071154674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d17fff700 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2071154674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 msgr2=0x557e42b469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2071154674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 msgr2=0x557e42b45f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b45f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2071154674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e4290e3b0 con 0x557e42b47400 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.565+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.566+0000 7f0d15ffb700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d000a1f60 0x7f0d000a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.566+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b47820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005950 tx=0x7f0d0c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7cc81a0079e4a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.566+0000 7f0d15ffb700 1 -- 172.21.15.7:0/627417746 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0d08014070 con 0x7f0d18073430 2024-01-11T17:58:40.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.567+0000 7f0cfeffd700 1 -- 172.21.15.7:0/2071154674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c01c070 con 0x557e42b47400 2024-01-11T17:58:40.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.567+0000 7f0cfeffd700 1 -- 172.21.15.7:0/2071154674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0c00b560 con 0x557e42b47400 2024-01-11T17:58:40.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.567+0000 7f0cfeffd700 1 -- 172.21.15.7:0/2071154674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c005400 con 0x557e42b47400 2024-01-11T17:58:40.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.567+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d000a1f60 0x7f0d000a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.568+0000 7f0d32995b80 1 -- 172.21.15.7:0/2071154674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 msgr2=0x557e42b47820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.568+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b47820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005950 tx=0x7f0d0c00bef0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 -- 172.21.15.7:0/2071154674 shutdown_connections 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d1c99b700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d000a1f60 0x7f0d000a4420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d0809a720 tx=0x7f0d08005120 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b47820 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2071154674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b45f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 -- 172.21.15.7:0/2071154674 >> 172.21.15.7:0/2071154674 conn(0x557e42abb7a0 msgr2=0x557e42b40de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 -- 172.21.15.7:0/2071154674 shutdown_connections 2024-01-11T17:58:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.569+0000 7f0d32995b80 1 -- 172.21.15.7:0/2071154674 wait complete. 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.570+0000 7f0d32995b80 1 -- start start 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.570+0000 7f0d32995b80 1 Processor -- start 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.571+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.571+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d1d19c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d1d19c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b4fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44564/0 (socket says 172.21.15.7:44564) 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2544863690 learned_addr learned my addr 172.21.15.7:0/2544863690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b57400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42a99d30 con 0x557e42b47400 2024-01-11T17:58:40.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42a083a0 con 0x557e42b47d80 2024-01-11T17:58:40.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42a08040 con 0x557e42b46570 2024-01-11T17:58:40.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.572+0000 7f0d32995b80 10 client.0 _ll_register_callbacks cb 0x557e42a3a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b57400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 msgr2=0x557e42b57400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b57400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2544863690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 msgr2=0x557e42b503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d18112b60 con 0x557e42b46570 2024-01-11T17:58:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b4fe80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00fb70 tx=0x7f0d0c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea967990cfb7d808 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.573+0000 7f0cf7fff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.574+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2544863690 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c01c070 con 0x557e42b46570 2024-01-11T17:58:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.574+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2544863690 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0d0c005680 con 0x557e42b46570 2024-01-11T17:58:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.574+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e42b39010 con 0x557e42b46570 2024-01-11T17:58:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.575+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2544863690 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d0c003680 con 0x557e42b46570 2024-01-11T17:58:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.575+0000 7f0d32995b80 10 client.25666 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.575+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557e428bab60 con 0x557e42b46570 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.575+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2544863690 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e42b39010 con 0x557e42b46570 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2544863690 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0d0c015020 con 0x557e42b46570 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d32995b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d32995b80 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d32995b80 20 client.25666 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42b5b700 0x557e42b5db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d32995b80 4 client.25666 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e42a92bc0 con 0x557e42b5b700 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.576+0000 7f0d17fff700 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42b5b700 0x557e42b5db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.577+0000 7f0d17fff700 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42b5b700 0x557e42b5db20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7d4f3ea7c943c2 server_cookie=66e569e7a383c535 in_seq=0 out_seq=0 2024-01-11T17:58:40.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.577+0000 7f0cf7fff700 10 client.25666 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.578+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2544863690 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e42a92bc0 con 0x557e42b5b700 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.578+0000 7f0cf7fff700 10 client.25666 handle_command_reply: tid=0 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.578+0000 7f0d32995b80 1 client.25666 shutdown 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 msgr2=0x557e42b4fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b4fe80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00fb70 tx=0x7f0d0c005b00 comp rx=0 tx=0).stop 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 shutdown_connections 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42b5b700 0x557e42b5db20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b4fe80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b57400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2544863690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47400 0x557e42b503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 >> 172.21.15.7:0/2544863690 conn(0x557e42abb7a0 msgr2=0x557e42b44e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0cf7fff700 0 client.25666 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:40.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 shutdown_connections 2024-01-11T17:58:40.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 1 -- 172.21.15.7:0/2544863690 wait complete. 2024-01-11T17:58:40.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.579+0000 7f0d32995b80 20 client.25666 trim_cache size 0 max 16384 2024-01-11T17:58:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.606+0000 7f0d32995b80 1 Processor -- start 2024-01-11T17:58:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.607+0000 7f0d32995b80 1 -- start start 2024-01-11T17:58:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b46570 0x557e42b45e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 0x557e42b573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b46950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d32995b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e429106a0 con 0x557e42b46570 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d32995b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42a92bc0 con 0x557e42b47d80 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d32995b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42b58580 con 0x557e42b47400 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 0x557e42b573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.608+0000 7f0d1d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b46570 0x557e42b45e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 0x557e42b573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44574/0 (socket says 172.21.15.7:44574) 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 -- 172.21.15.7:0/2390648849 learned_addr learned my addr 172.21.15.7:0/2390648849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b46950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 -- 172.21.15.7:0/2390648849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 msgr2=0x557e42b46950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b46950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 -- 172.21.15.7:0/2390648849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b46570 msgr2=0x557e42b45e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b46570 0x557e42b45e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d17fff700 1 -- 172.21.15.7:0/2390648849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e42a5b230 con 0x557e42b47400 2024-01-11T17:58:40.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.609+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b46950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:58:40.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.610+0000 7f0d17fff700 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 0x557e42b573c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x557e42b5e390 tx=0x7f0d10004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d150a905c7d254ee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.610+0000 7f0cfe7fc700 1 -- 172.21.15.7:0/2390648849 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1001c040 con 0x557e42b47400 2024-01-11T17:58:40.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.611+0000 7f0cfe7fc700 1 -- 172.21.15.7:0/2390648849 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d100228d0 con 0x557e42b47400 2024-01-11T17:58:40.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.611+0000 7f0cfe7fc700 1 -- 172.21.15.7:0/2390648849 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100193c0 con 0x557e42b47400 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 -- 172.21.15.7:0/2390648849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 msgr2=0x557e42b573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 0x557e42b573c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x557e42b5e390 tx=0x7f0d10004ad0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 -- 172.21.15.7:0/2390648849 shutdown_connections 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47d80 0x557e42b46950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b46570 0x557e42b45e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 --2- 172.21.15.7:0/2390648849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b47400 0x557e42b573c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.613+0000 7f0d32995b80 1 -- 172.21.15.7:0/2390648849 >> 172.21.15.7:0/2390648849 conn(0x557e42abb7a0 msgr2=0x557e42b4f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.614+0000 7f0d32995b80 1 -- 172.21.15.7:0/2390648849 shutdown_connections 2024-01-11T17:58:40.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.614+0000 7f0d32995b80 1 -- 172.21.15.7:0/2390648849 wait complete. 2024-01-11T17:58:40.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.614+0000 7f0d32995b80 1 -- start start 2024-01-11T17:58:40.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.615+0000 7f0d32995b80 1 Processor -- start 2024-01-11T17:58:40.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 0x557e42b55370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47d80 0x557e42b56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42b56870 con 0x557e42b47d80 2024-01-11T17:58:40.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42b569e0 con 0x557e42b47400 2024-01-11T17:58:40.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e42b56ba0 con 0x557e42b46570 2024-01-11T17:58:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d32995b80 10 client.0 _ll_register_callbacks cb 0x557e42a3a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.616+0000 7f0d1d19c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 0x557e42b55370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 0x557e42b55370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48402/0 (socket says 172.21.15.7:48402) 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 -- 172.21.15.7:0/1008781531 learned_addr learned my addr 172.21.15.7:0/1008781531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47d80 0x557e42b56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 -- 172.21.15.7:0/1008781531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 msgr2=0x557e42b54e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b54e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 -- 172.21.15.7:0/1008781531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47d80 msgr2=0x557e42b56330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47d80 0x557e42b56330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.617+0000 7f0d17fff700 1 -- 172.21.15.7:0/1008781531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d10003d60 con 0x557e42b47400 2024-01-11T17:58:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.618+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 0x557e42b55370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000de30 tx=0x7f0d10019f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0e4918a98c0f75 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.618+0000 7f0cf7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.618+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1008781531 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1001c040 con 0x557e42b47400 2024-01-11T17:58:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.618+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1008781531 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0d10017a60 con 0x557e42b47400 2024-01-11T17:58:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.618+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d0c00b560 con 0x557e42b47400 2024-01-11T17:58:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.619+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1008781531 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d10017c60 con 0x557e42b47400 2024-01-11T17:58:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.619+0000 7f0d32995b80 10 client.25637 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.619+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557e42912cd0 con 0x557e42b47400 2024-01-11T17:58:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.619+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1008781531 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0c00b560 con 0x557e42b47400 2024-01-11T17:58:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.619+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1008781531 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0d1000b130 con 0x557e42b47400 2024-01-11T17:58:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d32995b80 10 client.25637 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d32995b80 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d32995b80 20 client.25637 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d32995b80 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42bb27f0 0x557e42bb4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d32995b80 4 client.25637 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-01-11T17:58:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e42a928e0 con 0x557e42bb27f0 2024-01-11T17:58:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.620+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42bb27f0 0x557e42bb4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.621+0000 7f0d1d19c700 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42bb27f0 0x557e42bb4c10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb9533f172ca48f server_cookie=5fa5b5b771e4d640 in_seq=0 out_seq=0 2024-01-11T17:58:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.621+0000 7f0cf7fff700 10 client.25637 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.622+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1008781531 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e42a928e0 con 0x557e42bb27f0 2024-01-11T17:58:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.622+0000 7f0cf7fff700 10 client.25637 handle_command_reply: tid=0 2024-01-11T17:58:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.622+0000 7f0d32995b80 1 client.25637 shutdown 2024-01-11T17:58:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.623+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 msgr2=0x557e42b55370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.623+0000 7f0d32995b80 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 0x557e42b55370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000de30 tx=0x7f0d10019f10 comp rx=0 tx=0).stop 2024-01-11T17:58:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.623+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 shutdown_connections 2024-01-11T17:58:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.623+0000 7f0d32995b80 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557e42bb27f0 0x557e42bb4c10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.624+0000 7f0d32995b80 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e42b46570 0x557e42b54e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.624+0000 7f0d32995b80 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e42b47d80 0x557e42b56330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.624+0000 7f0d32995b80 1 --2- 172.21.15.7:0/1008781531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e42b47400 0x557e42b55370 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.624+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 >> 172.21.15.7:0/1008781531 conn(0x557e42abb7a0 msgr2=0x557e42b4ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.624+0000 7f0cf7fff700 0 client.25637 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.625+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 shutdown_connections 2024-01-11T17:58:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.625+0000 7f0d32995b80 1 -- 172.21.15.7:0/1008781531 wait complete. 2024-01-11T17:58:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.625+0000 7f0d32995b80 20 client.25637 trim_cache size 0 max 16384 2024-01-11T17:58:40.626 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d000a1f60 msgr2=0x7f0d000a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d000a1f60 0x7f0d000a4420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d0809a720 tx=0x7f0d08005120 comp rx=0 tx=0).stop 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 msgr2=0x7f0d1819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 0x7f0d1819da50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800e970 tx=0x7f0d0800ed30 comp rx=0 tx=0).stop 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 shutdown_connections 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0d000a1f60 0x7f0d000a4420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1810fa80 0x7f0d181a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d18073430 0x7f0d1819da50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 --2- 172.21.15.7:0/627417746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d18108c00 0x7f0d1819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 >> 172.21.15.7:0/627417746 conn(0x7f0d1806e530 msgr2=0x7f0d1810b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 shutdown_connections 2024-01-11T17:58:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:40.626+0000 7f0d1ebff700 1 -- 172.21.15.7:0/627417746 wait complete. 2024-01-11T17:58:41.662 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-01-11T17:58:41.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-01-11T17:58:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:41 smithi016 ceph-mon[43195]: pgmap v326: 129 pgs: 129 active+clean; 1.0 GiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 45 MiB/s wr, 360 op/s 2024-01-11T17:58:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.019+0000 7f21321ff700 1 -- 172.21.15.7:0/2140660709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c108610 msgr2=0x7f212c10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.019+0000 7f21321ff700 1 --2- 172.21.15.7:0/2140660709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c108610 0x7f212c10aa40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f211c009990 tx=0x7f211c009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.021+0000 7f21321ff700 1 -- 172.21.15.7:0/2140660709 shutdown_connections 2024-01-11T17:58:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.021+0000 7f21321ff700 1 --2- 172.21.15.7:0/2140660709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c108610 0x7f212c10aa40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.021+0000 7f21321ff700 1 --2- 172.21.15.7:0/2140660709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10d950 0x7f212c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.021+0000 7f21321ff700 1 --2- 172.21.15.7:0/2140660709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c10af80 0x7f212c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.021+0000 7f21321ff700 1 -- 172.21.15.7:0/2140660709 >> 172.21.15.7:0/2140660709 conn(0x7f212c06e530 msgr2=0x7f212c070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.025+0000 7f21321ff700 1 -- 172.21.15.7:0/2140660709 shutdown_connections 2024-01-11T17:58:42.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:41 smithi038 ceph-mon[43582]: pgmap v326: 129 pgs: 129 active+clean; 1.0 GiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 45 MiB/s wr, 360 op/s 2024-01-11T17:58:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.026+0000 7f21321ff700 1 -- 172.21.15.7:0/2140660709 wait complete. 2024-01-11T17:58:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.029+0000 7f21321ff700 1 Processor -- start 2024-01-11T17:58:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.030+0000 7f21321ff700 1 -- start start 2024-01-11T17:58:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.032+0000 7f21321ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 0x7f212c114b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f21321ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10af80 0x7f212c115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f21321ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c10d950 0x7f212c1b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f21321ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c1180d0 con 0x7f212c10af80 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 0x7f212c114b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f21321ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c118240 con 0x7f212c108610 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 0x7f212c114b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51722/0 (socket says 172.21.15.7:51722) 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f212b7fe700 1 -- 172.21.15.7:0/3151168762 learned_addr learned my addr 172.21.15.7:0/3151168762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.033+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c118400 con 0x7f212c10d950 2024-01-11T17:58:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.034+0000 7f212b7fe700 1 -- 172.21.15.7:0/3151168762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c10d950 msgr2=0x7f212c1b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.034+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c10d950 0x7f212c1b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.034+0000 7f212b7fe700 1 -- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10af80 msgr2=0x7f212c115060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.034+0000 7f212affd700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10af80 0x7f212c115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.035+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10af80 0x7f212c115060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.035+0000 7f212b7fe700 1 -- 172.21.15.7:0/3151168762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c0095f0 con 0x7f212c108610 2024-01-11T17:58:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.035+0000 7f212affd700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10af80 0x7f212c115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.035+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 0x7f212c114b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f211c0037f0 tx=0x7f211c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.036+0000 7f2128ff9700 1 -- 172.21.15.7:0/3151168762 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c01d070 con 0x7f212c108610 2024-01-11T17:58:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.037+0000 7f2128ff9700 1 -- 172.21.15.7:0/3151168762 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c00fcc0 con 0x7f212c108610 2024-01-11T17:58:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.037+0000 7f2128ff9700 1 -- 172.21.15.7:0/3151168762 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c017650 con 0x7f212c108610 2024-01-11T17:58:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.038+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212c1b6b30 con 0x7f212c108610 2024-01-11T17:58:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.038+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c1b6ea0 con 0x7f212c108610 2024-01-11T17:58:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.047+0000 7f2128ff9700 1 -- 172.21.15.7:0/3151168762 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f211c022470 con 0x7f212c108610 2024-01-11T17:58:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.052+0000 7f2145f3bb80 1 Processor -- start 2024-01-11T17:58:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.052+0000 7f2128ff9700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21140a2030 0x7f21140a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.053+0000 7f2145f3bb80 1 -- start start 2024-01-11T17:58:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.053+0000 7f2128ff9700 1 -- 172.21.15.7:0/3151168762 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f211c0cf3a0 con 0x7f212c108610 2024-01-11T17:58:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.054+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.054+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af66010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.054+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af665f0 0x55ca4af66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.054+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4ae5d110 con 0x55ca4af665f0 2024-01-11T17:58:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.054+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4aec0ba0 con 0x55ca4af67e00 2024-01-11T17:58:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.054+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4aebe4c0 con 0x55ca4af67480 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.055+0000 7f212bfff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21140a2030 0x7f21140a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.055+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.055+0000 7f212affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af66010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.055+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af678a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54672/0 (socket says 172.21.15.7:54672) 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212b7fe700 1 -- 172.21.15.7:0/845045620 learned_addr learned my addr 172.21.15.7:0/845045620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212bfff700 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af665f0 0x55ca4af66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212b7fe700 1 -- 172.21.15.7:0/845045620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 msgr2=0x55ca4af66010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212b7fe700 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af66010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212b7fe700 1 -- 172.21.15.7:0/845045620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af665f0 msgr2=0x55ca4af66a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212b7fe700 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af665f0 0x55ca4af66a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212b7fe700 1 -- 172.21.15.7:0/845045620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4ad2e430 con 0x55ca4af67480 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212bfff700 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af665f0 0x55ca4af66a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.056+0000 7f212affd700 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af66010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.057+0000 7f212b7fe700 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af678a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f211c098300 tx=0x7f211c022cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c19746e55f050442 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.057+0000 7f2111ffb700 1 -- 172.21.15.7:0/845045620 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c09e020 con 0x55ca4af67480 2024-01-11T17:58:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.057+0000 7f2111ffb700 1 -- 172.21.15.7:0/845045620 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c0a2070 con 0x55ca4af67480 2024-01-11T17:58:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.057+0000 7f2111ffb700 1 -- 172.21.15.7:0/845045620 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c09b2f0 con 0x55ca4af67480 2024-01-11T17:58:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.059+0000 7f2145f3bb80 1 -- 172.21.15.7:0/845045620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 msgr2=0x55ca4af678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.059+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af678a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f211c098300 tx=0x7f211c022cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f212bfff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21140a2030 0x7f21140a44f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2124009bd0 tx=0x7f21240091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 -- 172.21.15.7:0/845045620 shutdown_connections 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af678a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af665f0 0x55ca4af66a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/845045620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af66010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 -- 172.21.15.7:0/845045620 >> 172.21.15.7:0/845045620 conn(0x55ca4aedb820 msgr2=0x55ca4af60e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 -- 172.21.15.7:0/845045620 shutdown_connections 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.060+0000 7f2145f3bb80 1 -- 172.21.15.7:0/845045620 wait complete. 2024-01-11T17:58:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.061+0000 7f2145f3bb80 1 -- start start 2024-01-11T17:58:42.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.061+0000 7f2145f3bb80 1 Processor -- start 2024-01-11T17:58:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.064+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af6fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.065+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.065+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af77230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.065+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4aeb9db0 con 0x55ca4af67e00 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.065+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4ae28420 con 0x55ca4af665f0 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.065+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4ae280c0 con 0x55ca4af67480 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.065+0000 7f2145f3bb80 10 client.0 _ll_register_callbacks cb 0x55ca4ae5a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.066+0000 7f212affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.066+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af6fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.066+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af6fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51734/0 (socket says 172.21.15.7:51734) 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.066+0000 7f212b7fe700 1 -- 172.21.15.7:0/3120789675 learned_addr learned my addr 172.21.15.7:0/3120789675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.066+0000 7f212bfff700 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af77230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.067+0000 7f212b7fe700 1 -- 172.21.15.7:0/3120789675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 msgr2=0x55ca4af701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.067+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.067+0000 7f212b7fe700 1 -- 172.21.15.7:0/3120789675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 msgr2=0x55ca4af77230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.067+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af77230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.067+0000 7f212b7fe700 1 -- 172.21.15.7:0/3120789675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c1b6ea0 con 0x55ca4af665f0 2024-01-11T17:58:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.069+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af6fcb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f211c004800 tx=0x7f211c00fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c72204b5f85c038 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.069+0000 7f210affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.069+0000 7f210affd700 1 -- 172.21.15.7:0/3120789675 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c09e070 con 0x55ca4af665f0 2024-01-11T17:58:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.070+0000 7f210affd700 1 -- 172.21.15.7:0/3120789675 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f211c09ff50 con 0x55ca4af665f0 2024-01-11T17:58:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.070+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca4af59090 con 0x55ca4af665f0 2024-01-11T17:58:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.070+0000 7f210affd700 1 -- 172.21.15.7:0/3120789675 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f211c0a3e80 con 0x55ca4af665f0 2024-01-11T17:58:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.070+0000 7f2145f3bb80 10 client.25646 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.070+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ca4acdabe0 con 0x55ca4af665f0 2024-01-11T17:58:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:41 smithi007 ceph-mon[33361]: pgmap v326: 129 pgs: 129 active+clean; 1.0 GiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 45 MiB/s wr, 360 op/s 2024-01-11T17:58:42.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.072+0000 7f210affd700 1 -- 172.21.15.7:0/3120789675 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca4af59090 con 0x55ca4af665f0 2024-01-11T17:58:42.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.072+0000 7f210affd700 1 -- 172.21.15.7:0/3120789675 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f211c09ad60 con 0x55ca4af665f0 2024-01-11T17:58:42.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.072+0000 7f2145f3bb80 10 client.25646 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:42.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.072+0000 7f2145f3bb80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.072+0000 7f2145f3bb80 20 client.25646 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.073+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4af7b5f0 0x55ca4af7da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.073+0000 7f2145f3bb80 4 client.25646 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.073+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca4aeb2c40 con 0x55ca4af7b5f0 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.073+0000 7f212affd700 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4af7b5f0 0x55ca4af7da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.074+0000 7f212affd700 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4af7b5f0 0x55ca4af7da10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c0ddfb3b5aa4d3 server_cookie=a96fb496a3b5f1fe in_seq=0 out_seq=0 2024-01-11T17:58:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.074+0000 7f210affd700 10 client.25646 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.075+0000 7f210affd700 1 -- 172.21.15.7:0/3120789675 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca4aeb2c40 con 0x55ca4af7b5f0 2024-01-11T17:58:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.075+0000 7f210affd700 10 client.25646 handle_command_reply: tid=0 2024-01-11T17:58:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.075+0000 7f2145f3bb80 1 client.25646 shutdown 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.075+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 msgr2=0x55ca4af6fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.075+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af6fcb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f211c004800 tx=0x7f211c00fe30 comp rx=0 tx=0).stop 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 shutdown_connections 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4af7b5f0 0x55ca4af7da10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af77230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af6fcb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3120789675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 >> 172.21.15.7:0/3120789675 conn(0x55ca4aedb820 msgr2=0x55ca4af64ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f210affd700 0 client.25646 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 shutdown_connections 2024-01-11T17:58:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3120789675 wait complete. 2024-01-11T17:58:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.076+0000 7f2145f3bb80 20 client.25646 trim_cache size 0 max 16384 2024-01-11T17:58:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.112+0000 7f2145f3bb80 1 Processor -- start 2024-01-11T17:58:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.112+0000 7f2145f3bb80 1 -- start start 2024-01-11T17:58:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.112+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 0x55ca4af65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.112+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67480 0x55ca4af771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4ad30720 con 0x55ca4af67480 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 0x55ca4af65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4aebe4c0 con 0x55ca4af67e00 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f2145f3bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4af70bb0 con 0x55ca4af665f0 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 0x55ca4af65ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54700/0 (socket says 172.21.15.7:54700) 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f212b7fe700 1 -- 172.21.15.7:0/2894112386 learned_addr learned my addr 172.21.15.7:0/2894112386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.113+0000 7f212bfff700 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212affd700 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67480 0x55ca4af771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212b7fe700 1 -- 172.21.15.7:0/2894112386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 msgr2=0x55ca4af6fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212b7fe700 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af6fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212b7fe700 1 -- 172.21.15.7:0/2894112386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67480 msgr2=0x55ca4af771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212b7fe700 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67480 0x55ca4af771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212b7fe700 1 -- 172.21.15.7:0/2894112386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4af66ca0 con 0x55ca4af665f0 2024-01-11T17:58:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.114+0000 7f212bfff700 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.115+0000 7f212b7fe700 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 0x55ca4af65ee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f211c09c460 tx=0x7f211c022cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa026fcc63f2982 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.115+0000 7f21117fa700 1 -- 172.21.15.7:0/2894112386 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c09e070 con 0x55ca4af665f0 2024-01-11T17:58:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.115+0000 7f21117fa700 1 -- 172.21.15.7:0/2894112386 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c09ff50 con 0x55ca4af665f0 2024-01-11T17:58:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.115+0000 7f21117fa700 1 -- 172.21.15.7:0/2894112386 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c022b40 con 0x55ca4af665f0 2024-01-11T17:58:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.116+0000 7f2145f3bb80 1 -- 172.21.15.7:0/2894112386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 msgr2=0x55ca4af65ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.116+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 0x55ca4af65ee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f211c09c460 tx=0x7f211c022cd0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.116+0000 7f2145f3bb80 1 -- 172.21.15.7:0/2894112386 shutdown_connections 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.117+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af665f0 0x55ca4af65ee0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.117+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af67e00 0x55ca4af6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.117+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/2894112386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67480 0x55ca4af771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.117+0000 7f2145f3bb80 1 -- 172.21.15.7:0/2894112386 >> 172.21.15.7:0/2894112386 conn(0x55ca4aedb820 msgr2=0x55ca4af6de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.117+0000 7f2145f3bb80 1 -- 172.21.15.7:0/2894112386 shutdown_connections 2024-01-11T17:58:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.117+0000 7f2145f3bb80 1 -- 172.21.15.7:0/2894112386 wait complete. 2024-01-11T17:58:42.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.118+0000 7f2145f3bb80 1 -- start start 2024-01-11T17:58:42.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.118+0000 7f2145f3bb80 1 Processor -- start 2024-01-11T17:58:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.120+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af77090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.121+0000 7f2145f3bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.121+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af77090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.121+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af77090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51754/0 (socket says 172.21.15.7:51754) 2024-01-11T17:58:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.121+0000 7f212b7fe700 1 -- 172.21.15.7:0/3490413424 learned_addr learned my addr 172.21.15.7:0/3490413424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.121+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.121+0000 7f212affd700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.122+0000 7f212bfff700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.122+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4af78eb0 con 0x55ca4af67e00 2024-01-11T17:58:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.122+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4af76340 con 0x55ca4af665f0 2024-01-11T17:58:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.122+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4af76500 con 0x55ca4af67480 2024-01-11T17:58:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.122+0000 7f2145f3bb80 10 client.0 _ll_register_callbacks cb 0x55ca4ae5a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212b7fe700 1 -- 172.21.15.7:0/3490413424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 msgr2=0x55ca4af756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af756f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212b7fe700 1 -- 172.21.15.7:0/3490413424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 msgr2=0x55ca4af75c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af75c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212b7fe700 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212400e8a0 con 0x55ca4af665f0 2024-01-11T17:58:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212bfff700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:58:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f212b7fe700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af77090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f211c098300 tx=0x7f211c00fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b98a5ef970a2556 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.123+0000 7f210affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.124+0000 7f210affd700 1 -- 172.21.15.7:0/3490413424 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c09e070 con 0x55ca4af665f0 2024-01-11T17:58:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.124+0000 7f210affd700 1 -- 172.21.15.7:0/3490413424 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f211c0a4100 con 0x55ca4af665f0 2024-01-11T17:58:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.124+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f211c0a2070 con 0x55ca4af665f0 2024-01-11T17:58:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.125+0000 7f210affd700 1 -- 172.21.15.7:0/3490413424 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f211c0afb50 con 0x55ca4af665f0 2024-01-11T17:58:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.125+0000 7f2145f3bb80 10 client.25652 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.125+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ca4acdabe0 con 0x55ca4af665f0 2024-01-11T17:58:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.125+0000 7f210affd700 1 -- 172.21.15.7:0/3490413424 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c0a2070 con 0x55ca4af665f0 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f210affd700 1 -- 172.21.15.7:0/3490413424 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f211c081070 con 0x55ca4af665f0 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f2145f3bb80 10 client.25652 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f2145f3bb80 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f2145f3bb80 20 client.25652 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4afd2cb0 0x55ca4afd50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f2145f3bb80 4 client.25652 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-01-11T17:58:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.126+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ca4aeb2960 con 0x55ca4afd2cb0 2024-01-11T17:58:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.127+0000 7f212affd700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4afd2cb0 0x55ca4afd50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.128+0000 7f210affd700 10 client.25652 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.128+0000 7f212affd700 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4afd2cb0 0x55ca4afd50a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c981eed8bccb3344 server_cookie=37804eeb337a0ece in_seq=0 out_seq=0 2024-01-11T17:58:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.129+0000 7f210affd700 1 -- 172.21.15.7:0/3490413424 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ca4aeb2960 con 0x55ca4afd2cb0 2024-01-11T17:58:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.129+0000 7f210affd700 10 client.25652 handle_command_reply: tid=0 2024-01-11T17:58:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.129+0000 7f2145f3bb80 1 client.25652 shutdown 2024-01-11T17:58:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.129+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 msgr2=0x55ca4af77090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.129+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af77090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f211c098300 tx=0x7f211c00fe30 comp rx=0 tx=0).stop 2024-01-11T17:58:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 shutdown_connections 2024-01-11T17:58:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ca4afd2cb0 0x55ca4afd50a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca4af67e00 0x55ca4af75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca4af665f0 0x55ca4af77090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 --2- 172.21.15.7:0/3490413424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ca4af67480 0x55ca4af756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 >> 172.21.15.7:0/3490413424 conn(0x55ca4aedb820 msgr2=0x55ca4af6fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f210affd700 0 client.25652 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 shutdown_connections 2024-01-11T17:58:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.130+0000 7f2145f3bb80 1 -- 172.21.15.7:0/3490413424 wait complete. 2024-01-11T17:58:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.131+0000 7f2145f3bb80 20 client.25652 trim_cache size 0 max 16384 2024-01-11T17:58:42.132 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:58:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-01-11T17:58:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "91b694ce-99ef-4fe7-adda-7010f038c107", 2024-01-11T17:58:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-01-11T17:58:42.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:58:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.132+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21140a2030 msgr2=0x7f21140a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.132+0000 7f21321ff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21140a2030 0x7f21140a44f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2124009bd0 tx=0x7f21240091c0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.132+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 msgr2=0x7f212c114b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.132+0000 7f21321ff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 0x7f212c114b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f211c0037f0 tx=0x7f211c00f790 comp rx=0 tx=0).stop 2024-01-11T17:58:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 shutdown_connections 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f21140a2030 0x7f21140a44f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f212c10d950 0x7f212c1b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f212c108610 0x7f212c114b20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 --2- 172.21.15.7:0/3151168762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212c10af80 0x7f212c115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 >> 172.21.15.7:0/3151168762 conn(0x7f212c06e530 msgr2=0x7f212c10d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 shutdown_connections 2024-01-11T17:58:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.133+0000 7f21321ff700 1 -- 172.21.15.7:0/3151168762 wait complete. 2024-01-11T17:58:42.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-01-11T17:58:42.532 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.173+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000270 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3575 auth v916 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000270:[<0x10000000323/3575 v916>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000270:[<0x10000000323/3575 v916>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.173+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000272 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_fsm auth v920 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000272:[<0x10000000323/2617_fsm v920>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000272:[<0x10000000323/2617_fsm v920>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.174+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000273 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3119 auth v922 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000273:[<0x10000000323/3119 v922>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000273:[<0x10000000323/3119 v922>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.174+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3603_vm auth v914 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026f:[<0x10000000323/3603_vm v914>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000026f:[<0x10000000323/3603_vm v914>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.174+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000276 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_fsm auth v928 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d28000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000276:[<0x10000000323/2603_fsm v928>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000276:[<0x10000000323/2603_fsm v928>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.175+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000274 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2674 auth v924 ap=1 s=385024 n(v0 b385024 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000274:[<0x10000000323/2674 v924>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000274:[<0x10000000323/2674 v924>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.175+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000275 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2668 auth v926 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000275:[<0x10000000323/2668 v926>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000275:[<0x10000000323/2668 v926>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.175+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000278 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2702 auth v932 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d28b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000278:[<0x10000000323/2702 v932>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000278:[<0x10000000323/2702 v932>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.175+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000277 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6110 auth v930 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d28580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000277:[<0x10000000323/6110 v930>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000277:[<0x10000000323/6110 v930>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.176+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000279 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2655 auth v934 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d29080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000279:[<0x10000000323/2655 v934>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000279:[<0x10000000323/2655 v934>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.176+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000271 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3467 auth v918 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000271:[<0x10000000323/3467 v918>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000271:[<0x10000000323/3467 v918>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.533 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.176+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3712 auth v938 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d29b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027b:[<0x10000000323/3712 v938>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000027b:[<0x10000000323/3712 v938>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.177+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608 auth v940 ap=1 s=442368 n(v0 rc2024-01-11T17:58:14.161770+0000 b442368 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d2a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027c:[<0x10000000323/2608 v940>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000027c:[<0x10000000323/2608 v940>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.180+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3597 auth v936 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d29600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027a:[<0x10000000323/3597 v936>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000027a:[<0x10000000323/3597 v936>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.181+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187_vm auth v944 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d2ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027e:[<0x10000000323/12187_vm v944>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000027e:[<0x10000000323/12187_vm v944>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.182+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2831 auth v942 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d2a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027d:[<0x10000000323/2831 v942>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000027d:[<0x10000000323/2831 v942>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.182+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3164 auth v946 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d2b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027f:[<0x10000000323/3164 v946>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000027f:[<0x10000000323/3164 v946>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.182+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000280 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3501_vm auth v582 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d2b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000280:[<0x10000000323/3501_vm v582>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000280:[<0x10000000323/3501_vm v582>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.182+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000281 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6111 auth v948 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.162770+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d36000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000281:[<0x10000000323/6111 v948>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000281:[<0x10000000323/6111 v948>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.183+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000282 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2682 auth v950 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d36580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000282:[<0x10000000323/2682 v950>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000282:[<0x10000000323/2682 v950>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.183+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000284 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3604 auth v954 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d37080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000284:[<0x10000000323/3604 v954>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000284:[<0x10000000323/3604 v954>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.183+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000283 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/113 auth v952 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d36b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000283:[<0x10000000323/113 v952>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000283:[<0x10000000323/113 v952>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.184+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000285 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2691 auth v956 ap=1 s=270336 n(v0 b270336 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d37600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000285:[<0x10000000323/2691 v956>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000285:[<0x10000000323/2691 v956>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.184+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000286 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605_vm auth v958 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d37b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000286:[<0x10000000323/2605_vm v958>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000286:[<0x10000000323/2605_vm v958>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.534 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.184+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2836 auth v604 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d39700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000028b:[<0x10000000323/2836 v604>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000028b:[<0x10000000323/2836 v604>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.185+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/548 auth v968 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d42000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000028c:[<0x10000000323/548 v968>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000028c:[<0x10000000323/548 v968>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.185+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000287 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3440 auth v960 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d38100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000287:[<0x10000000323/3440 v960>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000287:[<0x10000000323/3440 v960>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.185+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000288 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2685 auth v962 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d38680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000288:[<0x10000000323/2685 v962>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000288:[<0x10000000323/2685 v962>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.186+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000289 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2663 auth v964 ap=1 s=49152 n(v0 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d38c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000289:[<0x10000000323/2663 v964>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000289:[<0x10000000323/2663 v964>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.186+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838 auth v966 ap=1 s=417792 n(v0 b417792 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d39180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000028a:[<0x10000000323/2838 v966>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000028a:[<0x10000000323/2838 v966>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.186+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603 auth v972 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d42b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000028e:[<0x10000000323/2603 v972>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000028e:[<0x10000000323/2603 v972>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.186+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000290 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3080 auth v976 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d43600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000290:[<0x10000000323/3080 v976>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000290:[<0x10000000323/3080 v976>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.186+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000291 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3394_vm auth v978 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d43b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000291:[<0x10000000323/3394_vm v978>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000291:[<0x10000000323/3394_vm v978>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.187+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000292 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2754 auth v980 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d44100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000292:[<0x10000000323/2754 v980>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000292:[<0x10000000323/2754 v980>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.187+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247 auth v970 ap=1 s=73728 n(v0 b73728 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d42580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000028d:[<0x10000000323/1247 v970>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000028d:[<0x10000000323/1247 v970>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.187+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000293 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12199 auth v620 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d44680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000293:[<0x10000000323/12199 v620>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000293:[<0x10000000323/12199 v620>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.187+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000028f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764 auth v974 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d43080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000028f:[<0x10000000323/3764 v974>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000028f:[<0x10000000323/3764 v974>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.535 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.187+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000294 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3455 auth v982 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d44c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000294:[<0x10000000323/3455 v982>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000294:[<0x10000000323/3455 v982>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.187+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000295 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12182_fsm auth v984 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d45180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000295:[<0x10000000323/12182_fsm v984>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000295:[<0x10000000323/12182_fsm v984>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.188+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000296 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259 auth v986 ap=1 s=90112 n(v0 b90112 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d45700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000296:[<0x10000000323/1259 v986>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000296:[<0x10000000323/1259 v986>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.188+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000297 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12207 auth v988 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d4e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000297:[<0x10000000323/12207 v988>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000297:[<0x10000000323/12207 v988>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.188+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000299 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3606 auth v992 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d4eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000299:[<0x10000000323/3606 v992>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000299:[<0x10000000323/3606 v992>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.188+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2615 auth v386 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b2:[<0x10000000323/2615 v386>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b2:[<0x10000000323/2615 v386>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.189+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_fsm auth v390 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b3:[<0x10000000323/3456_fsm v390>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b3:[<0x10000000323/3456_fsm v390>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.189+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000298 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610 auth v990 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d4e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000298:[<0x10000000323/2610 v990>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000298:[<0x10000000323/2610 v990>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.189+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202 auth v392 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b4:[<0x10000000323/12202 v392>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b4:[<0x10000000323/12202 v392>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.189+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_vm auth v394 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b5:[<0x10000000323/1247_vm v394>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b5:[<0x10000000323/1247_vm v394>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.190+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000029a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2613 auth v634 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d4f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000029a:[<0x10000000323/2613 v634>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000029a:[<0x10000000323/2613 v634>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.190+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_vm auth v402 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b8:[<0x10000000323/2608_vm v402>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b8:[<0x10000000323/2608_vm v402>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.536 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.190+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764_fsm auth v400 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b7:[<0x10000000323/3764_fsm v400>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b7:[<0x10000000323/3764_fsm v400>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.537 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.190+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3767 auth v450 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b9:[<0x10000000323/3767 v450>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b9:[<0x10000000323/3767 v450>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.537 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.190+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605 auth v396 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b6:[<0x10000000323/2605 v396>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b6:[<0x10000000323/2605 v396>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.537 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.191+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2995 auth v106 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ceab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001bc:[<0x10000000323/2995 v106>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001bc:[<0x10000000323/2995 v106>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.191+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2669 auth v454 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cea580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001bb:[<0x10000000323/2669 v454>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001bb:[<0x10000000323/2669 v454>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.191+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ba [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_vm auth v452 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cea000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ba:[<0x10000000323/3456_vm v452>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ba:[<0x10000000323/3456_vm v452>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.191+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001be [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12201 auth v458 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ceb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001be:[<0x10000000323/12201 v458>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001be:[<0x10000000323/12201 v458>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.191+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bf [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607_fsm auth v462 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cebb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001bf:[<0x10000000323/2607_fsm v462>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001bf:[<0x10000000323/2607_fsm v462>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.192+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3503 auth v456 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ceb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001bd:[<0x10000000323/3503 v456>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001bd:[<0x10000000323/3503 v456>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.192+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2699 auth v464 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cec100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c0:[<0x10000000323/2699 v464>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c0:[<0x10000000323/2699 v464>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.192+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2658 auth v468 ap=1 s=106496 n(v0 b106496 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cec680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c1:[<0x10000000323/2658 v468>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c1:[<0x10000000323/2658 v468>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.192+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/828 auth v510 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ced180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c3:[<0x10000000323/828 v510>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c3:[<0x10000000323/828 v510>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.540 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.193+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3605 auth v516 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c5:[<0x10000000323/3605 v516>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c5:[<0x10000000323/3605 v516>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.193+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3609 auth v522 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c7:[<0x10000000323/3609 v522>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c7:[<0x10000000323/3609 v522>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.193+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_vm auth v518 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c6:[<0x10000000323/2616_vm v518>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c6:[<0x10000000323/2616_vm v518>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.194+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2666 auth v524 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c8:[<0x10000000323/2666 v524>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c8:[<0x10000000323/2666 v524>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.194+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_vm auth v554 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c9:[<0x10000000323/2603_vm v554>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c9:[<0x10000000323/2603_vm v554>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.194+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001cb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_vm auth v556 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001cb:[<0x10000000323/2617_vm v556>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001cb:[<0x10000000323/2617_vm v556>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.194+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001cc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601_vm auth v558 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001cc:[<0x10000000323/2601_vm v558>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001cc:[<0x10000000323/2601_vm v558>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.195+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001cd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3541 auth v560 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001cd:[<0x10000000323/3541 v560>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001cd:[<0x10000000323/3541 v560>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.195+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001cf [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2688 auth v568 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001cf:[<0x10000000323/2688 v568>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001cf:[<0x10000000323/2688 v568>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.195+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2604 auth v156 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cfe000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d0:[<0x10000000323/2604 v156>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d0:[<0x10000000323/2604 v156>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.196+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ca [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/826 auth v134 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ca:[<0x10000000323/826 v134>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ca:[<0x10000000323/826 v134>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.196+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3439_vm auth v164 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cfe580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d1:[<0x10000000323/3439_vm v164>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d1:[<0x10000000323/3439_vm v164>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.197+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612_vm auth v508 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cecc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c2:[<0x10000000323/2612_vm v508>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c2:[<0x10000000323/2612_vm v508>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.541 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.197+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609 auth v664 ap=1 s=286720 n(v0 rc2024-01-11T17:58:14.132771+0000 b286720 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cfeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d2:[<0x10000000323/2609 v664>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d2:[<0x10000000323/2609 v664>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.198+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1255 auth v666 ap=1 s=606208 n(v0 rc2024-01-11T17:58:14.133771+0000 b606208 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cff080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d3:[<0x10000000323/1255 v666>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d3:[<0x10000000323/1255 v666>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.198+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2650 auth v670 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cffb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d5:[<0x10000000323/2650 v670>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d5:[<0x10000000323/2650 v670>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.198+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2602 auth v672 ap=1 s=49152 n(v0 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d00100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d6:[<0x10000000323/2602 v672>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d6:[<0x10000000323/2602 v672>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.198+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2834_vm auth v182 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d00680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d7:[<0x10000000323/2834_vm v182>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d7:[<0x10000000323/2834_vm v182>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.199+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202_vm auth v514 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ced700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001c4:[<0x10000000323/12202_vm v514>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001c4:[<0x10000000323/12202_vm v514>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.199+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ce [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3600 auth v564 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cf9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ce:[<0x10000000323/3600 v564>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ce:[<0x10000000323/3600 v564>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.199+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3395 auth v676 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d00c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d8:[<0x10000000323/3395 v676>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d8:[<0x10000000323/3395 v676>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.200+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612 auth v668 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cff600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d4:[<0x10000000323/2612 v668>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d4:[<0x10000000323/2612 v668>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.200+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2675 auth v718 ap=1 s=180224 n(v0 b180224 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d01180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001d9:[<0x10000000323/2675 v718>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001d9:[<0x10000000323/2675 v718>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.200+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001da [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12186 auth v720 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d01700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001da:[<0x10000000323/12186 v720>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001da:[<0x10000000323/12186 v720>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.200+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001db [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/174 auth v722 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001db:[<0x10000000323/174 v722>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001db:[<0x10000000323/174 v722>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.200+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001dc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3602 auth v724 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001dc:[<0x10000000323/3602 v724>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001dc:[<0x10000000323/3602 v724>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.542 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.200+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001dd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2224 auth v200 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001dd:[<0x10000000323/2224 v200>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001dd:[<0x10000000323/2224 v200>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.201+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001de [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2689 auth v726 ap=1 s=49152 n(v0 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d0d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001de:[<0x10000000323/2689 v726>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001de:[<0x10000000323/2689 v726>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.201+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2662 auth v76 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000018a:[<0x10000000323/2662 v76>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000018a:[<0x10000000323/2662 v76>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.201+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000324 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3766 auth v1044 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.121771+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000324:[<0x10000000323/3766 v1044>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000324:[<0x10000000323/3766 v1044>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.201+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3257 auth v146 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000018b:[<0x10000000323/3257 v146>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000018b:[<0x10000000323/3257 v146>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.201+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6113 auth v150 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000018d:[<0x10000000323/6113 v150>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000018d:[<0x10000000323/6113 v150>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.201+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601 auth v152 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000018e:[<0x10000000323/2601 v152>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000018e:[<0x10000000323/2601 v152>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.202+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753_vm auth v154 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000018f:[<0x10000000323/2753_vm v154>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000018f:[<0x10000000323/2753_vm v154>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.202+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192_vm auth v148 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000018c:[<0x10000000323/12192_vm v148>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000018c:[<0x10000000323/12192_vm v148>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.202+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000190 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259_vm auth v158 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000190:[<0x10000000323/1259_vm v158>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000190:[<0x10000000323/1259_vm v158>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.202+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000191 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607 auth v160 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000191:[<0x10000000323/2607 v160>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000191:[<0x10000000323/2607 v160>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.202+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000193 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609_fsm auth v168 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000193:[<0x10000000323/2609_fsm v168>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000193:[<0x10000000323/2609_fsm v168>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.202+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000194 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12204 auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000194:[<0x10000000323/12204 v24>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000194:[<0x10000000323/12204 v24>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.543 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.203+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000192 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2757 auth v162 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000192:[<0x10000000323/2757 v162>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000192:[<0x10000000323/2757 v162>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.203+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000195 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2835 auth v170 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000195:[<0x10000000323/2835 v170>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000195:[<0x10000000323/2835 v170>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.203+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000196 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2837 auth v172 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c7d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000196:[<0x10000000323/2837 v172>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000196:[<0x10000000323/2837 v172>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.203+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000199 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3118_vm auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c82b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000199:[<0x10000000323/3118_vm v34>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000199:[<0x10000000323/3118_vm v34>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.204+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187 auth v186 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c83080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000019a:[<0x10000000323/12187 v186>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000019a:[<0x10000000323/12187 v186>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.204+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/5002 auth v190 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c83600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000019b:[<0x10000000323/5002 v190>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000019b:[<0x10000000323/5002 v190>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.205+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3596_vm auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c84100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000019d:[<0x10000000323/3596_vm v42>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000019d:[<0x10000000323/3596_vm v42>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.205+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_fsm auth v212 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c84c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000019f:[<0x10000000323/1247_fsm v212>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000019f:[<0x10000000323/1247_fsm v212>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.206+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2665 auth v222 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c85180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a0:[<0x10000000323/2665 v222>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a0:[<0x10000000323/2665 v222>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.206+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3598 auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c84680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000019e:[<0x10000000323/3598 v44>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000019e:[<0x10000000323/3598 v44>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.206+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000197 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3381_vm auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c82000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000197:[<0x10000000323/3381_vm v30>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000197:[<0x10000000323/3381_vm v30>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.207+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000198 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1417 auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c82580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000198:[<0x10000000323/1417 v32>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000198:[<0x10000000323/1417 v32>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.207+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2679 auth v228 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c85700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a1:[<0x10000000323/2679 v228>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a1:[<0x10000000323/2679 v228>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.544 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.207+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2660 auth v196 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c83b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000019c:[<0x10000000323/2660 v196>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000019c:[<0x10000000323/2660 v196>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.207+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12212 auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c46000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a2:[<0x10000000323/12212 v52>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a2:[<0x10000000323/12212 v52>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.207+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456 auth v240 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c46580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a3:[<0x10000000323/3456 v240>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a3:[<0x10000000323/3456 v240>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.208+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12209 auth v56 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c46b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a4:[<0x10000000323/12209 v56>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a4:[<0x10000000323/12209 v56>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.208+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3379 auth v252 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c47080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a5:[<0x10000000323/3379 v252>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a5:[<0x10000000323/3379 v252>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.208+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610_vm auth v258 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c47600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a6:[<0x10000000323/2610_vm v258>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a6:[<0x10000000323/2610_vm v258>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.208+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2683 auth v264 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c47b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a7:[<0x10000000323/2683 v264>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a7:[<0x10000000323/2683 v264>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.208+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12196 auth v322 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c48100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a8:[<0x10000000323/12196 v322>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a8:[<0x10000000323/12196 v322>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.209+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753 auth v324 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c48680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001a9:[<0x10000000323/2753 v324>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001a9:[<0x10000000323/2753 v324>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.209+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001aa [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192 auth v326 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c48c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001aa:[<0x10000000323/12192 v326>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001aa:[<0x10000000323/12192 v326>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.209+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ae [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_fsm auth v336 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c90580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ae:[<0x10000000323/2616_fsm v336>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ae:[<0x10000000323/2616_fsm v336>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.210+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ab [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_fsm auth v328 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c49180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ab:[<0x10000000323/2838_fsm v328>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ab:[<0x10000000323/2838_fsm v328>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.210+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ad [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2619_fsm auth v332 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c90000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ad:[<0x10000000323/2619_fsm v332>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ad:[<0x10000000323/2619_fsm v332>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.545 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.210+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001af [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3576_vm auth v80 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c90b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001af:[<0x10000000323/3576_vm v80>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001af:[<0x10000000323/3576_vm v80>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.210+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2656 auth v344 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c91600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b1:[<0x10000000323/2656 v344>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b1:[<0x10000000323/2656 v344>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.210+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079_vm auth v994 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d4f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c8:[<0x10000000420/3079_vm v994>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c8:[<0x10000000420/3079_vm v994>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.211+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_fsm auth v340 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c91080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001b0:[<0x10000000323/2608_fsm v340>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001b0:[<0x10000000323/2608_fsm v340>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.211+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2600_fsm auth v996 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d4fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c9:[<0x10000000420/2600_fsm v996>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c9:[<0x10000000420/2600_fsm v996>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.211+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_filenode.map auth v1000 ap=1 s=512 n(v0 b512 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d50680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cb:[<0x10000000420/pg_filenode.map v1000>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003cb:[<0x10000000420/pg_filenode.map v1000>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.211+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ac [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_vm auth v330 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c49700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000001ac:[<0x10000000323/2838_vm v330>,<0x10000000014/12346 v2858>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000001ac:[<0x10000000323/2838_vm v330>,<0x10000000014/12346 v3403>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.212+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840_fsm auth v1002 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d50c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cc:[<0x10000000420/2840_fsm v1002>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003cc:[<0x10000000420/2840_fsm v1002>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.212+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3997 auth v1004 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d51180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cd:[<0x10000000420/3997 v1004>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003cd:[<0x10000000420/3997 v1004>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.212+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ca [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_vm auth v998 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.574764+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d50100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ca:[<0x10000000420/12197_vm v998>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ca:[<0x10000000420/12197_vm v998>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.212+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ce [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2832_vm auth v626 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10d51700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ce:[<0x10000000420/2832_vm v626>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ce:[<0x10000000420/2832_vm v626>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.546 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.212+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cf [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606_vm auth v1006 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cf:[<0x10000000420/2606_vm v1006>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003cf:[<0x10000000420/2606_vm v1006>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.212+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2653 auth v1008 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d0:[<0x10000000420/2653 v1008>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d0:[<0x10000000420/2653 v1008>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.213+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2755 auth v1010 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d1:[<0x10000000420/2755 v1010>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d1:[<0x10000000420/2755 v1010>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.213+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118 auth v636 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d3:[<0x10000000420/3118 v636>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d3:[<0x10000000420/3118 v636>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.213+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_fsm auth v1018 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abe680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d6:[<0x10000000420/3541_fsm v1018>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d6:[<0x10000000420/3541_fsm v1018>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.213+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3534 auth v1020 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.577764+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d7:[<0x10000000420/3534 v1020>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d7:[<0x10000000420/3534 v1020>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.214+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3081 auth v1022 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abf700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d9:[<0x10000000420/3081 v1022>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d9:[<0x10000000420/3081 v1022>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.215+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_vm auth v1014 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d4:[<0x10000000420/2618_vm v1014>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d4:[<0x10000000420/2618_vm v1014>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.215+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2659 auth v1063 ap=1 s=81920 n(v0 b81920 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10abe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d5:[<0x10000000420/2659 v1016>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d5:[<0x10000000420/2659 v1063>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.215+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/175 auth v1012 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d2:[<0x10000000420/175 v1012>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d2:[<0x10000000420/175 v1012>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.215+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439 auth v660 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10abf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d8:[<0x10000000420/3439 v660>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003d8:[<0x10000000420/3439 v660>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003da [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256_vm auth v666 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003da:[<0x10000000420/3256_vm v666>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003da:[<0x10000000420/3256_vm v666>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003db [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255_fsm auth v1024 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003db:[<0x10000000420/1255_fsm v1024>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003db:[<0x10000000420/1255_fsm v1024>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606 auth v1026 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003dc:[<0x10000000420/2606 v1026>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003dc:[<0x10000000420/2606 v1026>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.547 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2328 auth v672 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003dd:[<0x10000000420/2328 v672>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003dd:[<0x10000000420/2328 v672>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003de [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830 auth v674 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003de:[<0x10000000420/2830 v674>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003de:[<0x10000000420/2830 v674>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003df [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6102_vm auth v676 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003df:[<0x10000000420/6102_vm v676>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003df:[<0x10000000420/6102_vm v676>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.216+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6104_vm auth v678 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e0:[<0x10000000420/6104_vm v678>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e0:[<0x10000000420/6104_vm v678>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.217+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_vm auth v1028 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e1:[<0x10000000420/2619_vm v1028>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e1:[<0x10000000420/2619_vm v1028>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.217+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079 auth v1030 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e2:[<0x10000000420/3079 v1030>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e2:[<0x10000000420/3079 v1030>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.217+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826_vm auth v684 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e3:[<0x10000000420/826_vm v684>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e3:[<0x10000000420/826_vm v684>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.217+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2686 auth v1032 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ac7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e4:[<0x10000000420/2686 v1032>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e4:[<0x10000000420/2686 v1032>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.218+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2337 auth v1034 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ace000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e5:[<0x10000000420/2337 v1034>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e5:[<0x10000000420/2337 v1034>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.218+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2693 auth v1036 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ace580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e6:[<0x10000000420/2693 v1036>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e6:[<0x10000000420/2693 v1036>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.218+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3574 auth v1038 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10acf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e8:[<0x10000000420/3574 v1038>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e8:[<0x10000000420/3574 v1038>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.218+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2704 auth v1065 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10acf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e9:[<0x10000000420/2704 v1040>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e9:[<0x10000000420/2704 v1065>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.548 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.218+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830_vm auth v692 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10aceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e7:[<0x10000000420/2830_vm v692>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003e7:[<0x10000000420/2830_vm v692>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.219+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_vm auth v914 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ba9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039c:[<0x10000000420/3603_vm v914>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000039c:[<0x10000000420/3603_vm v914>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.219+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ea [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_fsm auth v1042 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10acfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ea:[<0x10000000420/12207_fsm v1042>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ea:[<0x10000000420/12207_fsm v1042>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.219+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3575 auth v916 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ba9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039d:[<0x10000000420/3575 v916>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000039d:[<0x10000000420/3575 v916>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.219+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387_fsm auth v1054 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ba9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f3:[<0x10000000420/16387_fsm v1054>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003f3:[<0x10000000420/16387_fsm v1054>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.219+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3467 auth v1072 ap=1 s=8192 n(v0 b8192 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10baa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039e:[<0x10000000420/3467 v918>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000039e:[<0x10000000420/3467 v1072>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.219+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16393 auth v1050 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10baa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f2:[<0x10000000420/16393 v1050>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003f2:[<0x10000000420/16393 v1050>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.220+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_fsm auth v920 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10baac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039f:[<0x10000000420/2617_fsm v920>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000039f:[<0x10000000420/2617_fsm v920>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.220+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3119 auth v922 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.565764+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bab180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a0:[<0x10000000420/3119 v922>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a0:[<0x10000000420/3119 v922>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.549 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.220+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2674 auth v1081 ap=1 s=385024 n(v0 b385024 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10bab700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a1:[<0x10000000420/2674 v924>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a1:[<0x10000000420/2674 v1081>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.538+0000 7fb5f1fff700 1 -- 172.21.15.7:0/3910814056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1072d0 msgr2=0x7fb5ec1096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.538+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/3910814056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1096e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009990 tx=0x7fb5e4009ca0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.538+0000 7fb5f1fff700 1 -- 172.21.15.7:0/3910814056 shutdown_connections 2024-01-11T17:58:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.539+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/3910814056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1096e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.539+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/3910814056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 0x7fb5ec104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.539+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/3910814056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1049a0 0x7fb5ec106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.539+0000 7fb5f1fff700 1 -- 172.21.15.7:0/3910814056 >> 172.21.15.7:0/3910814056 conn(0x7fb5ec0fba00 msgr2=0x7fb5ec0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.539+0000 7fb5f1fff700 1 -- 172.21.15.7:0/3910814056 shutdown_connections 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.539+0000 7fb5f1fff700 1 -- 172.21.15.7:0/3910814056 wait complete. 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.542+0000 7fb5f1fff700 1 Processor -- start 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 -- start start 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 0x7fb5ec19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1049a0 0x7fb5ec19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec1a5600 con 0x7fb5ec102070 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec1100d0 con 0x7fb5ec1072d0 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.543+0000 7fb5f1fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec110290 con 0x7fb5ec1049a0 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.544+0000 7fb5e2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1049a0 0x7fb5ec19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.544+0000 7fb5e2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1049a0 0x7fb5ec19e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54738/0 (socket says 172.21.15.7:54738) 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.544+0000 7fb5e2ffd700 1 -- 172.21.15.7:0/2975687678 learned_addr learned my addr 172.21.15.7:0/2975687678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.544+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 0x7fb5ec19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.544+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5ebfff700 1 -- 172.21.15.7:0/2975687678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1049a0 msgr2=0x7fb5ec19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1049a0 0x7fb5ec19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5ebfff700 1 -- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 msgr2=0x7fb5ec19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 0x7fb5ec19db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5ebfff700 1 -- 172.21.15.7:0/2975687678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40095f0 con 0x7fb5ec1072d0 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 0x7fb5ec19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.545+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1a5030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40098e0 tx=0x7fb5e400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.546+0000 7fb5e97fa700 1 -- 172.21.15.7:0/2975687678 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e401d070 con 0x7fb5ec1072d0 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.546+0000 7fb5e97fa700 1 -- 172.21.15.7:0/2975687678 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5e400f780 con 0x7fb5ec1072d0 2024-01-11T17:58:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.546+0000 7fb5e97fa700 1 -- 172.21.15.7:0/2975687678 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4004870 con 0x7fb5ec1072d0 2024-01-11T17:58:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.546+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec110510 con 0x7fb5ec1072d0 2024-01-11T17:58:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.547+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec1109f0 con 0x7fb5ec1072d0 2024-01-11T17:58:42.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.555+0000 7fb605d52b80 1 Processor -- start 2024-01-11T17:58:42.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.557+0000 7fb605d52b80 1 -- start start 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.557+0000 7fb5e97fa700 1 -- 172.21.15.7:0/2975687678 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fb5e400f8f0 con 0x7fb5ec1072d0 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e23820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb605d52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243d19090 con 0x55c243e22570 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb605d52b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243d7cb20 con 0x55c243e23400 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb605d52b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243d7a440 con 0x55c243e23d80 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb5eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb5eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e21f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54744/0 (socket says 172.21.15.7:54744) 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.558+0000 7fb5eb7fe700 1 -- 172.21.15.7:0/1124651152 learned_addr learned my addr 172.21.15.7:0/1124651152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e23820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5eb7fe700 1 -- 172.21.15.7:0/1124651152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 msgr2=0x55c243e23820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e23820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5eb7fe700 1 -- 172.21.15.7:0/1124651152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 msgr2=0x55c243e229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5eb7fe700 1 -- 172.21.15.7:0/1124651152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c243bea3b0 con 0x55c243e23d80 2024-01-11T17:58:42.562 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.220+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2668 auth v926 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bd8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a2:[<0x10000000420/2668 v926>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a2:[<0x10000000420/2668 v926>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.562 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.220+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_fsm auth v928 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bd8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a3:[<0x10000000420/2603_fsm v928>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a3:[<0x10000000420/2603_fsm v928>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.221+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6110 auth v930 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.566764+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bd8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a4:[<0x10000000420/6110 v930>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a4:[<0x10000000420/6110 v930>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.221+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2655 auth v1074 ap=1 s=32768 n(v0 b32768 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10bd9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a6:[<0x10000000420/2655 v934>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a6:[<0x10000000420/2655 v1074>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.221+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3597 auth v936 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bd9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a7:[<0x10000000420/3597 v936>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a7:[<0x10000000420/3597 v936>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.221+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608 auth v940 ap=1 s=442368 n(v0 rc2024-01-11T17:58:14.567764+0000 b442368 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bda680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a9:[<0x10000000420/2608 v940>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a9:[<0x10000000420/2608 v940>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.221+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ab [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_vm auth v944 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bdb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ab:[<0x10000000420/12187_vm v944>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ab:[<0x10000000420/12187_vm v944>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.222+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3164 auth v946 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bdb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ac:[<0x10000000420/3164 v946>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ac:[<0x10000000420/3164 v946>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.222+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003aa [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2831 auth v942 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.568764+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bdac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003aa:[<0x10000000420/2831 v942>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003aa:[<0x10000000420/2831 v942>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.222+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ae [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6111 auth v948 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ae:[<0x10000000420/6111 v948>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ae:[<0x10000000420/6111 v948>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.223+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003af [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2682 auth v950 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.569764+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003af:[<0x10000000420/2682 v950>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003af:[<0x10000000420/2682 v950>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.224+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/113 auth v952 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b0:[<0x10000000420/113 v952>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b0:[<0x10000000420/113 v952>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.224+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3712 auth v938 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bda100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a8:[<0x10000000420/3712 v938>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a8:[<0x10000000420/3712 v938>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.563 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.224+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2702 auth v932 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bd9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a5:[<0x10000000420/2702 v932>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003a5:[<0x10000000420/2702 v932>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.225+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2691 auth v956 ap=1 s=270336 n(v0 b270336 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b2:[<0x10000000420/2691 v956>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b2:[<0x10000000420/2691 v956>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.225+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3604 auth v954 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b1:[<0x10000000420/3604 v954>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b1:[<0x10000000420/3604 v954>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.225+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_vm auth v958 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b3:[<0x10000000420/2605_vm v958>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b3:[<0x10000000420/2605_vm v958>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.227+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2663 auth v1079 ap=1 s=49152 n(v0 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10be5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b6:[<0x10000000420/2663 v964>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b6:[<0x10000000420/2663 v1079>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.229+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838 auth v966 ap=1 s=417792 n(v0 b417792 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bec000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b7:[<0x10000000420/2838 v966>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b7:[<0x10000000420/2838 v966>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.230+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2836 auth v570 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bec580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b8:[<0x10000000420/2836 v570>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b8:[<0x10000000420/2836 v570>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.232+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/548 auth v968 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10becb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b9:[<0x10000000420/548 v968>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b9:[<0x10000000420/548 v968>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.233+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ad [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3501_vm auth v546 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ad:[<0x10000000420/3501_vm v546>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ad:[<0x10000000420/3501_vm v546>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.236+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2685 auth v1068 ap=1 s=16384 n(v0 b16384 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10be4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b5:[<0x10000000420/2685 v962>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b5:[<0x10000000420/2685 v1068>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.236+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000523 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_internal.init auth v1085 ap=1 s=0 n(v0 rc2024-01-11T17:58:16.184740+0000 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10be5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000523:[<0x10000000420/pg_internal.init v1046>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000523:[<0x10000000420/pg_internal.init v1085>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.237+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764 auth v974 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bedb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bc:[<0x10000000420/3764 v974>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003bc:[<0x10000000420/3764 v974>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.564 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.237+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003be [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394_vm auth v978 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bee680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003be:[<0x10000000420/3394_vm v978>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003be:[<0x10000000420/3394_vm v978>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.237+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3440 auth v960 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10be4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b4:[<0x10000000420/3440 v960>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003b4:[<0x10000000420/3440 v960>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.238+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12199 auth v598 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bef180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c0:[<0x10000000420/12199 v598>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c0:[<0x10000000420/12199 v598>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.238+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3080 auth v976 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bd:[<0x10000000420/3080 v976>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003bd:[<0x10000000420/3080 v976>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.240+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_fsm auth v984 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c2:[<0x10000000420/12182_fsm v984>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c2:[<0x10000000420/12182_fsm v984>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.240+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3455 auth v1083 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10bef700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c1:[<0x10000000420/3455 v982>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c1:[<0x10000000420/3455 v1083>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.241+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207 auth v988 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c4:[<0x10000000420/12207 v988>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c4:[<0x10000000420/12207 v988>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.241+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259 auth v986 ap=1 s=90112 n(v0 b90112 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c3:[<0x10000000420/1259 v986>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c3:[<0x10000000420/1259 v986>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.243+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610 auth v990 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c5:[<0x10000000420/2610 v990>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c5:[<0x10000000420/2610 v990>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.249+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3606 auth v992 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c6:[<0x10000000420/3606 v992>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c6:[<0x10000000420/3606 v992>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.249+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bf [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2754 auth v980 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10beec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bf:[<0x10000000420/2754 v980>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003bf:[<0x10000000420/2754 v980>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.250+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ba [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247 auth v970 ap=1 s=73728 n(v0 rc2024-01-11T17:58:14.572764+0000 b73728 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bed080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ba:[<0x10000000420/1247 v970>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ba:[<0x10000000420/1247 v970>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.565 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.251+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000378 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2701 auth v864 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c39600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000378:[<0x10000000420/2701 v864>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000378:[<0x10000000420/2701 v864>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.252+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000376 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_fsm auth v860 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000376:[<0x10000000420/2601_fsm v860>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000376:[<0x10000000420/2601_fsm v860>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.253+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603 auth v868 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c3a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037a:[<0x10000000420/3603 v868>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000037a:[<0x10000000420/3603 v868>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.255+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_vm auth v870 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c3a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037b:[<0x10000000420/3764_vm v870>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000037b:[<0x10000000420/3764_vm v870>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.256+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3601_vm auth v872 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c3ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037c:[<0x10000000420/3601_vm v872>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000037c:[<0x10000000420/3601_vm v872>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.256+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12184 auth v404 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c3b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037d:[<0x10000000420/12184 v404>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000037d:[<0x10000000420/12184 v404>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.256+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000377 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3542 auth v862 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c39080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000377:[<0x10000000420/3542 v862>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000377:[<0x10000000420/3542 v862>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.257+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3350_vm auth v406 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c3b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037e:[<0x10000000420/3350_vm v406>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000037e:[<0x10000000420/3350_vm v406>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.257+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000380 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2611_vm auth v410 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000380:[<0x10000000420/2611_vm v410>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000380:[<0x10000000420/2611_vm v410>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.259+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000381 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3466 auth v412 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c52b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000381:[<0x10000000420/3466 v412>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000381:[<0x10000000420/3466 v412>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.260+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000382 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3380 auth v876 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.559764+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000382:[<0x10000000420/3380 v876>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000382:[<0x10000000420/3380 v876>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.264+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603 auth v1076 ap=1 s=32768 n(v0 b32768 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10bed600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bb:[<0x10000000420/2603 v972>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003bb:[<0x10000000420/2603 v1076>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.264+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613 auth v612 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c7:[<0x10000000420/2613 v612>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003c7:[<0x10000000420/2613 v612>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.566 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.269+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000379 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_fsm auth v866 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c39b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000379:[<0x10000000420/2602_fsm v866>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000379:[<0x10000000420/2602_fsm v866>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.269+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000386 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/549 auth v882 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c54680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000386:[<0x10000000420/549 v882>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000386:[<0x10000000420/549 v882>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.271+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000383 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2692 auth v878 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000383:[<0x10000000420/2692 v878>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000383:[<0x10000000420/2692 v878>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.271+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000388 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418 auth v444 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c55180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000388:[<0x10000000420/1418 v444>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000388:[<0x10000000420/1418 v444>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.272+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000389 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12216 auth v886 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c55700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000389:[<0x10000000420/12216 v886>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000389:[<0x10000000420/12216 v886>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.273+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1249 auth v1061 ap=1 s=393216 n(v0 rc2024-01-11T17:58:14.561764+0000 b393216 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10c5e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000038a:[<0x10000000420/1249 v888>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000038a:[<0x10000000420/1249 v1061>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.274+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/PG_VERSION auth v890 ap=1 s=3 n(v0 b3 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c5e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000038b:[<0x10000000420/PG_VERSION v890>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000038b:[<0x10000000420/PG_VERSION v890>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.275+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12206 auth v892 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c5eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000038c:[<0x10000000420/12206 v892>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000038c:[<0x10000000420/12206 v892>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.280+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000387 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_fsm auth v884 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c54c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000387:[<0x10000000420/2605_fsm v884>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000387:[<0x10000000420/2605_fsm v884>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.280+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418_vm auth v456 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c5f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000038d:[<0x10000000420/1418_vm v456>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000038d:[<0x10000000420/1418_vm v456>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.281+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576 auth v458 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c5fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000038e:[<0x10000000420/3576 v458>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000038e:[<0x10000000420/3576 v458>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.282+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840 auth v874 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037f:[<0x10000000420/2840 v874>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000037f:[<0x10000000420/2840 v874>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.567 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.282+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000384 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6112 auth v880 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c53b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000384:[<0x10000000420/6112 v880>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000384:[<0x10000000420/6112 v880>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.282+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000390 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834 auth v462 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c60680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000390:[<0x10000000420/2834 v462>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000390:[<0x10000000420/2834 v462>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.282+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000391 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106_vm auth v464 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c60c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000391:[<0x10000000420/6106_vm v464>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000391:[<0x10000000420/6106_vm v464>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.283+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000392 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256 auth v466 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c61180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000392:[<0x10000000420/3256 v466>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000392:[<0x10000000420/3256 v466>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.283+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000394 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_vm auth v898 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000394:[<0x10000000420/2602_vm v898>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000394:[<0x10000000420/2602_vm v898>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.283+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000393 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2652 auth v896 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c61700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000393:[<0x10000000420/2652 v896>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000393:[<0x10000000420/2652 v896>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.284+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000395 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3502 auth v900 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000395:[<0x10000000420/3502 v900>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000395:[<0x10000000420/3502 v900>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.284+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000396 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394 auth v902 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000396:[<0x10000000420/3394 v902>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000396:[<0x10000000420/3394 v902>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.285+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000397 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2841 auth v904 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000397:[<0x10000000420/2841 v904>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000397:[<0x10000000420/2841 v904>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.285+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000398 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_fsm auth v1054 ap=1 s=24576 n(v0 b24576 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10c6b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000398:[<0x10000000420/1259_fsm v906>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000398:[<0x10000000420/1259_fsm v1054>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.285+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2651 auth v910 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039a:[<0x10000000420/2651 v910>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000039a:[<0x10000000420/2651 v910>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.286+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000385 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995_vm auth v436 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000385:[<0x10000000420/2995_vm v436>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000385:[<0x10000000420/2995_vm v436>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.568 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.286+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16390 auth v1044 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c5f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f1:[<0x10000000420/16390 v1044>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003f1:[<0x10000000420/16390 v1044>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.286+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_vm auth v894 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c60100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000038f:[<0x10000000420/3541_vm v894>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000038f:[<0x10000000420/3541_vm v894>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.286+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2673 auth v1063 ap=1 s=327680 n(v0 rc2024-01-11T17:58:14.564764+0000 b327680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10c6c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039b:[<0x10000000420/2673 v912>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000039b:[<0x10000000420/2673 v1063>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.287+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000359 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224_vm auth v320 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c91b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000359:[<0x10000000420/2224_vm v320>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000359:[<0x10000000420/2224_vm v320>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.287+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2681 auth v816 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c92680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035a:[<0x10000000420/2681 v816>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000035a:[<0x10000000420/2681 v816>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.287+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_vm auth v818 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035b:[<0x10000000420/12207_vm v818>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000035b:[<0x10000000420/12207_vm v818>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.287+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12191 auth v822 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c93700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035d:[<0x10000000420/12191 v822>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000035d:[<0x10000000420/12191 v822>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.288+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613_vm auth v330 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035e:[<0x10000000420/2613_vm v330>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000035e:[<0x10000000420/2613_vm v330>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.288+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3607 auth v824 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035f:[<0x10000000420/3607 v824>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000035f:[<0x10000000420/3607 v824>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.289+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000360 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618 auth v826 ap=1 s=98304 n(v0 b98304 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000360:[<0x10000000420/2618 v826>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000360:[<0x10000000420/2618 v826>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.290+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000361 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_fsm auth v828 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000361:[<0x10000000420/2618_fsm v828>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000361:[<0x10000000420/2618_fsm v828>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.293+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000399 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_vm auth v908 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c6bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000399:[<0x10000000420/3602_vm v908>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000399:[<0x10000000420/3602_vm v908>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.569 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.293+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000362 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_fsm auth v830 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000362:[<0x10000000420/3603_fsm v830>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000362:[<0x10000000420/3603_fsm v830>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.294+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000363 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2690 auth v832 ap=1 s=81920 n(v0 b81920 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000363:[<0x10000000420/2690 v832>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000363:[<0x10000000420/2690 v832>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.294+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ef [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16384 auth v1044 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c92100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ef:[<0x10000000420/16384 v1044>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003ef:[<0x10000000420/16384 v1044>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.294+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000364 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2684 auth v1084 ap=1 s=16384 n(v0 b16384 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10cbe680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000364:[<0x10000000420/2684 v834>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000364:[<0x10000000420/2684 v1084>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.295+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000366 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2620_vm auth v358 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000366:[<0x10000000420/2620_vm v358>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000366:[<0x10000000420/2620_vm v358>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.296+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_vm auth v820 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c93180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035c:[<0x10000000420/12182_vm v820>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000035c:[<0x10000000420/12182_vm v820>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.297+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000365 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2680 auth v836 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000365:[<0x10000000420/2680 v836>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000365:[<0x10000000420/2680 v836>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.297+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2187 auth v844 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cccb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000036a:[<0x10000000420/2187 v844>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000036a:[<0x10000000420/2187 v844>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.298+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387 auth v1046 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f0:[<0x10000000420/16387 v1046>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003f0:[<0x10000000420/16387 v1046>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.298+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000367 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2696 auth v838 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cbf700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000367:[<0x10000000420/2696 v838>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000367:[<0x10000000420/2696 v838>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.299+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2654 auth v846 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000036c:[<0x10000000420/2654 v846>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000036c:[<0x10000000420/2654 v846>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.299+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000368 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2678 auth v1087 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10ccc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000368:[<0x10000000420/2678 v840>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000368:[<0x10000000420/2678 v1087>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.570 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.299+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_fsm auth v848 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000036d:[<0x10000000420/3602_fsm v848>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000036d:[<0x10000000420/3602_fsm v848>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.299+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381 auth v368 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000036b:[<0x10000000420/3381 v368>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000036b:[<0x10000000420/3381 v368>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.300+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417_vm auth v374 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cce100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000036e:[<0x10000000420/1417_vm v374>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000036e:[<0x10000000420/1417_vm v374>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.300+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000369 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_fsm auth v842 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000369:[<0x10000000420/12192_fsm v842>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000369:[<0x10000000420/12192_fsm v842>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.300+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000370 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106 auth v378 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000370:[<0x10000000420/6106 v378>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000370:[<0x10000000420/6106 v378>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.300+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_fsm auth v850 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cce680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000036f:[<0x10000000420/2753_fsm v850>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000036f:[<0x10000000420/2753_fsm v850>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.300+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000371 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3085 auth v852 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000371:[<0x10000000420/3085 v852>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000371:[<0x10000000420/3085 v852>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.300+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000372 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_fsm auth v854 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ccf700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000372:[<0x10000000420/12197_fsm v854>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000372:[<0x10000000420/12197_fsm v854>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.301+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000373 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_fsm auth v856 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cda000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000373:[<0x10000000420/12187_fsm v856>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000373:[<0x10000000420/12187_fsm v856>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.301+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000375 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12189 auth v388 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10cdab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000375:[<0x10000000420/12189 v388>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000375:[<0x10000000420/12189 v388>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.301+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000374 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2703 auth v1089 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10cda580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000374:[<0x10000000420/2703 v858>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000374:[<0x10000000420/2703 v1089>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.571 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.301+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000421 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3766 auth v1044 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.523765+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000421:[<0x10000000420/3766 v1044>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000421:[<0x10000000420/3766 v1044>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.301+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2662 auth v1075 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10bf8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c0:[<0x10000000420/2662 v56>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c0:[<0x10000000420/2662 v1075>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.302+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3257 auth v58 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c1:[<0x10000000420/3257 v58>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c1:[<0x10000000420/3257 v58>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.302+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601 auth v100 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c4:[<0x10000000420/2601 v100>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c4:[<0x10000000420/2601 v100>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.302+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_vm auth v138 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c5:[<0x10000000420/2753_vm v138>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c5:[<0x10000000420/2753_vm v138>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.302+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_vm auth v1071 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10c1a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c6:[<0x10000000420/1259_vm v140>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c6:[<0x10000000420/1259_vm v1071>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.303+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2757 auth v158 ap=1 s=32768 n(v0 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c8:[<0x10000000420/2757 v158>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c8:[<0x10000000420/2757 v158>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.303+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607 auth v144 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c7:[<0x10000000420/2607 v144>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c7:[<0x10000000420/2607 v144>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.303+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609_fsm auth v170 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c9:[<0x10000000420/2609_fsm v170>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c9:[<0x10000000420/2609_fsm v170>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.303+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002cb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2835 auth v208 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002cb:[<0x10000000420/2835 v208>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002cb:[<0x10000000420/2835 v208>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.303+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_vm auth v94 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c2:[<0x10000000420/12192_vm v94>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c2:[<0x10000000420/12192_vm v94>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.572 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.304+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002c3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6113 auth v96 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10bf9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002c3:[<0x10000000420/6113 v96>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002c3:[<0x10000000420/6113 v96>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.304+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ca [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12204 auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ca:[<0x10000000420/12204 v24>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ca:[<0x10000000420/12204 v24>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.304+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ce [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417 auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ce:[<0x10000000420/1417 v32>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ce:[<0x10000000420/1417 v32>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.304+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002cf [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118_vm auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002cf:[<0x10000000420/3118_vm v34>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002cf:[<0x10000000420/3118_vm v34>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.304+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002cc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2837 auth v224 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002cc:[<0x10000000420/2837 v224>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002cc:[<0x10000000420/2837 v224>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.304+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002cd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381_vm auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c1cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002cd:[<0x10000000420/3381_vm v30>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002cd:[<0x10000000420/3381_vm v30>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.305+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187 auth v338 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.526765+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c22000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d0:[<0x10000000420/12187 v338>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d0:[<0x10000000420/12187 v338>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.305+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2660 auth v342 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c22b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d2:[<0x10000000420/2660 v342>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d2:[<0x10000000420/2660 v342>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.305+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3596_vm auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c23080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d3:[<0x10000000420/3596_vm v42>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d3:[<0x10000000420/3596_vm v42>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.305+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/5002 auth v340 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c22580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d1:[<0x10000000420/5002 v340>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d1:[<0x10000000420/5002 v340>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.305+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3598 auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c23600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d4:[<0x10000000420/3598 v44>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d4:[<0x10000000420/3598 v44>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.305+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_fsm auth v1069 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@5},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55ee10c23b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d5:[<0x10000000420/1247_fsm v344>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d5:[<0x10000000420/1247_fsm v1069>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.306+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2665 auth v346 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c24100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d6:[<0x10000000420/2665 v346>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d6:[<0x10000000420/2665 v346>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.573 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.306+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2679 auth v1077 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10c24680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d7:[<0x10000000420/2679 v350>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d7:[<0x10000000420/2679 v1077>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.306+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12212 auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c24c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d8:[<0x10000000420/12212 v52>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d8:[<0x10000000420/12212 v52>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.306+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456 auth v416 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c25180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d9:[<0x10000000420/3456 v416>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002d9:[<0x10000000420/3456 v416>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.306+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002dc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610_vm auth v420 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c30580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002dc:[<0x10000000420/2610_vm v420>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002dc:[<0x10000000420/2610_vm v420>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.307+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002de [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12196 auth v424 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c31080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002de:[<0x10000000420/12196 v424>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002de:[<0x10000000420/12196 v424>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.307+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002df [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753 auth v428 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c31600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002df:[<0x10000000420/2753 v428>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002df:[<0x10000000420/2753 v428>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.307+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_fsm auth v434 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c32100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e1:[<0x10000000420/2838_fsm v434>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e1:[<0x10000000420/2838_fsm v434>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.308+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002dd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2683 auth v422 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c30b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002dd:[<0x10000000420/2683 v422>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002dd:[<0x10000000420/2683 v422>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.308+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002da [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12209 auth v60 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c25700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002da:[<0x10000000420/12209 v60>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002da:[<0x10000000420/12209 v60>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.308+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192 auth v430 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c31b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e0:[<0x10000000420/12192 v430>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e0:[<0x10000000420/12192 v430>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.308+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_vm auth v440 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c32680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e2:[<0x10000000420/2838_vm v440>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e2:[<0x10000000420/2838_vm v440>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.308+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_fsm auth v482 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c33180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e4:[<0x10000000420/2616_fsm v482>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e4:[<0x10000000420/2616_fsm v482>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.574 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.309+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576_vm auth v82 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c33700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e5:[<0x10000000420/3576_vm v82>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e5:[<0x10000000420/3576_vm v82>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.309+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000011c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603_vm auth v441 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.064772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000011c:[<0x10000000015/3603_vm v441>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000011c:[<0x10000000015/3603_vm v441>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.309+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002db [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3379 auth v418 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c30000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002db:[<0x10000000420/3379 v418>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002db:[<0x10000000420/3379 v418>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.309+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_fsm auth v446 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c32c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e3:[<0x10000000420/2619_fsm v446>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e3:[<0x10000000420/2619_fsm v446>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.310+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_fsm auth v1073 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ee10c38000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e6:[<0x10000000420/2608_fsm v486>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e6:[<0x10000000420/2608_fsm v1073>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.310+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000e0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3467 auth v445 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.155785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000e0:[<0x10000000015/3467 v445>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000e0:[<0x10000000015/3467 v445>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.310+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000dc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3575 auth v443 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.157785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000dc:[<0x10000000015/3575 v443>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000dc:[<0x10000000015/3575 v443>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.310+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000f3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3119 auth v449 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.136785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000f3:[<0x10000000015/3119 v449>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000f3:[<0x10000000015/3119 v449>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.310+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2656 auth v488 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10c38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e7:[<0x10000000420/2656 v488>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e7:[<0x10000000420/2656 v488>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.311+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000b6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2674 auth v451 ap=1 s=385024 n(v0 rc2024-01-11T17:58:14.062772+0000 b385024 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000b6:[<0x10000000015/2674 v451>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000b6:[<0x10000000015/2674 v451>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.311+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000b0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2668 auth v453 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.061772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000b0:[<0x10000000015/2668 v453>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000b0:[<0x10000000015/2668 v453>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.311+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000003c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603_fsm auth v455 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.058772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000003c:[<0x10000000015/2603_fsm v455>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000003c:[<0x10000000015/2603_fsm v455>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.311+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000009b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2655 auth v461 ap=1 s=32768 n(v0 rc2024-01-11T17:58:14.060772+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b86b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000009b:[<0x10000000015/2655 v461>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000009b:[<0x10000000015/2655 v461>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.575 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.311+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000101 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6110 auth v457 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.121786+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b86000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000101:[<0x10000000015/6110 v457>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000101:[<0x10000000015/6110 v457>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.311+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000df [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2702 auth v459 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.155785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b86580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000df:[<0x10000000015/2702 v459>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000df:[<0x10000000015/2702 v459>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.312+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000f7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3597 auth v463 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.131785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b87080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000f7:[<0x10000000015/3597 v463>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000f7:[<0x10000000015/3597 v463>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.312+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000e3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3712 auth v465 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.065772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b87600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000e3:[<0x10000000015/3712 v465>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000e3:[<0x10000000015/3712 v465>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.312+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000004f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608 auth v467 ap=1 s=442368 n(v0 rc2024-01-11T17:58:14.059772+0000 b442368 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b87b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000004f:[<0x10000000015/2608 v467>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000004f:[<0x10000000015/2608 v467>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.312+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000032 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2617_fsm auth v447 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.060772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000032:[<0x10000000015/2617_fsm v447>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000032:[<0x10000000015/2617_fsm v447>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.312+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000183 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187_vm auth v471 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b88680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000183:[<0x10000000015/12187_vm v471>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000183:[<0x10000000015/12187_vm v471>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.313+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000aa [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3164 auth v473 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.064772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b88c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000aa:[<0x10000000015/3164 v473>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000aa:[<0x10000000015/3164 v473>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.313+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000007d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2831 auth v469 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.242784+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b88100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000007d:[<0x10000000015/2831 v469>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000007d:[<0x10000000015/2831 v469>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.313+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000133 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3501_vm auth v414 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b89180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000133:[<0x10000000015/3501_vm v414>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000133:[<0x10000000015/3501_vm v414>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.313+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000102 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6111 auth v475 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.120786+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b89700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000102:[<0x10000000015/6111 v475>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000102:[<0x10000000015/6111 v475>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.313+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000c2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2682 auth v477 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.062772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b90000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000c2:[<0x10000000015/2682 v477>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000c2:[<0x10000000015/2682 v477>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.314+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000ef [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/113 auth v479 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.140785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b90580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000ef:[<0x10000000015/113 v479>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000ef:[<0x10000000015/113 v479>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.576 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.314+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000e5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3604 auth v481 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.065772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b90b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000e5:[<0x10000000015/3604 v481>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000e5:[<0x10000000015/3604 v481>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.314+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000d2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2691 auth v483 ap=1 s=270336 n(v0 rc2024-01-11T17:58:14.062772+0000 b270336 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b91080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000d2:[<0x10000000015/2691 v483>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000d2:[<0x10000000015/2691 v483>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.314+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000008c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3440 auth v487 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.232784+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b91b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000008c:[<0x10000000015/3440 v487>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000008c:[<0x10000000015/3440 v487>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.314+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000013c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_vm auth v485 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.058772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b91600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000013c:[<0x10000000015/2605_vm v485>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000013c:[<0x10000000015/2605_vm v485>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.314+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000a8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2663 auth v491 ap=1 s=49152 n(v0 rc2024-01-11T17:58:14.061772+0000 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b92680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000a8:[<0x10000000015/2663 v491>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000a8:[<0x10000000015/2663 v491>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.315+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000c9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2685 auth v489 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.062772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b92100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000c9:[<0x10000000015/2685 v489>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000c9:[<0x10000000015/2685 v489>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.315+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000003b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603 auth v499 ap=1 s=32768 n(v0 rc2024-01-11T17:58:14.058772+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000003b:[<0x10000000015/2603 v499>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000003b:[<0x10000000015/2603 v499>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.316+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000085 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838 auth v493 ap=1 s=417792 n(v0 rc2024-01-11T17:58:14.063772+0000 b417792 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000085:[<0x10000000015/2838 v493>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000085:[<0x10000000015/2838 v493>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.316+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000083 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2836 auth v180 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b93180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000083:[<0x10000000015/2836 v180>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000083:[<0x10000000015/2836 v180>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.316+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000027 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247 auth v497 ap=1 s=73728 n(v0 rc2024-01-11T17:58:14.056772+0000 b73728 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000027:[<0x10000000015/1247 v497>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000027:[<0x10000000015/1247 v497>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.316+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000ee [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/548 auth v495 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.140785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b93700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000ee:[<0x10000000015/548 v495>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000ee:[<0x10000000015/548 v495>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.316+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000130 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394_vm auth v505 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.064772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000130:[<0x10000000015/3394_vm v505>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000130:[<0x10000000015/3394_vm v505>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.316+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000f9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3080 auth v503 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.064772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000f9:[<0x10000000015/3080 v503>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000f9:[<0x10000000015/3080 v503>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.577 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.317+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000064 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764 auth v501 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000064:[<0x10000000015/3764 v501>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000064:[<0x10000000015/3764 v501>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.317+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000179 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12182_fsm auth v511 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.065772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000179:[<0x10000000015/12182_fsm v511>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000179:[<0x10000000015/12182_fsm v511>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.317+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000ce [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2754 auth v507 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.063772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000ce:[<0x10000000015/2754 v507>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000ce:[<0x10000000015/2754 v507>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.317+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000a9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3455 auth v509 ap=1 s=40960 n(v0 rc2024-01-11T17:58:14.064772+0000 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000a9:[<0x10000000015/3455 v509>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000a9:[<0x10000000015/3455 v509>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.317+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000016e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12199 auth v429 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000016e:[<0x10000000015/12199 v429>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000016e:[<0x10000000015/12199 v429>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.318+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000002b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259 auth v513 ap=1 s=90112 n(v0 rc2024-01-11T17:58:14.365767+0000 b90112 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000002b:[<0x10000000015/1259 v513>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000002b:[<0x10000000015/1259 v513>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.318+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000173 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12207 auth v515 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b9f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000173:[<0x10000000015/12207 v515>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000173:[<0x10000000015/12207 v515>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.318+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000030 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610 auth v517 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.059772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ba8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000030:[<0x10000000015/2610 v517>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000030:[<0x10000000015/2610 v517>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.318+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000e7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3606 auth v519 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.065772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ba8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000e7:[<0x10000000015/3606 v519>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000e7:[<0x10000000015/3606 v519>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.318+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000040 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2613 auth v72 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10ba8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000040:[<0x10000000015/2613 v72>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000040:[<0x10000000015/2613 v72>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.318+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000038 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601_fsm auth v440 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.058772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10998c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000038:[<0x10000000015/2601_fsm v440>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000038:[<0x10000000015/2601_fsm v440>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.319+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000fb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3542 auth v442 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.064772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10999700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000fb:[<0x10000000015/3542 v442>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000fb:[<0x10000000015/3542 v442>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.319+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000de [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2701 auth v444 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.156785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10a4b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000de:[<0x10000000015/2701 v444>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000de:[<0x10000000015/2701 v444>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.578 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.319+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000005e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603 auth v448 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.064772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b3e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000005e:[<0x10000000015/3603 v448>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000005e:[<0x10000000015/3603 v448>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.319+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000119 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764_vm auth v450 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b3eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000119:[<0x10000000015/3764_vm v450>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000119:[<0x10000000015/3764_vm v450>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.319+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000011a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3601_vm auth v452 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.064772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b3f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000011a:[<0x10000000015/3601_vm v452>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000011a:[<0x10000000015/3601_vm v452>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.319+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000003a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_fsm auth v446 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.058772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b3e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000003a:[<0x10000000015/2602_fsm v446>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000003a:[<0x10000000015/2602_fsm v446>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.320+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000089 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2840 auth v454 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.063772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b40100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000089:[<0x10000000015/2840 v454>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000089:[<0x10000000015/2840 v454>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.320+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000010b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3350_vm auth v408 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b3fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000010b:[<0x10000000015/3350_vm v408>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000010b:[<0x10000000015/3350_vm v408>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.320+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000131 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2611_vm auth v422 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b40680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000131:[<0x10000000015/2611_vm v422>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000131:[<0x10000000015/2611_vm v422>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.320+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000047 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3466 auth v88 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b40c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000047:[<0x10000000015/3466 v88>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000047:[<0x10000000015/3466 v88>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.320+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000d3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2692 auth v458 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.062772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b41700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000d3:[<0x10000000015/2692 v458>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000d3:[<0x10000000015/2692 v458>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.321+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000012d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2995_vm auth v420 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000012d:[<0x10000000015/2995_vm v420>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000012d:[<0x10000000015/2995_vm v420>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.321+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000165 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12184 auth v430 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b3f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000165:[<0x10000000015/12184 v430>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000165:[<0x10000000015/12184 v430>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.321+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000f0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/549 auth v462 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.139785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000f0:[<0x10000000015/549 v462>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000f0:[<0x10000000015/549 v462>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.321+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000069 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418 auth v134 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000069:[<0x10000000015/1418 v134>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000069:[<0x10000000015/1418 v134>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.579 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.321+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000c5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3380 auth v456 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.183785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b41180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000c5:[<0x10000000015/3380 v456>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000c5:[<0x10000000015/3380 v456>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.322+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000103 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6112 auth v460 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.119786+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000103:[<0x10000000015/6112 v460>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000103:[<0x10000000015/6112 v460>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.322+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000178 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12216 auth v438 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000178:[<0x10000000015/12216 v438>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000178:[<0x10000000015/12216 v438>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.322+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000029 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1249 auth v466 ap=1 s=393216 n(v0 rc2024-01-11T17:58:14.057772+0000 b393216 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000029:[<0x10000000015/1249 v466>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000029:[<0x10000000015/1249 v466>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.322+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000004a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_fsm auth v464 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.058772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000004a:[<0x10000000015/2605_fsm v464>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000004a:[<0x10000000015/2605_fsm v464>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.322+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000153 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/PG_VERSION auth v428 ap=1 s=3 n(v0 b3 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000153:[<0x10000000015/PG_VERSION v428>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000153:[<0x10000000015/PG_VERSION v428>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.323+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000172 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12206 auth v434 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.723777+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000172:[<0x10000000015/12206 v434>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000172:[<0x10000000015/12206 v434>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.323+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000115 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418_vm auth v412 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000115:[<0x10000000015/1418_vm v412>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000115:[<0x10000000015/1418_vm v412>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.323+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000076 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576 auth v156 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b4d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000076:[<0x10000000015/3576 v156>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000076:[<0x10000000015/3576 v156>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.323+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000010e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3541_vm auth v468 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.064772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000010e:[<0x10000000015/3541_vm v468>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000010e:[<0x10000000015/3541_vm v468>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.323+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000081 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2834 auth v176 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000081:[<0x10000000015/2834 v176>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000081:[<0x10000000015/2834 v176>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.323+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000109 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6106_vm auth v406 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b52b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000109:[<0x10000000015/6106_vm v406>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000109:[<0x10000000015/6106_vm v406>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.580 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.324+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000006b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3256 auth v138 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000006b:[<0x10000000015/3256 v138>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000006b:[<0x10000000015/3256 v138>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.324+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000097 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2652 auth v470 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.060772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000097:[<0x10000000015/2652 v470>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000097:[<0x10000000015/2652 v470>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.324+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000142 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_vm auth v472 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.058772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b53b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000142:[<0x10000000015/2602_vm v472>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000142:[<0x10000000015/2602_vm v472>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.324+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000b9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3502 auth v474 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.195784+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000b9:[<0x10000000015/3502 v474>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000b9:[<0x10000000015/3502 v474>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.324+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000008a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2841 auth v478 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.063772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b54c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000008a:[<0x10000000015/2841 v478>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000008a:[<0x10000000015/2841 v478>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.325+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000006e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394 auth v476 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.064772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b54680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000006e:[<0x10000000015/3394 v476>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000006e:[<0x10000000015/3394 v476>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.325+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000002c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_fsm auth v480 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.058772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b55180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000002c:[<0x10000000015/1259_fsm v480>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000002c:[<0x10000000015/1259_fsm v480>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.325+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000011d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3602_vm auth v482 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.064772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b55700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000011d:[<0x10000000015/3602_vm v482>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000011d:[<0x10000000015/3602_vm v482>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.325+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000096 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2651 auth v484 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.060772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000096:[<0x10000000015/2651 v484>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000096:[<0x10000000015/2651 v484>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.581 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T17:58:42.325+0000 7f7b4ce97700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000b5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2673 auth v486 ap=1 s=327680 n(v0 rc2024-01-11T17:58:14.061772+0000 b327680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ee10b6a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000b5:[<0x10000000015/2673 v486>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000b5:[<0x10000000015/2673 v486>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.559+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e21f90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009490 tx=0x7fb5dc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e07f4168034d4ee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.560+0000 7fb5e1ffb700 1 -- 172.21.15.7:0/1124651152 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc003c00 con 0x55c243e23d80 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.561+0000 7fb5e97fa700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5d80a20e0 0x7fb5d80a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.561+0000 7fb5e97fa700 1 -- 172.21.15.7:0/2975687678 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5e40ce8e0 con 0x7fb5ec1072d0 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.561+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5d80a20e0 0x7fb5d80a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.562+0000 7fb5e1ffb700 1 -- 172.21.15.7:0/1124651152 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5dc00f040 con 0x55c243e23d80 2024-01-11T17:58:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.562+0000 7fb5e1ffb700 1 -- 172.21.15.7:0/1124651152 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc003c00 con 0x55c243e23d80 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 -- 172.21.15.7:0/1124651152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 msgr2=0x55c243e21f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e21f90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009490 tx=0x7fb5dc009920 comp rx=0 tx=0).stop 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 -- 172.21.15.7:0/1124651152 shutdown_connections 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e23820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1124651152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e21f90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.563+0000 7fb605d52b80 1 -- 172.21.15.7:0/1124651152 >> 172.21.15.7:0/1124651152 conn(0x55c243d977a0 msgr2=0x55c243e1cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.564+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5d80a20e0 0x7fb5d80a45a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55c243e21f20 tx=0x7fb5d4009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.565+0000 7fb605d52b80 1 -- 172.21.15.7:0/1124651152 shutdown_connections 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.565+0000 7fb605d52b80 1 -- 172.21.15.7:0/1124651152 wait complete. 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.566+0000 7fb605d52b80 1 -- start start 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.566+0000 7fb605d52b80 1 Processor -- start 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e22570 0x55c243e2be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23400 0x55c243e2c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243d75d30 con 0x55c243e23400 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243ce43a0 con 0x55c243e22570 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243ce4040 con 0x55c243e23d80 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.567+0000 7fb605d52b80 10 client.0 _ll_register_callbacks cb 0x55c243d16720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e333e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54748/0 (socket says 172.21.15.7:54748) 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 -- 172.21.15.7:0/3859370259 learned_addr learned my addr 172.21.15.7:0/3859370259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e22570 0x55c243e2be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23400 0x55c243e2c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 -- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e22570 msgr2=0x55c243e2be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e22570 0x55c243e2be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 -- 172.21.15.7:0/3859370259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23400 msgr2=0x55c243e2c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23400 0x55c243e2c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.568+0000 7fb5ebfff700 1 -- 172.21.15.7:0/3859370259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc009170 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.569+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23400 0x55c243e2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.569+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e333e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400fbc0 tx=0x7fb5e400fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b622f066873b22f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.569+0000 7fb5caffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.569+0000 7fb5caffd700 1 -- 172.21.15.7:0/3859370259 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e409f490 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.569+0000 7fb5caffd700 1 -- 172.21.15.7:0/3859370259 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5e40a3640 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.569+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c243e15010 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.570+0000 7fb5caffd700 1 -- 172.21.15.7:0/3859370259 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e409f170 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.570+0000 7fb605d52b80 10 client.25699 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.570+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c243b96b60 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.570+0000 7fb5caffd700 1 -- 172.21.15.7:0/3859370259 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5e40b7020 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.571+0000 7fb5caffd700 1 -- 172.21.15.7:0/3859370259 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb5e40b9290 con 0x55c243e23d80 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.571+0000 7fb605d52b80 10 client.25699 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.571+0000 7fb605d52b80 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.133+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d/ auth v4 ap=1 f() n(v0 rc2024-01-11T17:56:35.344163+0000 1=0+1) (inest lock) (iversion lock) | dirfrag=0 replicated=0 authpin=1 scrubqueue=0 0x55fb5b16d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000003:[<0x10000000002/5ef7e487-9779-443a-aaca-b2f24334f17d v4>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[]","memoryvalue":"(2)0x10000000003:[<0x10000000002/5ef7e487-9779-443a-aaca-b2f24334f17d v4>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 rc2024-01-11T17:56:35.344163+0000 1=0+1)","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.133+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [2,head] /volumes/qa/sv_0/.meta auth v14 ap=1 s=117 n(v0 rc2024-01-11T17:56:35.352871+0000 b117 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55fb5c198000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000004:[<0x10000000002/.meta v14>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000004:[<0x10000000002/.meta v14>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.178+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000520 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/log/postgresql-Thu.log auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c199080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000520:[<0x1000000051d/postgresql-Thu.log v2>,<0x10000000006/log v464>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000520:[<0x1000000051d/postgresql-Thu.log v2>,<0x10000000006/log v464>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.265+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000e9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3766 auth v466 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.065772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000e9:[<0x10000000015/3766 v466>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000e9:[<0x10000000015/3766 v466>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.265+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000a7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2662 auth v468 ap=1 s=32768 n(v0 rc2024-01-11T17:58:14.061772+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000a7:[<0x10000000015/2662 v468>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000a7:[<0x10000000015/2662 v468>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.266+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000fc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3257 auth v470 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.126785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000fc:[<0x10000000015/3257 v470>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000fc:[<0x10000000015/3257 v470>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.266+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000180 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192_vm auth v472 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000180:[<0x10000000015/12192_vm v472>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000180:[<0x10000000015/12192_vm v472>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.591 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.266+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000037 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601 auth v476 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.058772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000037:[<0x10000000015/2601 v476>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000037:[<0x10000000015/2601 v476>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.571+0000 7fb605d52b80 20 client.25699 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:42.592 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.267+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000128 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753_vm auth v478 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.063772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000128:[<0x10000000015/2753_vm v478>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000128:[<0x10000000015/2753_vm v478>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.592 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.267+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000013b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_vm auth v480 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.058772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000013b:[<0x10000000015/1259_vm v480>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000013b:[<0x10000000015/1259_vm v480>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.592 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.267+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000104 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6113 auth v474 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.117786+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000104:[<0x10000000015/6113 v474>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000104:[<0x10000000015/6113 v474>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.592 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.268+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000004e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2607 auth v482 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.058772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000004e:[<0x10000000015/2607 v482>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000004e:[<0x10000000015/2607 v482>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.592 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.268+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000015e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2609_fsm auth v486 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.059772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000015e:[<0x10000000015/2609_fsm v486>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000015e:[<0x10000000015/2609_fsm v486>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.593 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.269+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000009c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2757 auth v484 ap=1 s=32768 n(v0 rc2024-01-11T17:58:14.063772+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000009c:[<0x10000000015/2757 v484>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000009c:[<0x10000000015/2757 v484>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.593 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.269+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000171 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12204 auth v452 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000171:[<0x10000000015/12204 v452>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000171:[<0x10000000015/12204 v452>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.593 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.269+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000082 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2835 auth v488 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.240784+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000082:[<0x10000000015/2835 v488>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000082:[<0x10000000015/2835 v488>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.571+0000 7fb605d52b80 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e37960 0x55c243e39d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.572+0000 7fb605d52b80 4 client.25699 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.572+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c243d6ebc0 con 0x55c243e37960 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.270+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000114 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3118_vm auth v408 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000114:[<0x10000000015/3118_vm v408>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000114:[<0x10000000015/3118_vm v408>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.271+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000012c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3381_vm auth v414 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000012c:[<0x10000000015/3381_vm v414>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000012c:[<0x10000000015/3381_vm v414>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.271+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000068 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1417 auth v132 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000068:[<0x10000000015/1417 v132>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000068:[<0x10000000015/1417 v132>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.271+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000084 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2837 auth v490 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.238784+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000084:[<0x10000000015/2837 v490>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000084:[<0x10000000015/2837 v490>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.272+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000111 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3596_vm auth v406 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c422000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000111:[<0x10000000015/3596_vm v406>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000111:[<0x10000000015/3596_vm v406>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.272+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000d5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/5002 auth v494 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.164785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000d5:[<0x10000000015/5002 v494>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000d5:[<0x10000000015/5002 v494>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.273+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000087 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3598 auth v188 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c422580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000087:[<0x10000000015/3598 v188>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000087:[<0x10000000015/3598 v188>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.273+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000a5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2660 auth v496 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.061772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000a5:[<0x10000000015/2660 v496>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000a5:[<0x10000000015/2660 v496>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.273+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000167 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187 auth v492 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c41cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000167:[<0x10000000015/12187 v492>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000167:[<0x10000000015/12187 v492>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.274+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000ad [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2665 auth v500 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.061772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c423080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000ad:[<0x10000000015/2665 v500>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000ad:[<0x10000000015/2665 v500>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.594 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.276+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000028 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247_fsm auth v498 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.056772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c422b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000028:[<0x10000000015/1247_fsm v498>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000028:[<0x10000000015/1247_fsm v498>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.574+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e37960 0x55c243e39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.595 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.276+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000176 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12212 auth v456 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c423b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000176:[<0x10000000015/12212 v456>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000176:[<0x10000000015/12212 v456>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.595 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.279+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000bd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2679 auth v502 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.062772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c423600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000bd:[<0x10000000015/2679 v502>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000bd:[<0x10000000015/2679 v502>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.595 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.280+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000132 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610_vm auth v508 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.059772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c425180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000132:[<0x10000000015/2610_vm v508>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000132:[<0x10000000015/2610_vm v508>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.596 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.280+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000174 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12209 auth v454 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c424680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000174:[<0x10000000015/12209 v454>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000174:[<0x10000000015/12209 v454>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.596 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.281+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000c7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3379 auth v506 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.181785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c424c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000c7:[<0x10000000015/3379 v506>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000c7:[<0x10000000015/3379 v506>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.596 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.281+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000000c3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2683 auth v510 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.184785+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c425700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000000c3:[<0x10000000015/2683 v510>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000000c3:[<0x10000000015/2683 v510>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.596 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.282+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000016c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12196 auth v450 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c42e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000016c:[<0x10000000015/12196 v450>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000016c:[<0x10000000015/12196 v450>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.596 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.283+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000033 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753 auth v512 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.063772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c42e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000033:[<0x10000000015/2753 v512>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000033:[<0x10000000015/2753 v512>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.574+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e37960 0x55c243e39d80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471292130c1f41fe server_cookie=85c92c091db1aa60 in_seq=0 out_seq=0 2024-01-11T17:58:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.574+0000 7fb5caffd700 10 client.25699 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.283+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000016a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192 auth v514 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.065772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c42eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000016a:[<0x10000000015/12192 v514>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000016a:[<0x10000000015/12192 v514>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.284+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000014c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_vm auth v518 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.063772+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c42f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000014c:[<0x10000000015/2838_vm v518>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000014c:[<0x10000000015/2838_vm v518>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.285+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000158 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_fsm auth v516 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.063772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c42f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000158:[<0x10000000015/2838_fsm v516>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000158:[<0x10000000015/2838_fsm v516>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.285+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000017a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2619_fsm auth v520 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.060772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c42fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000017a:[<0x10000000015/2619_fsm v520>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000017a:[<0x10000000015/2619_fsm v520>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.286+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000071 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3456 auth v504 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.064772+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c424100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000071:[<0x10000000015/3456 v504>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000071:[<0x10000000015/3456 v504>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.286+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000157 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608_fsm auth v524 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.059772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c430c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000157:[<0x10000000015/2608_fsm v524>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000157:[<0x10000000015/2608_fsm v524>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.287+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000120 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576_vm auth v410 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c430680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000120:[<0x10000000015/3576_vm v410>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000120:[<0x10000000015/3576_vm v410>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.288+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000036 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2616_fsm auth v522 ap=1 s=24576 n(v0 rc2024-01-11T17:58:14.060772+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c430100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000036:[<0x10000000015/2616_fsm v522>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000036:[<0x10000000015/2616_fsm v522>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.288+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000009d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2656 auth v526 ap=1 s=8192 n(v0 rc2024-01-11T17:58:13.218784+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c431180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000009d:[<0x10000000015/2656 v526>,<0x10000000014/1 v3300>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000009d:[<0x10000000015/2656 v526>,<0x10000000014/1 v3300>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.597 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.290+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_fsm auth v494 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e9:[<0x10000000420/3456_fsm v494>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e9:[<0x10000000420/3456_fsm v494>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.575+0000 7fb5caffd700 1 -- 172.21.15.7:0/3859370259 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c243d6ebc0 con 0x55c243e37960 2024-01-11T17:58:42.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.575+0000 7fb5caffd700 10 client.25699 handle_command_reply: tid=0 2024-01-11T17:58:42.598 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.290+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ea [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202 auth v496 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ea:[<0x10000000420/12202 v496>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ea:[<0x10000000420/12202 v496>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.598 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.290+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002eb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_vm auth v1087 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55fb5c3c5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002eb:[<0x10000000420/1247_vm v526>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002eb:[<0x10000000420/1247_vm v1087>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.598 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.291+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ed [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_fsm auth v530 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ed:[<0x10000000420/3764_fsm v530>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ed:[<0x10000000420/3764_fsm v530>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.598 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.291+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ee [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_vm auth v1089 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@3},l=15405 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55fb5c3c6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ee:[<0x10000000420/2608_vm v532>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ee:[<0x10000000420/2608_vm v1089>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.598 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.292+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ef [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3767 auth v534 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ef:[<0x10000000420/3767 v534>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ef:[<0x10000000420/3767 v534>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.292+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f1 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2669 auth v540 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f1:[<0x10000000420/2669 v540>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f1:[<0x10000000420/2669 v540>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.292+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f2 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995 auth v114 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f2:[<0x10000000420/2995 v114>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f2:[<0x10000000420/2995 v114>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.292+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f3 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3503 auth v568 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f3:[<0x10000000420/3503 v568>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f3:[<0x10000000420/3503 v568>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.293+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12201 auth v572 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f4:[<0x10000000420/12201 v572>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f4:[<0x10000000420/12201 v572>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.294+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f5 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607_fsm auth v574 ap=1 s=24576 n(v0 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f5:[<0x10000000420/2607_fsm v574>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f5:[<0x10000000420/2607_fsm v574>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.294+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2615 auth v1082 ap=1 s=8192 n(v0 b8192 1=1+0) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55fb5c3c4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e8:[<0x10000000420/2615 v490>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002e8:[<0x10000000420/2615 v1082>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.575+0000 7fb605d52b80 1 client.25699 shutdown 2024-01-11T17:58:42.600 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.295+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f7 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2658 auth v578 ap=1 s=106496 n(v0 b106496 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f7:[<0x10000000420/2658 v578>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f7:[<0x10000000420/2658 v578>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.600 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.295+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ec [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605 auth v528 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ec:[<0x10000000420/2605 v528>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ec:[<0x10000000420/2605 v528>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.600 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.295+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f0 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_vm auth v538 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f0:[<0x10000000420/3456_vm v538>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f0:[<0x10000000420/3456_vm v538>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.576+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 msgr2=0x55c243e333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.576+0000 7fb605d52b80 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e333e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400fbc0 tx=0x7fb5e400fef0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.296+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f6 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2699 auth v576 ap=1 s=8192 n(v0 rc2024-01-11T17:58:14.534765+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f6:[<0x10000000420/2699 v576>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f6:[<0x10000000420/2699 v576>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.296+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f8 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612_vm auth v582 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3c9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f8:[<0x10000000420/2612_vm v582>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f8:[<0x10000000420/2612_vm v582>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.296+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f9 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/828 auth v586 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3ca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f9:[<0x10000000420/828 v586>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002f9:[<0x10000000420/828 v586>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.296+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fa [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202_vm auth v638 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3ca680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fa:[<0x10000000420/12202_vm v638>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002fa:[<0x10000000420/12202_vm v638>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.296+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fb [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3605 auth v640 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3cac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fb:[<0x10000000420/3605 v640>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002fb:[<0x10000000420/3605 v640>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.297+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fc [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_vm auth v642 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3cb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fc:[<0x10000000420/2616_vm v642>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002fc:[<0x10000000420/2616_vm v642>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.297+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fe [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2666 auth v646 ap=1 s=16384 n(v0 rc2024-01-11T17:58:14.536765+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fe:[<0x10000000420/2666 v646>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002fe:[<0x10000000420/2666 v646>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.297+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000300 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826 auth v148 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000300:[<0x10000000420/826 v148>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000300:[<0x10000000420/826 v148>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 shutdown_connections 2024-01-11T17:58:42.602 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.297+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000301 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_vm auth v656 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000301:[<0x10000000420/2617_vm v656>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000301:[<0x10000000420/2617_vm v656>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.602 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.298+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fd [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3609 auth v644 ap=1 s=32768 n(v0 rc2024-01-11T17:58:14.535765+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3cb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fd:[<0x10000000420/3609 v644>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002fd:[<0x10000000420/3609 v644>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb605d52b80 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e37960 0x55c243e39d80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb605d52b80 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23d80 0x55c243e333e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb605d52b80 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e22570 0x55c243e2be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb605d52b80 1 --2- 172.21.15.7:0/3859370259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23400 0x55c243e2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 >> 172.21.15.7:0/3859370259 conn(0x55c243d977a0 msgr2=0x55c243e20e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.577+0000 7fb5caffd700 0 client.25699 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.578+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 shutdown_connections 2024-01-11T17:58:42.603 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.298+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000302 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_vm auth v662 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000302:[<0x10000000420/2601_vm v662>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000302:[<0x10000000420/2601_vm v662>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.603 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.299+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000303 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541 auth v700 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000303:[<0x10000000420/3541 v700>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000303:[<0x10000000420/3541 v700>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.603 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.299+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000306 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2604 auth v162 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000306:[<0x10000000420/2604 v162>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000306:[<0x10000000420/2604 v162>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.604 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.299+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000305 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2688 auth v704 ap=1 s=40960 n(v0 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000305:[<0x10000000420/2688 v704>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000305:[<0x10000000420/2688 v704>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.604 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.300+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000304 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3600 auth v702 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000304:[<0x10000000420/3600 v702>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000304:[<0x10000000420/3600 v702>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.604 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.300+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000307 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439_vm auth v164 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000307:[<0x10000000420/3439_vm v164>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000307:[<0x10000000420/3439_vm v164>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.604 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.301+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000309 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255 auth v708 ap=1 s=606208 n(v0 rc2024-01-11T17:58:14.539765+0000 b606208 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000309:[<0x10000000420/1255 v708>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000309:[<0x10000000420/1255 v708>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.604 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.301+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ff [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_vm auth v648 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ff:[<0x10000000420/2603_vm v648>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000002ff:[<0x10000000420/2603_vm v648>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.578+0000 7fb605d52b80 1 -- 172.21.15.7:0/3859370259 wait complete. 2024-01-11T17:58:42.605 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.301+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030a [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612 auth v710 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030a:[<0x10000000420/2612 v710>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000030a:[<0x10000000420/2612 v710>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.605 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.301+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000308 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609 auth v706 ap=1 s=286720 n(v0 rc2024-01-11T17:58:14.538765+0000 b286720 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000308:[<0x10000000420/2609 v706>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000308:[<0x10000000420/2609 v706>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.605 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.302+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030d [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834_vm auth v178 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030d:[<0x10000000420/2834_vm v178>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000030d:[<0x10000000420/2834_vm v178>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.605 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.302+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030e [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3395 auth v716 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030e:[<0x10000000420/3395 v716>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000030e:[<0x10000000420/3395 v716>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.605 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.303+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030f [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2675 auth v718 ap=1 s=180224 n(v0 b180224 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3da100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030f:[<0x10000000420/2675 v718>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000030f:[<0x10000000420/2675 v718>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.303+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030c [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602 auth v714 ap=1 s=49152 n(v0 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030c:[<0x10000000420/2602 v714>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000030c:[<0x10000000420/2602 v714>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.303+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000310 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12186 auth v720 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3da680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000310:[<0x10000000420/12186 v720>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000310:[<0x10000000420/12186 v720>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.304+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000312 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602 auth v724 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3db180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000312:[<0x10000000420/3602 v724>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000312:[<0x10000000420/3602 v724>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.305+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000311 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/174 auth v722 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3dac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000311:[<0x10000000420/174 v722>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000311:[<0x10000000420/174 v722>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.305+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030b [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2650 auth v712 ap=1 s=16384 n(v0 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3d8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030b:[<0x10000000420/2650 v712>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x1000000030b:[<0x10000000420/2650 v712>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.305+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f4 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16396 auth v1066 ap=1 s=292143104 n(v0 rc2024-01-11T17:58:22.673645+0000 b292143104 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-587202560@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55fb5c3e6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f4:[<0x10000000420/16396 v1064>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x100000003f4:[<0x10000000420/16396 v1066>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.306+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000313 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224 auth v190 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15405 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3db700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000313:[<0x10000000420/2224 v190>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000313:[<0x10000000420/2224 v190>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T17:58:42.306+0000 7f17cb1ef700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000314 [2,head] /volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2689 auth v726 ap=1 s=49152 n(v0 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15405=0-4194304@1} caps={15405=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15405 | request=0 lock=0 importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55fb5c3e6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000314:[<0x10000000420/2689 v726>,<0x10000000014/12347 v3341>,<0x10000000006/base v615>,<0x10000000005/data v121>,<0x100000001fb/postgres v23>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v37>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v45>,<0x1/volumes v41>]//[3]","memoryvalue":"(4)0x10000000314:[<0x10000000420/2689 v726>,<0x10000000014/12347 v3401>,<0x10000000006/base v660>,<0x10000000005/data v150>,<0x100000001fb/postgres v42>,<0x100000001fa/09d4c4b1-901d-4a41-b788-7617dcac494e v54>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v25>,<0x1/volumes v55>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-01-11T17:58:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.579+0000 7fb605d52b80 20 client.25699 trim_cache size 0 max 16384 2024-01-11T17:58:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.608+0000 7fb605d52b80 1 Processor -- start 2024-01-11T17:58:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 -- start start 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e39d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23400 0x55c243e333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 0x55c243e22950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243bec6a0 con 0x55c243e22570 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243e37780 con 0x55c243e23d80 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb605d52b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243e19810 con 0x55c243e23400 2024-01-11T17:58:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 0x55c243e22950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb5ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 0x55c243e22950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51796/0 (socket says 172.21.15.7:51796) 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.609+0000 7fb5ebfff700 1 -- 172.21.15.7:0/1047954638 learned_addr learned my addr 172.21.15.7:0/1047954638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5ebfff700 1 -- 172.21.15.7:0/1047954638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23400 msgr2=0x55c243e333a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23400 0x55c243e333a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5ebfff700 1 -- 172.21.15.7:0/1047954638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 msgr2=0x55c243e39d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e39d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5ebfff700 1 -- 172.21.15.7:0/1047954638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c243d37230 con 0x55c243e23d80 2024-01-11T17:58:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.610+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 0x55c243e22950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5e409b2d0 tx=0x7fb5e409b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695bab5e5d329922 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.611+0000 7fb5e17fa700 1 -- 172.21.15.7:0/1047954638 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40a3640 con 0x55c243e23d80 2024-01-11T17:58:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.611+0000 7fb5e17fa700 1 -- 172.21.15.7:0/1047954638 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5e40b7070 con 0x55c243e23d80 2024-01-11T17:58:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.611+0000 7fb5e17fa700 1 -- 172.21.15.7:0/1047954638 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40a15d0 con 0x55c243e23d80 2024-01-11T17:58:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 -- 172.21.15.7:0/1047954638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 msgr2=0x55c243e22950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 0x55c243e22950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5e409b2d0 tx=0x7fb5e409b880 comp rx=0 tx=0).stop 2024-01-11T17:58:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 -- 172.21.15.7:0/1047954638 shutdown_connections 2024-01-11T17:58:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23d80 0x55c243e22950 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e22570 0x55c243e39d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 --2- 172.21.15.7:0/1047954638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e23400 0x55c243e333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.612+0000 7fb605d52b80 1 -- 172.21.15.7:0/1047954638 >> 172.21.15.7:0/1047954638 conn(0x55c243d977a0 msgr2=0x55c243e1cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.614+0000 7fb605d52b80 1 -- 172.21.15.7:0/1047954638 shutdown_connections 2024-01-11T17:58:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.615+0000 7fb605d52b80 1 -- 172.21.15.7:0/1047954638 wait complete. 2024-01-11T17:58:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.616+0000 7fb605d52b80 1 -- start start 2024-01-11T17:58:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.617+0000 7fb605d52b80 1 Processor -- start 2024-01-11T17:58:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.618+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e22570 0x55c243e30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.618+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e30c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.618+0000 7fb605d52b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23d80 0x55c243e31c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.618+0000 7fb605d52b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243e321a0 con 0x55c243e23d80 2024-01-11T17:58:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.618+0000 7fb605d52b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243e32310 con 0x55c243e23400 2024-01-11T17:58:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.618+0000 7fb605d52b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243e324d0 con 0x55c243e22570 2024-01-11T17:58:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.619+0000 7fb605d52b80 10 client.0 _ll_register_callbacks cb 0x55c243d16720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.619+0000 7fb5eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e22570 0x55c243e30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.620+0000 7fb5e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e30c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.620+0000 7fb5e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e30c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51808/0 (socket says 172.21.15.7:51808) 2024-01-11T17:58:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.620+0000 7fb5e2ffd700 1 -- 172.21.15.7:0/720945036 learned_addr learned my addr 172.21.15.7:0/720945036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.621+0000 7fb5e2ffd700 1 -- 172.21.15.7:0/720945036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e22570 msgr2=0x55c243e30710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.621+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e22570 0x55c243e30710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.621+0000 7fb5e2ffd700 1 -- 172.21.15.7:0/720945036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23d80 msgr2=0x55c243e31c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.621+0000 7fb5ebfff700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23d80 0x55c243e31c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.621+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23d80 0x55c243e31c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.621+0000 7fb5e2ffd700 1 -- 172.21.15.7:0/720945036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c243b96b60 con 0x55c243e23400 2024-01-11T17:58:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.622+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e22570 0x55c243e30710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:58:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.622+0000 7fb5e2ffd700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e30c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4013f70 tx=0x7fb5d4009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08705b07ef3a85f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.622+0000 7fb5caffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.623+0000 7fb5caffd700 1 -- 172.21.15.7:0/720945036 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4010e70 con 0x55c243e23400 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.623+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5dc00f040 con 0x55c243e23400 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.623+0000 7fb5caffd700 1 -- 172.21.15.7:0/720945036 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5d4029410 con 0x55c243e23400 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.623+0000 7fb5caffd700 1 -- 172.21.15.7:0/720945036 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d40295d0 con 0x55c243e23400 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.623+0000 7fb605d52b80 10 client.25667 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.623+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c243beecd0 con 0x55c243e23400 2024-01-11T17:58:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.624+0000 7fb5caffd700 1 -- 172.21.15.7:0/720945036 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5dc00f040 con 0x55c243e23400 2024-01-11T17:58:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.624+0000 7fb5caffd700 1 -- 172.21.15.7:0/720945036 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb5d4028570 con 0x55c243e23400 2024-01-11T17:58:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb605d52b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb605d52b80 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb605d52b80 20 client.25667 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb605d52b80 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e9f490 0x55c243ea18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb605d52b80 4 client.25667 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-01-11T17:58:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c243d6e8e0 con 0x55c243e9f490 2024-01-11T17:58:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.625+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e9f490 0x55c243ea18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.626+0000 7fb5eb7fe700 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e9f490 0x55c243ea18b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9ad9dff84261c8 server_cookie=e32517e1a3f506d0 in_seq=0 out_seq=0 2024-01-11T17:58:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.628+0000 7fb5caffd700 10 client.25667 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.628+0000 7fb5caffd700 1 -- 172.21.15.7:0/720945036 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c243d6e8e0 con 0x55c243e9f490 2024-01-11T17:58:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.628+0000 7fb5caffd700 10 client.25667 handle_command_reply: tid=0 2024-01-11T17:58:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.628+0000 7fb605d52b80 1 client.25667 shutdown 2024-01-11T17:58:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.629+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 msgr2=0x55c243e30c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.629+0000 7fb605d52b80 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e30c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4013f70 tx=0x7fb5d4009290 comp rx=0 tx=0).stop 2024-01-11T17:58:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.629+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 shutdown_connections 2024-01-11T17:58:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.629+0000 7fb605d52b80 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55c243e9f490 0x55c243ea18b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.629+0000 7fb605d52b80 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c243e22570 0x55c243e30710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.630+0000 7fb5caffd700 0 client.25667 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.630+0000 7fb605d52b80 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c243e23d80 0x55c243e31c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.630+0000 7fb605d52b80 1 --2- 172.21.15.7:0/720945036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c243e23400 0x55c243e30c50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.630+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 >> 172.21.15.7:0/720945036 conn(0x55c243d977a0 msgr2=0x55c243e26a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.630+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 shutdown_connections 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.630+0000 7fb605d52b80 1 -- 172.21.15.7:0/720945036 wait complete. 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.631+0000 7fb605d52b80 20 client.25667 trim_cache size 0 max 16384 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stdout: "91b694ce-99ef-4fe7-adda-7010f038c107": { 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-01-11T17:58:42.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "91b694ce-99ef-4fe7-adda-7010f038c107", 2024-01-11T17:58:42.633 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-01-11T17:58:42.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:58:42.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:58:42.633 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:58:42.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5d80a20e0 msgr2=0x7fb5d80a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5d80a20e0 0x7fb5d80a45a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55c243e21f20 tx=0x7fb5d4009530 comp rx=0 tx=0).stop 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1072d0 msgr2=0x7fb5ec1a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1a5030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40098e0 tx=0x7fb5e400f690 comp rx=0 tx=0).stop 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 shutdown_connections 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5d80a20e0 0x7fb5d80a45a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5ec1072d0 0x7fb5ec1a5030 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5ec102070 0x7fb5ec19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 --2- 172.21.15.7:0/2975687678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5ec1049a0 0x7fb5ec19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.632+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 >> 172.21.15.7:0/2975687678 conn(0x7fb5ec0fba00 msgr2=0x7fb5ec1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.633+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 shutdown_connections 2024-01-11T17:58:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:58:42.633+0000 7fb5f1fff700 1 -- 172.21.15.7:0/2975687678 wait complete. 2024-01-11T17:58:42.672 INFO:tasks.cephfs.filesystem:scrub status for tag:91b694ce-99ef-4fe7-adda-7010f038c107 - {'path': '/', 'tag': '91b694ce-99ef-4fe7-adda-7010f038c107', 'options': 'recursive,force'} 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub queued for path: / 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: active paths [/] 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: active paths [/] 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000003 (/volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000004 (/volumes/qa/sv_0/.meta) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:42.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: active paths [/] 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: active paths [/] 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000520 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/log/postgresql-Thu.log) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3766) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2662) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000fc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3257) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000180 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000037 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000128 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000013b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000104 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6113) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000004e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2607) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000015e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2609_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000009c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2757) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000171 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12204) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000082 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2835) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000114 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3118_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000012c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3381_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000068 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1417) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000084 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2837) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000111 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3596_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/5002) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000087 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3598) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2660) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000167 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2665) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000028 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000176 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12212) see mds.a log and `damage ls` output for details 2024-01-11T17:58:42.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2679) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000132 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000174 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12209) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3379) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2683) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000016c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12196) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000033 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000016a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000014c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000158 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000017a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2619_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000071 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3456) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000157 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000120 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000036 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2616_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000009d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2656) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ea (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002eb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ed (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ee (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3767) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2669) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3503) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12201) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2615) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2658) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ec (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2699) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002f9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/828) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002fa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002fb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3605) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002fc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002fe (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2666) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000300 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000301 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002fd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3609) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000302 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000303 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000306 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2604) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000305 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2688) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000304 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3600) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000307 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000309 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ff (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000030a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000308 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000030d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000030e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3395) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000030f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2675) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000030c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000310 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12186) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000312 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000311 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/174) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000030b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2650) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16396) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000313 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.003 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000314 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2689) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.004 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:42 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub queued for path: / 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: active paths [/] 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: active paths [/] 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000003 (/volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000004 (/volumes/qa/sv_0/.meta) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: active paths [/] 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: active paths [/] 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000520 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/log/postgresql-Thu.log) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3766) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2662) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000fc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3257) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000180 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000037 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000128 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000013b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000104 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6113) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000004e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2607) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000015e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2609_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000009c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2757) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000171 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12204) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000082 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2835) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000114 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3118_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000012c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3381_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000068 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1417) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000084 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2837) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000111 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3596_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/5002) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000087 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3598) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2660) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000167 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2665) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000028 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000176 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12212) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2679) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000132 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000174 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12209) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3379) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2683) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000016c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12196) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000033 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000016a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000014c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000158 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000017a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2619_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000071 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3456) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000157 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000120 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000036 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2616_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000009d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2656) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ea (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002eb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ed (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ee (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3767) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2669) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3503) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12201) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2615) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2658) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ec (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2699) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002f9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/828) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002fa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002fb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3605) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002fc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002fe (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2666) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000300 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000301 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002fd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3609) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000302 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000303 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000306 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2604) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000305 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2688) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000304 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3600) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000307 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000309 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ff (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000030a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000308 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000030d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000030e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3395) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000030f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2675) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000030c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000310 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12186) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000312 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000311 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/174) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000030b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2650) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16396) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000313 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000314 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2689) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:42 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub queued for path: / 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: active paths [/] 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: active paths [/] 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000003 (/volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000004 (/volumes/qa/sv_0/.meta) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: active paths [/] 2024-01-11T17:58:43.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: active paths [/] 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000520 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/log/postgresql-Thu.log) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3766) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2662) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000fc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3257) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000180 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000037 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000128 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000013b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000104 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6113) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000004e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2607) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000015e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2609_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000009c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2757) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000171 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12204) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000082 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2835) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000114 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3118_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000012c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3381_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000068 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1417) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000084 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2837) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000111 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3596_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/5002) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000087 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3598) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2660) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000167 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2665) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000028 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000176 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12212) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2679) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000132 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000174 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12209) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3379) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2683) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000016c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12196) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000033 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000016a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000014c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000158 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000017a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2619_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000071 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3456) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000157 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000120 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000036 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2616_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000009d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2656) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ea (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002eb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ed (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ee (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3767) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2669) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3503) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12201) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607_fsm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2615) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2658) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ec (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2699) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002f9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/828) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002fa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002fb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3605) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002fc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002fe (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2666) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000300 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000301 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002fd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3609) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000302 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000303 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000306 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2604) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000305 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2688) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000304 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3600) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000307 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000309 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ff (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000030a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000308 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000030d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834_vm) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000030e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3395) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000030f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2675) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000030c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000310 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12186) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000312 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000311 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/174) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000030b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2650) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16396) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000313 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000314 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2689) see mds.a log and `damage ls` output for details 2024-01-11T17:58:43.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:42 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: scrub summary: active paths [/] 2024-01-11T17:58:44.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000270 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000272 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000273 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000026f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000276 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000274 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000275 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000278 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000277 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000271 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000027b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000027c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000027a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000027e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000027d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000027f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000280 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000281 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000282 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000284 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000283 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000286 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000028b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000028c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000287 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000289 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000028a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000028e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000290 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000291 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000292 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000028d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000293 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000028f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000294 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000295 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000296 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000297 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000299 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2615) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000298 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000029a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3767) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001bc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2995) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001bb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2669) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ba (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001be (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12201) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001bf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3503) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2699) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2658) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/828) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3605) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3609) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2666) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.038 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3541) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2688) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/826) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3439_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.039 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1255) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2650) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2602) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2834_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.040 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3600) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3395) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2675) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12186) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/174) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.041 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3602) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.042 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2224) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.042 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2689) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.042 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000018a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2662) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.042 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000324 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3766) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.042 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000018b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3257) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.042 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000018d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000018e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000018f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000018c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000190 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.043 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000191 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000193 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000194 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12204) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000192 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2757) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000195 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2835) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000196 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2837) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000199 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3118_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.044 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000019a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000019b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/5002) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000019d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3596_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000019f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2665) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000019e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3598) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.045 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000197 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3381_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000198 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1417) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2679) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000019c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2660) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12212) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.046 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12209) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3379) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2683) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12196) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ae (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ab (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.047 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2619_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001af (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3576_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2656) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2600_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_filenode.map) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000001ac (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.048 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3997) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2832_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2653) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2755) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3534) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3081) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2659) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/175) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2328) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6102_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6104_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2686) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2337) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2693) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3574) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.051 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2704) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000039c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000039d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000039e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16393) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000039f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.052 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003af (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.053 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000523 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_internal.init) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003be (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.054 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000378 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2701) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000376 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000037a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000037b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000037c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000037d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12184) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000377 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3542) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.055 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000037e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3350_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000380 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2611_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000381 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3466) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000382 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3380) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000379 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000386 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/549) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000383 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2692) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000388 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000389 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12216) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.056 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000038a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1249) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000038b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/PG_VERSION) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000038c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12206) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000387 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000038d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000038e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000037f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000384 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6112) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000390 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000391 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000392 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000394 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000393 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2652) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.057 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000395 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3502) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000396 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000397 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2841) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000398 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000039a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2651) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000385 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16390) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000039b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2673) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000359 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000035a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2681) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000035b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000035d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12191) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000035e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000035f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.058 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000360 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000361 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000399 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000362 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000363 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2690) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16384) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000364 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2684) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000366 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2620_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000035c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000365 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2680) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000036a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000367 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2696) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000036c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2654) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.059 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000368 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2678) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000036b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000036e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000369 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000370 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000036f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000371 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3085) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000372 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000373 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000375 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12189) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000374 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2703) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3766) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2662) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3257) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2757) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2835) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12204) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2837) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2660) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3596_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/5002) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3598) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2665) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2679) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12212) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.062 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12196) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2683) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12209) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000011c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3379) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.063 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000002e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2656) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000003c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000009b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000101 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000f7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000004f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000032 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000183 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.064 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000007d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000133 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000102 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000008c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000013c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000003b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000085 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000083 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.065 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000027 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000ee (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000130 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000f9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000064 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000179 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000016e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000002b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000173 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000030 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000040 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.066 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000038 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000fb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3542) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2701) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000005e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000119 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000011a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000003a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000089 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2840) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000010b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3350_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000131 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2611_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000047 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3466) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2692) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000012d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2995_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000165 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12184) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.067 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/549) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000069 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3380) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000103 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6112) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000178 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12216) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000029 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1249) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000004a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000153 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/PG_VERSION) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000172 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12206) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000115 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000076 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000010e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3541_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000081 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2834) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000109 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6106_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.068 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000006b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3256) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000097 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2652) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000142 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3502) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000008a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2841) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000006e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000002c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x1000000011d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x10000000096 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2651) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: Scrub error on inode 0x100000000b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2673) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.069 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:43 smithi038 ceph-mon[43582]: pgmap v327: 129 pgs: 129 active+clean; 1.1 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 43 MiB/s wr, 503 op/s 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: scrub summary: active paths [/] 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000270 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000272 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000273 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000026f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000276 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000274 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000275 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000278 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000277 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000271 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000027b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000027c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000027a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000027e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000027d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000027f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000280 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000281 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000282 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000284 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000283 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000286 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000028b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000028c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000287 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000289 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000028a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000028e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000290 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000291 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000292 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000028d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000293 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000028f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000294 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000295 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000296 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000297 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000299 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2615) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000298 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000029a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3767) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001bc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2995) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001bb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2669) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ba (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001be (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12201) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001bf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3503) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2699) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2658) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/828) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3605) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3609) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2666) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3541) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2688) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/826) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3439_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1255) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2650) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2602) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2834_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3600) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3395) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2675) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12186) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/174) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3602) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2224) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2689) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000018a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2662) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000324 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3766) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000018b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3257) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000018d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000018e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.080 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000018f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000018c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000190 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000191 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000193 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000194 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12204) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000192 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2757) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000195 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2835) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000196 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2837) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000199 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3118_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000019a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000019b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/5002) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000019d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3596_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000019f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2665) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000019e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3598) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000197 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3381_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000198 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1417) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2679) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000019c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2660) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12212) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12209) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3379) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2683) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12196) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ae (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ab (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2619_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001af (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3576_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2656) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2600_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_filenode.map) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000001ac (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3997) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2832_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2653) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2755) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3534) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3081) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2659) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/175) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2328) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6102_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6104_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2686) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2337) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2693) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3574) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2704) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000039c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000039d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000039e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16393) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000039f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.088 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003af (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.089 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000523 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_internal.init) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003be (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000378 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2701) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000376 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000037a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000037b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000037c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000037d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12184) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000377 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3542) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000037e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3350_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000380 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2611_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000381 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3466) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000382 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3380) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000379 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000386 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/549) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000383 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2692) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000388 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000389 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12216) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000038a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1249) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000038b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/PG_VERSION) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000038c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12206) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000387 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000038d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000038e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000037f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000384 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6112) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000390 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000391 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000392 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000394 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000393 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2652) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000395 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3502) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000396 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000397 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2841) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000398 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000039a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2651) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000385 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16390) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.093 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000039b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2673) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000359 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000035a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2681) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000035b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000035d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12191) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000035e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000035f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000360 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000361 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000399 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000362 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000363 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2690) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16384) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000364 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2684) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000366 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2620_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.094 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000035c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000365 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2680) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000036a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000367 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2696) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000036c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2654) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000368 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2678) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000036b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000036e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000369 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000370 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000036f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000371 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3085) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000372 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000373 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000375 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12189) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000374 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2703) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3766) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2662) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3257) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2757) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2835) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12204) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2837) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2660) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3596_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/5002) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3598) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2665) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2679) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12212) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12196) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2683) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12209) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000011c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3379) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.098 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000002e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2656) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000003c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000009b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000101 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000f7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000004f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.099 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000032 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000183 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000007d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000133 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000102 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000008c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000013c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000003b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000085 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000083 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000027 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000ee (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000130 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000f9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000064 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000179 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000016e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000002b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000173 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000030 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.101 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000040 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000038 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000fb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3542) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2701) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000005e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000119 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000011a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000003a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000089 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2840) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000010b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3350_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000131 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2611_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000047 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3466) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2692) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000012d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2995_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000165 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12184) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/549) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000069 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3380) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000103 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6112) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000178 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12216) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000029 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1249) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000004a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000153 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/PG_VERSION) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000172 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12206) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000115 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000076 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000010e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3541_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000081 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2834) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000109 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6106_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.103 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000006b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3256) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000097 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2652) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000142 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3502) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000008a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2841) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000006e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000002c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x1000000011d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x10000000096 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2651) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: Scrub error on inode 0x100000000b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2673) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.104 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:43 smithi007 ceph-mon[33361]: pgmap v327: 129 pgs: 129 active+clean; 1.1 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 43 MiB/s wr, 503 op/s 2024-01-11T17:58:44.127 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 15.0 s, 1220.4 tps, lat 76.869 ms stddev 111.674 2024-01-11T17:58:44.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: scrub summary: active paths [/] 2024-01-11T17:58:44.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000270 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000272 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000273 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000026f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000276 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000274 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000275 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000278 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000277 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000271 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000027b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000027c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000027a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000027e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000027d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000027f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000280 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000281 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000282 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000284 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000283 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000286 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000028b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000028c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000287 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000289 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000028a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000028e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000290 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000291 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000292 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000028d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000293 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000028f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000294 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000295 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000296 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000297 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000299 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2615) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000298 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000029a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3767) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001bc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2995) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001bb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2669) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ba (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001be (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12201) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001bf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3503) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2699) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2658) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/828) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3605) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3609) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2666) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.252 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3541) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2688) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/826) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3439_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1255) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2650) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2602) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2834_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3600) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3395) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2675) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12186) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/174) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3602) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2224) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2689) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000018a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2662) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000324 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3766) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000018b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3257) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000018d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000018e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000018f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.254 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000018c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000190 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000191 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000193 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000194 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12204) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000192 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2757) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000195 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2835) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000196 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2837) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000199 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3118_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000019a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000019b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/5002) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000019d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3596_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000019f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2665) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000019e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3598) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000197 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3381_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000198 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1417) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2679) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.256 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000019c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2660) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12212) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12209) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3379) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2683) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12196) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.258 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ae (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ab (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2619_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001af (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3576_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2656) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2600_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_filenode.map) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000001ac (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3997) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.259 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2832_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2653) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2755) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3534) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3081) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2659) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.260 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/175) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2328) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6102_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6104_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2686) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2337) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2693) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3574) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.261 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2704) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000039c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000039d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000039e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16393) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000039f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.262 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003af (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.263 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000523 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_internal.init) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003be (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.264 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000378 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2701) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000376 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000037a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000037b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000037c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000037d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12184) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000377 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3542) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000037e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3350_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000380 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2611_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000381 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3466) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.265 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000382 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3380) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000379 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000386 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/549) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000383 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2692) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000388 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000389 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12216) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000038a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1249) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000038b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/PG_VERSION) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000038c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12206) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000387 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000038d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000038e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.266 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000037f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000384 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6112) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000390 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000391 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000392 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000394 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000393 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2652) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000395 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3502) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000396 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000397 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2841) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000398 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000039a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2651) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000385 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16390) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000039b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2673) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.267 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000359 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000035a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2681) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000035b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000035d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12191) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000035e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000035f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000360 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000361 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000399 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000362 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000363 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2690) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16384) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000364 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2684) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000366 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2620_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.268 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000035c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000365 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2680) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000036a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000367 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2696) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000036c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2654) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000368 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2678) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000036b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000036e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000369 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000370 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000036f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000371 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3085) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.269 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000372 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000373 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000375 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12189) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000374 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2703) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3766) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2662) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3257) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2757) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.270 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002cb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2835) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.271 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.271 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002c3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.271 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ca (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12204) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.271 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.271 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002cf (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.271 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002cc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2837) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002cd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2660) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3596_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/5002) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3598) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.272 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2665) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2679) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12212) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002d9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12196) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e1 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002dd (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2683) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002da (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12209) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e4 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.273 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000011c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002db (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3379) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000e0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3467) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000dc (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3575) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000f3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3119) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000002e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2656) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000b6 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2674) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000b0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2668) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000003c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000009b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2655) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000101 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6110) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000df (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2702) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000f7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3597) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.274 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000e3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3712) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000004f (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000032 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2617_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000183 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000aa (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3164) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000007d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2831) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000133 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3501_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000102 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6111) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000c2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2682) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000ef (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/113) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000e5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3604) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000d2 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2691) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000008c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3440) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000013c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000a8 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2663) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.275 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000c9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2685) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000003b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000085 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000083 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2836) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000027 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000ee (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/548) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000130 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000f9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3080) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000064 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000179 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12182_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000ce (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2754) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000a9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3455) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000016e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12199) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000002b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000173 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12207) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000030 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000e7 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3606) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000040 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2613) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000038 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000fb (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3542) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000de (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2701) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000005e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000119 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000011a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3601_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000003a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000089 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2840) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000010b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3350_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000131 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2611_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000047 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3466) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000d3 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2692) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000012d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2995_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.277 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000165 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12184) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000f0 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/549) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000069 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000c5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3380) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000103 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6112) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000178 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12216) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000029 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1249) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000004a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000153 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/PG_VERSION) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000172 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12206) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000115 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000076 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000010e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3541_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000081 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2834) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000109 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6106_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.278 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000006b (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3256) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000097 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2652) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000142 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000b9 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3502) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000008a (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2841) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000006e (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000002c (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_fsm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.279 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x1000000011d (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3602_vm) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.280 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x10000000096 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2651) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.280 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: Scrub error on inode 0x100000000b5 (/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2673) see mds.i log and `damage ls` output for details 2024-01-11T17:58:44.280 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: scrub summary: idle+waiting paths [/] 2024-01-11T17:58:44.280 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:43 smithi016 ceph-mon[43195]: pgmap v327: 129 pgs: 129 active+clean; 1.1 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 43 MiB/s wr, 503 op/s 2024-01-11T17:58:45.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:45 smithi016 ceph-mon[43195]: pgmap v328: 129 pgs: 129 active+clean; 1.1 GiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 48 MiB/s wr, 734 op/s 2024-01-11T17:58:45.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:45 smithi038 ceph-mon[43582]: pgmap v328: 129 pgs: 129 active+clean; 1.1 GiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 48 MiB/s wr, 734 op/s 2024-01-11T17:58:45.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:45 smithi007 ceph-mon[33361]: pgmap v328: 129 pgs: 129 active+clean; 1.1 GiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 48 MiB/s wr, 734 op/s 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 -- 172.21.15.7:0/3098280341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe94105150 msgr2=0x7fbe941055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 --2- 172.21.15.7:0/3098280341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe94105150 0x7fbe941055d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00b0a0 tx=0x7fbe8c00b3b0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 -- 172.21.15.7:0/3098280341 shutdown_connections 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 --2- 172.21.15.7:0/3098280341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe940ffe10 0x7fbe94102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 --2- 172.21.15.7:0/3098280341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe94105150 0x7fbe941055d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 --2- 172.21.15.7:0/3098280341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94102780 0x7fbe94104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.524+0000 7fbe99fff700 1 -- 172.21.15.7:0/3098280341 >> 172.21.15.7:0/3098280341 conn(0x7fbe940fba30 msgr2=0x7fbe940fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.525+0000 7fbe99fff700 1 -- 172.21.15.7:0/3098280341 shutdown_connections 2024-01-11T17:58:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.525+0000 7fbe99fff700 1 -- 172.21.15.7:0/3098280341 wait complete. 2024-01-11T17:58:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.527+0000 7fbe99fff700 1 Processor -- start 2024-01-11T17:58:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.529+0000 7fbe99fff700 1 -- start start 2024-01-11T17:58:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.531+0000 7fbe99fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe940ffe10 0x7fbe9419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.531+0000 7fbe99fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe94102780 0x7fbe9419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.531+0000 7fbe99fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 0x7fbe941a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe99fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a5710 con 0x7fbe940ffe10 2024-01-11T17:58:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe99fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a5850 con 0x7fbe94105150 2024-01-11T17:58:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe99fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a59e0 con 0x7fbe94102780 2024-01-11T17:58:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 0x7fbe941a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 0x7fbe941a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51830/0 (socket says 172.21.15.7:51830) 2024-01-11T17:58:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe93fff700 1 -- 172.21.15.7:0/48544221 learned_addr learned my addr 172.21.15.7:0/48544221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe94102780 0x7fbe9419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.532+0000 7fbe937fe700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe940ffe10 0x7fbe9419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.533+0000 7fbe93fff700 1 -- 172.21.15.7:0/48544221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe94102780 msgr2=0x7fbe9419e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.533+0000 7fbe93fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe94102780 0x7fbe9419e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.533+0000 7fbe93fff700 1 -- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe940ffe10 msgr2=0x7fbe9419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.533+0000 7fbe93fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe940ffe10 0x7fbe9419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.533+0000 7fbe93fff700 1 -- 172.21.15.7:0/48544221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009d00 con 0x7fbe94105150 2024-01-11T17:58:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.534+0000 7fbe93fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 0x7fbe941a5140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00b830 tx=0x7fbe8c004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.535+0000 7fbe90ff9700 1 -- 172.21.15.7:0/48544221 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00e040 con 0x7fbe94105150 2024-01-11T17:58:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.535+0000 7fbe90ff9700 1 -- 172.21.15.7:0/48544221 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8c01db40 con 0x7fbe94105150 2024-01-11T17:58:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.535+0000 7fbe90ff9700 1 -- 172.21.15.7:0/48544221 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c01bc80 con 0x7fbe94105150 2024-01-11T17:58:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.535+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe941a5c60 con 0x7fbe94105150 2024-01-11T17:58:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.537+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941a6090 con 0x7fbe94105150 2024-01-11T17:58:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.551+0000 7fbe90ff9700 1 -- 172.21.15.7:0/48544221 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fbe8c01dcb0 con 0x7fbe94105150 2024-01-11T17:58:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.554+0000 7fbe90ff9700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe7c0a1f50 0x7fbe7c0a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.555+0000 7fbe90ff9700 1 -- 172.21.15.7:0/48544221 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe8c0cf1a0 con 0x7fbe94105150 2024-01-11T17:58:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 Processor -- start 2024-01-11T17:58:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 -- start start 2024-01-11T17:58:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbe937fe700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe7c0a1f50 0x7fbe7c0a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5eef20 0x557d7c5ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5f05d0 0x557d7c5f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c4e6030 con 0x557d7c5f05d0 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.557+0000 7fbe92ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.557+0000 7fbe92ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54800/0 (socket says 172.21.15.7:54800) 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.557+0000 7fbe92ffd700 1 -- 172.21.15.7:0/4159100040 learned_addr learned my addr 172.21.15.7:0/4159100040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.557+0000 7fbe937fe700 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5eef20 0x557d7c5ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.558+0000 7fbe93fff700 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5f05d0 0x557d7c5f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.556+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c549ac0 con 0x557d7c5eef20 2024-01-11T17:58:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.558+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c5473e0 con 0x557d7c5efc10 2024-01-11T17:58:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe92ffd700 1 -- 172.21.15.7:0/4159100040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5eef20 msgr2=0x557d7c5ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5eef20 0x557d7c5ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe92ffd700 1 -- 172.21.15.7:0/4159100040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5f05d0 msgr2=0x557d7c5f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5f05d0 0x557d7c5f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe92ffd700 1 -- 172.21.15.7:0/4159100040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d7c3b7350 con 0x557d7c5efc10 2024-01-11T17:58:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe937fe700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe7c0a1f50 0x7fbe7c0a4410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005ea0 tx=0x7fbe84005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.559+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5f0090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009af0 tx=0x7fbe88009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935aa45c7a513aee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.560+0000 7fbe79ffb700 1 -- 172.21.15.7:0/4159100040 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe880095f0 con 0x557d7c5efc10 2024-01-11T17:58:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.560+0000 7fbe79ffb700 1 -- 172.21.15.7:0/4159100040 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8800ed90 con 0x557d7c5efc10 2024-01-11T17:58:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.560+0000 7fbe79ffb700 1 -- 172.21.15.7:0/4159100040 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe880123d0 con 0x557d7c5efc10 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 msgr2=0x557d7c5f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5f0090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009af0 tx=0x7fbe88009e00 comp rx=0 tx=0).stop 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 shutdown_connections 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5f05d0 0x557d7c5f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5eef20 0x557d7c5ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/4159100040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5f0090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 >> 172.21.15.7:0/4159100040 conn(0x557d7c564740 msgr2=0x557d7c5e9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.561+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 shutdown_connections 2024-01-11T17:58:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.562+0000 7fbeadc44b80 1 -- 172.21.15.7:0/4159100040 wait complete. 2024-01-11T17:58:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.562+0000 7fbeadc44b80 1 -- start start 2024-01-11T17:58:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.563+0000 7fbeadc44b80 1 Processor -- start 2024-01-11T17:58:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c5ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c542cd0 con 0x557d7c5eef20 2024-01-11T17:58:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c4b1340 con 0x557d7c5efc10 2024-01-11T17:58:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c4b0fe0 con 0x557d7c5f05d0 2024-01-11T17:58:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.564+0000 7fbeadc44b80 10 client.0 _ll_register_callbacks cb 0x557d7c4e36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c5ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c5ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54810/0 (socket says 172.21.15.7:54810) 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe93fff700 1 -- 172.21.15.7:0/926630078 learned_addr learned my addr 172.21.15.7:0/926630078 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe92ffd700 1 -- 172.21.15.7:0/926630078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 msgr2=0x557d7c5ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c5ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe92ffd700 1 -- 172.21.15.7:0/926630078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 msgr2=0x557d7c5f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe92ffd700 1 -- 172.21.15.7:0/926630078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe88009750 con 0x557d7c5efc10 2024-01-11T17:58:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.565+0000 7fbe93fff700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c5ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:58:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.566+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5f8ea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe88000c00 tx=0x7fbe88013920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=519f095979c2dab1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.567+0000 7fbe72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.567+0000 7fbe72ffd700 1 -- 172.21.15.7:0/926630078 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88006e70 con 0x557d7c5efc10 2024-01-11T17:58:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.567+0000 7fbe72ffd700 1 -- 172.21.15.7:0/926630078 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe8800eae0 con 0x557d7c5efc10 2024-01-11T17:58:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.567+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d7c5e1d60 con 0x557d7c5efc10 2024-01-11T17:58:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.567+0000 7fbe72ffd700 1 -- 172.21.15.7:0/926630078 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8801c450 con 0x557d7c5efc10 2024-01-11T17:58:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.568+0000 7fbeadc44b80 10 client.25676 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.568+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557d7c363b00 con 0x557d7c5efc10 2024-01-11T17:58:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.568+0000 7fbe72ffd700 1 -- 172.21.15.7:0/926630078 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d7c5e1d60 con 0x557d7c5efc10 2024-01-11T17:58:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.568+0000 7fbe72ffd700 1 -- 172.21.15.7:0/926630078 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbe88011070 con 0x557d7c5efc10 2024-01-11T17:58:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbeadc44b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbeadc44b80 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbeadc44b80 20 client.25676 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c6046c0 0x557d7c606ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbeadc44b80 4 client.25676 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d7c53bb60 con 0x557d7c6046c0 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.569+0000 7fbe937fe700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c6046c0 0x557d7c606ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.570+0000 7fbe72ffd700 10 client.25676 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.570+0000 7fbe937fe700 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c6046c0 0x557d7c606ae0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f3024014661e0f server_cookie=18433a7fea5ec38a in_seq=0 out_seq=0 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.570+0000 7fbe72ffd700 1 -- 172.21.15.7:0/926630078 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d7c53bb60 con 0x557d7c6046c0 2024-01-11T17:58:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.570+0000 7fbe72ffd700 10 client.25676 handle_command_reply: tid=0 2024-01-11T17:58:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.571+0000 7fbeadc44b80 1 client.25676 shutdown 2024-01-11T17:58:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.571+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 msgr2=0x557d7c5f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.571+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5f8ea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe88000c00 tx=0x7fbe88013920 comp rx=0 tx=0).stop 2024-01-11T17:58:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 shutdown_connections 2024-01-11T17:58:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c6046c0 0x557d7c606ae0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c5ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbe72ffd700 0 client.25676 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/926630078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5f8ea0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 >> 172.21.15.7:0/926630078 conn(0x557d7c564740 msgr2=0x557d7c5e8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 shutdown_connections 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.572+0000 7fbeadc44b80 1 -- 172.21.15.7:0/926630078 wait complete. 2024-01-11T17:58:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.573+0000 7fbeadc44b80 20 client.25676 trim_cache size 0 max 16384 2024-01-11T17:58:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.599+0000 7fbeadc44b80 1 Processor -- start 2024-01-11T17:58:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.600+0000 7fbeadc44b80 1 -- start start 2024-01-11T17:58:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.602+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.602+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.602+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c600590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.602+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c3b9640 con 0x557d7c5eef20 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.602+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c606ea0 con 0x557d7c5efc10 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.602+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c5ea7c0 con 0x557d7c5f05d0 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c600590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c600590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54826/0 (socket says 172.21.15.7:54826) 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 -- 172.21.15.7:0/2858601876 learned_addr learned my addr 172.21.15.7:0/2858601876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42760/0 (socket says 172.21.15.7:42760) 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 -- 172.21.15.7:0/2858601876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 msgr2=0x557d7c5ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 -- 172.21.15.7:0/2858601876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 msgr2=0x557d7c5f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.603+0000 7fbe93fff700 1 -- 172.21.15.7:0/2858601876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d7c5041d0 con 0x557d7c5f05d0 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.604+0000 7fbe93fff700 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c600590 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c098160 tx=0x7fbe8c009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f021131059b053 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.604+0000 7fbe797fa700 1 -- 172.21.15.7:0/2858601876 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c01cea0 con 0x557d7c5f05d0 2024-01-11T17:58:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.604+0000 7fbe797fa700 1 -- 172.21.15.7:0/2858601876 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8c09c360 con 0x557d7c5f05d0 2024-01-11T17:58:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.604+0000 7fbe797fa700 1 -- 172.21.15.7:0/2858601876 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c099e30 con 0x557d7c5f05d0 2024-01-11T17:58:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 -- 172.21.15.7:0/2858601876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 msgr2=0x557d7c600590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c600590 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c098160 tx=0x7fbe8c009040 comp rx=0 tx=0).stop 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 -- 172.21.15.7:0/2858601876 shutdown_connections 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5f05d0 0x557d7c600590 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/2858601876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5efc10 0x557d7c5ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.605+0000 7fbeadc44b80 1 -- 172.21.15.7:0/2858601876 >> 172.21.15.7:0/2858601876 conn(0x557d7c564740 msgr2=0x557d7c5f7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.606+0000 7fbeadc44b80 1 -- 172.21.15.7:0/2858601876 shutdown_connections 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.606+0000 7fbeadc44b80 1 -- 172.21.15.7:0/2858601876 wait complete. 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.607+0000 7fbeadc44b80 1 -- start start 2024-01-11T17:58:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.607+0000 7fbeadc44b80 1 Processor -- start 2024-01-11T17:58:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.608+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.608+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.608+0000 7fbeadc44b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 0x557d7c5fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c5ff010 con 0x557d7c5eef20 2024-01-11T17:58:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c5ff180 con 0x557d7c5f05d0 2024-01-11T17:58:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbeadc44b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d7c5ff340 con 0x557d7c5efc10 2024-01-11T17:58:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbeadc44b80 10 client.0 _ll_register_callbacks cb 0x557d7c4e36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:58:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 0x557d7c5fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 0x557d7c5fead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51864/0 (socket says 172.21.15.7:51864) 2024-01-11T17:58:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbe93fff700 1 -- 172.21.15.7:0/3215779130 learned_addr learned my addr 172.21.15.7:0/3215779130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:58:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbe93fff700 1 -- 172.21.15.7:0/3215779130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 msgr2=0x557d7c5fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.609+0000 7fbe937fe700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe93fff700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe93fff700 1 -- 172.21.15.7:0/3215779130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 msgr2=0x557d7c5fd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:58:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe93fff700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5fd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe93fff700 1 -- 172.21.15.7:0/3215779130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941a6090 con 0x557d7c5f05d0 2024-01-11T17:58:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe937fe700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe92ffd700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:58:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe93fff700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 0x557d7c5fead0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0cc9c0 tx=0x7fbe8c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d450100d575ef6ac server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe72ffd700 1 -- 172.21.15.7:0/3215779130 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c09fdb0 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe8800ed90 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.610+0000 7fbe72ffd700 1 -- 172.21.15.7:0/3215779130 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe8c099e10 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbe72ffd700 1 -- 172.21.15.7:0/3215779130 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8c01cea0 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 10 client.25682 fetch_fsmap learned FSMap version 26 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557d7c3bbc70 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbe72ffd700 1 -- 172.21.15.7:0/3215779130 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8c082080 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbe72ffd700 1 -- 172.21.15.7:0/3215779130 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbe8c0a0af0 con 0x557d7c5f05d0 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 26 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 20 client.25682 populate_metadata read hostname 'smithi007' 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c65b220 0x557d7c65d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 4 client.25682 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:58:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.611+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557d7c53b880 con 0x557d7c65b220 2024-01-11T17:58:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.612+0000 7fbe937fe700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c65b220 0x557d7c65d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:58:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.612+0000 7fbe937fe700 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c65b220 0x557d7c65d640 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3997f3fd6800273 server_cookie=107ba9ad8b932736 in_seq=0 out_seq=0 2024-01-11T17:58:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.612+0000 7fbe72ffd700 10 client.25682 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbe72ffd700 1 -- 172.21.15.7:0/3215779130 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557d7c53b880 con 0x557d7c65b220 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbe72ffd700 10 client.25682 handle_command_reply: tid=0 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 client.25682 shutdown 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 msgr2=0x557d7c5fead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 0x557d7c5fead0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0cc9c0 tx=0x7fbe8c009b80 comp rx=0 tx=0).stop 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 shutdown_connections 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x557d7c65b220 0x557d7c65d640 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d7c5f05d0 0x557d7c5fead0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d7c5eef20 0x557d7c5fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 --2- 172.21.15.7:0/3215779130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557d7c5efc10 0x557d7c5fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 >> 172.21.15.7:0/3215779130 conn(0x557d7c564740 msgr2=0x557d7c5f6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.634+0000 7fbe72ffd700 0 client.25682 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:58:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.635+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 shutdown_connections 2024-01-11T17:58:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.635+0000 7fbeadc44b80 1 -- 172.21.15.7:0/3215779130 wait complete. 2024-01-11T17:58:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.635+0000 7fbeadc44b80 20 client.25682 trim_cache size 0 max 16384 2024-01-11T17:58:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:58:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:58:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:58:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe7c0a1f50 msgr2=0x7fbe7c0a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe7c0a1f50 0x7fbe7c0a4410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005ea0 tx=0x7fbe84005e30 comp rx=0 tx=0).stop 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 msgr2=0x7fbe941a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 0x7fbe941a5140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00b830 tx=0x7fbe8c004a50 comp rx=0 tx=0).stop 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 shutdown_connections 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fbe7c0a1f50 0x7fbe7c0a4410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe94105150 0x7fbe941a5140 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe940ffe10 0x7fbe9419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 --2- 172.21.15.7:0/48544221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe94102780 0x7fbe9419e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 >> 172.21.15.7:0/48544221 conn(0x7fbe940fba30 msgr2=0x7fbe94100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 shutdown_connections 2024-01-11T17:58:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:58:47.636+0000 7fbe99fff700 1 -- 172.21.15.7:0/48544221 wait complete. 2024-01-11T17:58:47.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:47 smithi016 ceph-mon[43195]: pgmap v329: 129 pgs: 129 active+clean; 1.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 63 MiB/s wr, 1.06k op/s 2024-01-11T17:58:48.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:47 smithi038 ceph-mon[43582]: pgmap v329: 129 pgs: 129 active+clean; 1.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 63 MiB/s wr, 1.06k op/s 2024-01-11T17:58:48.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:47 smithi007 ceph-mon[33361]: pgmap v329: 129 pgs: 129 active+clean; 1.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 63 MiB/s wr, 1.06k op/s 2024-01-11T17:58:48.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:48 smithi016 ceph-mon[43195]: scrub summary: idle 2024-01-11T17:58:48.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:48 smithi016 ceph-mon[43195]: scrub summary: idle 2024-01-11T17:58:49.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:48 smithi038 ceph-mon[43582]: scrub summary: idle 2024-01-11T17:58:49.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:48 smithi038 ceph-mon[43582]: scrub summary: idle 2024-01-11T17:58:49.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:48 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T17:58:48.688+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:49.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:48 smithi007 ceph-mon[33361]: scrub summary: idle 2024-01-11T17:58:49.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:48 smithi007 ceph-mon[33361]: scrub summary: idle 2024-01-11T17:58:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 20.0 s, 1964.0 tps, lat 54.127 ms stddev 85.760 2024-01-11T17:58:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:49 smithi016 ceph-mon[43195]: scrub summary: idle 2024-01-11T17:58:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:49 smithi016 ceph-mon[43195]: pgmap v330: 129 pgs: 129 active+clean; 1.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 54 MiB/s wr, 989 op/s 2024-01-11T17:58:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:49 smithi016 ceph-mon[43195]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:49.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:49 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:active 2024-01-11T17:58:49.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:49 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:50.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:49 smithi038 ceph-mon[43582]: scrub summary: idle 2024-01-11T17:58:50.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:49 smithi038 ceph-mon[43582]: pgmap v330: 129 pgs: 129 active+clean; 1.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 54 MiB/s wr, 989 op/s 2024-01-11T17:58:50.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:49 smithi038 ceph-mon[43582]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:50.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:49 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:active 2024-01-11T17:58:50.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:49 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:50.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:49 smithi007 ceph-mon[33361]: scrub summary: idle 2024-01-11T17:58:50.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:49 smithi007 ceph-mon[33361]: pgmap v330: 129 pgs: 129 active+clean; 1.1 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 54 MiB/s wr, 989 op/s 2024-01-11T17:58:50.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:49 smithi007 ceph-mon[33361]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:50.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:49 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] up:active 2024-01-11T17:58:50.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:49 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:51 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:active 2024-01-11T17:58:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:51 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:51 smithi016 ceph-mon[43195]: pgmap v331: 129 pgs: 129 active+clean; 1.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 59 MiB/s wr, 1.17k op/s 2024-01-11T17:58:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:51 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:active 2024-01-11T17:58:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:51 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:51.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:51 smithi038 ceph-mon[43582]: pgmap v331: 129 pgs: 129 active+clean; 1.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 59 MiB/s wr, 1.17k op/s 2024-01-11T17:58:51.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:51 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] up:active 2024-01-11T17:58:51.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:51 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:51.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:51 smithi007 ceph-mon[33361]: pgmap v331: 129 pgs: 129 active+clean; 1.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 59 MiB/s wr, 1.17k op/s 2024-01-11T17:58:52.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:52 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:52.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:52 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:52.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:52 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T17:58:53.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:53 smithi016 ceph-mon[43195]: pgmap v332: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 58 MiB/s wr, 1.27k op/s 2024-01-11T17:58:53.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:53 smithi038 ceph-mon[43582]: pgmap v332: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 58 MiB/s wr, 1.27k op/s 2024-01-11T17:58:53.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:53 smithi007 ceph-mon[33361]: pgmap v332: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 58 MiB/s wr, 1.27k op/s 2024-01-11T17:58:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 25.0 s, 1685.4 tps, lat 59.753 ms stddev 90.706 2024-01-11T17:58:54.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:54 smithi016 ceph-mon[43195]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:54.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:54 smithi038 ceph-mon[43582]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:54.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:54 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T17:58:54.367+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:54.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:54 smithi007 ceph-mon[33361]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-01-11T17:58:55.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:55 smithi016 ceph-mon[43195]: pgmap v333: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 55 MiB/s wr, 1.30k op/s 2024-01-11T17:58:55.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:55 smithi038 ceph-mon[43582]: pgmap v333: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 55 MiB/s wr, 1.30k op/s 2024-01-11T17:58:55.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:55 smithi007 ceph-mon[33361]: pgmap v333: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 55 MiB/s wr, 1.30k op/s 2024-01-11T17:58:57.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:57 smithi016 ceph-mon[43195]: pgmap v334: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 46 MiB/s wr, 1.20k op/s 2024-01-11T17:58:58.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:57 smithi038 ceph-mon[43582]: pgmap v334: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 46 MiB/s wr, 1.20k op/s 2024-01-11T17:58:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:57 smithi007 ceph-mon[33361]: pgmap v334: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 46 MiB/s wr, 1.20k op/s 2024-01-11T17:58:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 30.0 s, 2033.2 tps, lat 49.447 ms stddev 63.453 2024-01-11T17:58:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:58:59 smithi016 ceph-mon[43195]: pgmap v335: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 MiB/s wr, 826 op/s 2024-01-11T17:59:00.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:58:59 smithi038 ceph-mon[43582]: pgmap v335: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 MiB/s wr, 826 op/s 2024-01-11T17:59:00.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:58:59 smithi007 ceph-mon[33361]: pgmap v335: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 MiB/s wr, 826 op/s 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.892+0000 7f45be9ff700 1 -- 172.21.15.7:0/1250792232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8109770 msgr2=0x7f45b8110020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.892+0000 7f45be9ff700 1 --2- 172.21.15.7:0/1250792232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8109770 0x7f45b8110020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009fc0 tx=0x7f45b000b330 comp rx=0 tx=0).stop 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 -- 172.21.15.7:0/1250792232 shutdown_connections 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 --2- 172.21.15.7:0/1250792232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45b8107be0 0x7f45b8108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 --2- 172.21.15.7:0/1250792232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8109770 0x7f45b8110020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 --2- 172.21.15.7:0/1250792232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8108dd0 0x7f45b8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 -- 172.21.15.7:0/1250792232 >> 172.21.15.7:0/1250792232 conn(0x7f45b8076b60 msgr2=0x7f45b8078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 -- 172.21.15.7:0/1250792232 shutdown_connections 2024-01-11T17:59:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.893+0000 7f45be9ff700 1 -- 172.21.15.7:0/1250792232 wait complete. 2024-01-11T17:59:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.894+0000 7f45be9ff700 1 Processor -- start 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 -- start start 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45b8107be0 0x7f45b819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 0x7f45b819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8109770 0x7f45b81a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b81a55c0 con 0x7f45b8109770 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b806b030 con 0x7f45b8108dd0 2024-01-11T17:59:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.895+0000 7f45be9ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b806b1c0 con 0x7f45b8107be0 2024-01-11T17:59:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 0x7f45b819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 0x7f45b819df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53226/0 (socket says 172.21.15.7:53226) 2024-01-11T17:59:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 -- 172.21.15.7:0/3562582722 learned_addr learned my addr 172.21.15.7:0/3562582722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8109770 0x7f45b81a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 -- 172.21.15.7:0/3562582722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45b8107be0 msgr2=0x7f45b819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45b8107be0 0x7f45b819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 -- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8109770 msgr2=0x7f45b81a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8109770 0x7f45b81a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45b77fe700 1 -- 172.21.15.7:0/3562582722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0009c40 con 0x7f45b8108dd0 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.896+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8109770 0x7f45b81a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.897+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 0x7f45b819df80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c920 tx=0x7f45a400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.897+0000 7f45b57fa700 1 -- 172.21.15.7:0/3562582722 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4007b00 con 0x7f45b8108dd0 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.897+0000 7f45b57fa700 1 -- 172.21.15.7:0/3562582722 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a4004cf0 con 0x7f45b8108dd0 2024-01-11T17:59:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.897+0000 7f45b57fa700 1 -- 172.21.15.7:0/3562582722 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4005630 con 0x7f45b8108dd0 2024-01-11T17:59:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.898+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b806b4a0 con 0x7f45b8108dd0 2024-01-11T17:59:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.899+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b806b960 con 0x7f45b8108dd0 2024-01-11T17:59:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.913+0000 7f45b57fa700 1 -- 172.21.15.7:0/3562582722 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f45a4004750 con 0x7f45b8108dd0 2024-01-11T17:59:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.915+0000 7f45d2779b80 1 Processor -- start 2024-01-11T17:59:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 -- start start 2024-01-11T17:59:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9790f20 0x5588e9791340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 0x5588e9792090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e9798e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e9688030 con 0x5588e9791c10 2024-01-11T17:59:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e96ebac0 con 0x5588e9790f20 2024-01-11T17:59:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.916+0000 7f45d2779b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e96e93e0 con 0x5588e97925d0 2024-01-11T17:59:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.917+0000 7f45b57fa700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45a80a2020 0x7f45a80a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.917+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 0x5588e9792090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.917+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 0x5588e9792090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37396/0 (socket says 172.21.15.7:37396) 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.917+0000 7f45b7fff700 1 -- 172.21.15.7:0/3929454375 learned_addr learned my addr 172.21.15.7:0/3929454375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.917+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e9798e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.918+0000 7f45b57fa700 1 -- 172.21.15.7:0/3562582722 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f45a40cd6d0 con 0x7f45b8108dd0 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.918+0000 7f45b7fff700 1 -- 172.21.15.7:0/3929454375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 msgr2=0x5588e9798e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.918+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e9798e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.918+0000 7f45b7fff700 1 -- 172.21.15.7:0/3929454375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9790f20 msgr2=0x5588e9791340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.918+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9790f20 0x5588e9791340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.918+0000 7f45b7fff700 1 -- 172.21.15.7:0/3929454375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e9559350 con 0x5588e9791c10 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.919+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45a80a2020 0x7f45a80a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.919+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 0x5588e9792090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009bb0 tx=0x7f45ac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bd1ae559704e76 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.920+0000 7f45a27fc700 1 -- 172.21.15.7:0/3929454375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00eb60 con 0x5588e9791c10 2024-01-11T17:59:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.920+0000 7f45a27fc700 1 -- 172.21.15.7:0/3929454375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45ac004cf0 con 0x5588e9791c10 2024-01-11T17:59:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.920+0000 7f45a27fc700 1 -- 172.21.15.7:0/3929454375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0056e0 con 0x5588e9791c10 2024-01-11T17:59:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.921+0000 7f45d2779b80 1 -- 172.21.15.7:0/3929454375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 msgr2=0x5588e9792090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.921+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 0x5588e9792090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009bb0 tx=0x7f45ac009ec0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.923+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45a80a2020 0x7f45a80a44e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009f90 tx=0x7f45b000b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.923+0000 7f45d2779b80 1 -- 172.21.15.7:0/3929454375 shutdown_connections 2024-01-11T17:59:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.923+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e9798e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.923+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9790f20 0x5588e9791340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.923+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3929454375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9791c10 0x5588e9792090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.923+0000 7f45d2779b80 1 -- 172.21.15.7:0/3929454375 >> 172.21.15.7:0/3929454375 conn(0x5588e9706740 msgr2=0x5588e978b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.925+0000 7f45d2779b80 1 -- 172.21.15.7:0/3929454375 shutdown_connections 2024-01-11T17:59:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.926+0000 7f45d2779b80 1 -- 172.21.15.7:0/3929454375 wait complete. 2024-01-11T17:59:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.926+0000 7f45d2779b80 1 -- start start 2024-01-11T17:59:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.927+0000 7f45d2779b80 1 Processor -- start 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.930+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.930+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e979af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.930+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.930+0000 7f45d2779b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e96e4cd0 con 0x5588e9790f20 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.930+0000 7f45d2779b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e9653340 con 0x5588e9791c10 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.930+0000 7f45d2779b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e9652fe0 con 0x5588e97925d0 2024-01-11T17:59:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.931+0000 7f45d2779b80 10 client.0 _ll_register_callbacks cb 0x5588e96856c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.931+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.931+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37406/0 (socket says 172.21.15.7:37406) 2024-01-11T17:59:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.931+0000 7f45b7fff700 1 -- 172.21.15.7:0/3829651975 learned_addr learned my addr 172.21.15.7:0/3829651975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.931+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e979af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.932+0000 7f45b77fe700 1 -- 172.21.15.7:0/3829651975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 msgr2=0x5588e97a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.932+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.932+0000 7f45b77fe700 1 -- 172.21.15.7:0/3829651975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 msgr2=0x5588e979a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.932+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.932+0000 7f45b77fe700 1 -- 172.21.15.7:0/3829651975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac009860 con 0x5588e9791c10 2024-01-11T17:59:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.932+0000 7f45977fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:59:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.933+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e979af20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45a4096bf0 tx=0x7f45a401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cfedf424decf6b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.933+0000 7f45977fe700 1 -- 172.21.15.7:0/3829651975 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a409a880 con 0x5588e9791c10 2024-01-11T17:59:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.933+0000 7f45977fe700 1 -- 172.21.15.7:0/3829651975 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f45a409a180 con 0x5588e9791c10 2024-01-11T17:59:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.933+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588e9783d60 con 0x5588e9791c10 2024-01-11T17:59:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.933+0000 7f45977fe700 1 -- 172.21.15.7:0/3829651975 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45a409a880 con 0x5588e9791c10 2024-01-11T17:59:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.934+0000 7f45d2779b80 10 client.25691 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.934+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5588e9505b00 con 0x5588e9791c10 2024-01-11T17:59:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.934+0000 7f45977fe700 1 -- 172.21.15.7:0/3829651975 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a409e2d0 con 0x5588e9791c10 2024-01-11T17:59:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45977fe700 1 -- 172.21.15.7:0/3829651975 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f45a409a3e0 con 0x5588e9791c10 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45d2779b80 10 client.25691 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45d2779b80 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45d2779b80 20 client.25691 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97a6900 0x5588e97a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45d2779b80 4 client.25691 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.936+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588e9783d60 con 0x5588e97a6900 2024-01-11T17:59:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.937+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97a6900 0x5588e97a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.939+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97a6900 0x5588e97a8d40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1900241fcc872c8c server_cookie=26aa22ce3cfe2481 in_seq=0 out_seq=0 2024-01-11T17:59:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.939+0000 7f45977fe700 10 client.25691 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.940+0000 7f45977fe700 1 -- 172.21.15.7:0/3829651975 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588e9783d60 con 0x5588e97a6900 2024-01-11T17:59:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.940+0000 7f45977fe700 10 client.25691 handle_command_reply: tid=0 2024-01-11T17:59:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.940+0000 7f45d2779b80 1 client.25691 shutdown 2024-01-11T17:59:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.940+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 msgr2=0x5588e979af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.940+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e979af20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45a4096bf0 tx=0x7f45a401ca00 comp rx=0 tx=0).stop 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 shutdown_connections 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97a6900 0x5588e97a8d40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45977fe700 0 client.25691 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3829651975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e979af20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 >> 172.21.15.7:0/3829651975 conn(0x5588e9706740 msgr2=0x5588e978af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 shutdown_connections 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 1 -- 172.21.15.7:0/3829651975 wait complete. 2024-01-11T17:59:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.941+0000 7f45d2779b80 20 client.25691 trim_cache size 0 max 16384 2024-01-11T17:59:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.980+0000 7f45d2779b80 1 Processor -- start 2024-01-11T17:59:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.980+0000 7f45d2779b80 1 -- start start 2024-01-11T17:59:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.982+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e9791ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.982+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e9791c10 0x5588e97a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.982+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 0x5588e9799f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.982+0000 7f45d2779b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e955b640 con 0x5588e9790f20 2024-01-11T17:59:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.982+0000 7f45d2779b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e97adf90 con 0x5588e97925d0 2024-01-11T17:59:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.982+0000 7f45d2779b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e978cbe0 con 0x5588e9791c10 2024-01-11T17:59:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 0x5588e9799f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 0x5588e9799f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53248/0 (socket says 172.21.15.7:53248) 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 -- 172.21.15.7:0/4250512636 learned_addr learned my addr 172.21.15.7:0/4250512636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45b77fe700 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e9791c10 0x5588e97a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 -- 172.21.15.7:0/4250512636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e9791c10 msgr2=0x5588e97a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e9791c10 0x5588e97a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 -- 172.21.15.7:0/4250512636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 msgr2=0x5588e9791ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e9791ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.983+0000 7f45bcf9c700 1 -- 172.21.15.7:0/4250512636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e96a61d0 con 0x5588e97925d0 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.984+0000 7f45b77fe700 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e9791c10 0x5588e97a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.984+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 0x5588e9799f60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f45b0015000 tx=0x7f45b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2839cd0536f584 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.985+0000 7f45a1ffb700 1 -- 172.21.15.7:0/4250512636 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000e040 con 0x5588e97925d0 2024-01-11T17:59:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.985+0000 7f45a1ffb700 1 -- 172.21.15.7:0/4250512636 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b0004e60 con 0x5588e97925d0 2024-01-11T17:59:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.985+0000 7f45a1ffb700 1 -- 172.21.15.7:0/4250512636 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0005040 con 0x5588e97925d0 2024-01-11T17:59:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.986+0000 7f45d2779b80 1 -- 172.21.15.7:0/4250512636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 msgr2=0x5588e9799f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.986+0000 7f45d2779b80 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 0x5588e9799f60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f45b0015000 tx=0x7f45b000b330 comp rx=0 tx=0).stop 2024-01-11T17:59:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.986+0000 7f45d2779b80 1 -- 172.21.15.7:0/4250512636 shutdown_connections 2024-01-11T17:59:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.987+0000 7f45d2779b80 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e97925d0 0x5588e9799f60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.987+0000 7f45d2779b80 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e9791ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.987+0000 7f45d2779b80 1 --2- 172.21.15.7:0/4250512636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e9791c10 0x5588e97a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.987+0000 7f45d2779b80 1 -- 172.21.15.7:0/4250512636 >> 172.21.15.7:0/4250512636 conn(0x5588e9706740 msgr2=0x5588e9798c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.987+0000 7f45d2779b80 1 -- 172.21.15.7:0/4250512636 shutdown_connections 2024-01-11T17:59:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.988+0000 7f45d2779b80 1 -- 172.21.15.7:0/4250512636 wait complete. 2024-01-11T17:59:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.988+0000 7f45d2779b80 1 -- start start 2024-01-11T17:59:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.989+0000 7f45d2779b80 1 Processor -- start 2024-01-11T17:59:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e97a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e97a1810 con 0x5588e9790f20 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e97a1980 con 0x5588e9791c10 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e97a1b40 con 0x5588e97925d0 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45d2779b80 10 client.0 _ll_register_callbacks cb 0x5588e96856c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.990+0000 7f45bcf9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.991+0000 7f45bcf9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:47070/0 (socket says 172.21.15.7:47070) 2024-01-11T17:59:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.991+0000 7f45b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.991+0000 7f45bcf9c700 1 -- 172.21.15.7:0/3284796867 learned_addr learned my addr 172.21.15.7:0/3284796867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.991+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e97a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.991+0000 7f45b7fff700 1 -- 172.21.15.7:0/3284796867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 msgr2=0x5588e97a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.991+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.992+0000 7f45b7fff700 1 -- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 msgr2=0x5588e97a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.992+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e97a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.992+0000 7f45b7fff700 1 -- 172.21.15.7:0/3284796867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b8110970 con 0x5588e9790f20 2024-01-11T17:59:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.992+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e97a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.992+0000 7f45bcf9c700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.993+0000 7f45b7fff700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979fd80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5588e97a9660 tx=0x7f45ac005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931c45037d7f1ff4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.993+0000 7f45977fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:59:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.994+0000 7f45977fe700 1 -- 172.21.15.7:0/3284796867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac004830 con 0x5588e9790f20 2024-01-11T17:59:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.994+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45ac004cf0 con 0x5588e9790f20 2024-01-11T17:59:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.995+0000 7f45977fe700 1 -- 172.21.15.7:0/3284796867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f45ac014070 con 0x5588e9790f20 2024-01-11T17:59:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.995+0000 7f45977fe700 1 -- 172.21.15.7:0/3284796867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45ac004990 con 0x5588e9790f20 2024-01-11T17:59:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.996+0000 7f45d2779b80 10 client.15660 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.996+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5588e955dc70 con 0x5588e9790f20 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.997+0000 7f45977fe700 1 -- 172.21.15.7:0/3284796867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45ac004cf0 con 0x5588e9790f20 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.997+0000 7f45977fe700 1 -- 172.21.15.7:0/3284796867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f45ac00ea70 con 0x5588e9790f20 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.997+0000 7f45d2779b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.997+0000 7f45d2779b80 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.997+0000 7f45d2779b80 20 client.15660 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.998+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97fdea0 0x5588e98002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.998+0000 7f45d2779b80 4 client.15660 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:59:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.998+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588e96ddb60 con 0x5588e97fdea0 2024-01-11T17:59:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:00.999+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97fdea0 0x5588e98002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.000+0000 7f45b77fe700 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97fdea0 0x5588e98002c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2249607af5953960 server_cookie=34caab04f819b7e5 in_seq=0 out_seq=0 2024-01-11T17:59:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.000+0000 7f45977fe700 10 client.15660 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.089+0000 7f45977fe700 1 -- 172.21.15.7:0/3284796867 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588e96ddb60 con 0x5588e97fdea0 2024-01-11T17:59:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.089+0000 7f45977fe700 10 client.15660 handle_command_reply: tid=0 2024-01-11T17:59:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.089+0000 7f45d2779b80 1 client.15660 shutdown 2024-01-11T17:59:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 msgr2=0x5588e979fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979fd80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5588e97a9660 tx=0x7f45ac005c90 comp rx=0 tx=0).stop 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 shutdown_connections 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5588e97fdea0 0x5588e98002c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588e97925d0 0x5588e97a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588e9790f20 0x5588e979fd80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 --2- 172.21.15.7:0/3284796867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588e9791c10 0x5588e97a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 >> 172.21.15.7:0/3284796867 conn(0x5588e9706740 msgr2=0x5588e9799ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.090+0000 7f45977fe700 0 client.15660 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.091+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 shutdown_connections 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.091+0000 7f45d2779b80 1 -- 172.21.15.7:0/3284796867 wait complete. 2024-01-11T17:59:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.091+0000 7f45d2779b80 20 client.15660 trim_cache size 0 max 16384 2024-01-11T17:59:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:59:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:59:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:59:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:59:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45a80a2020 msgr2=0x7f45a80a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45a80a2020 0x7f45a80a44e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009f90 tx=0x7f45b000b680 comp rx=0 tx=0).stop 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 msgr2=0x7f45b819df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 0x7f45b819df80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c920 tx=0x7f45a400cce0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 shutdown_connections 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f45a80a2020 0x7f45a80a44e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45b8107be0 0x7f45b819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8109770 0x7f45b81a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 --2- 172.21.15.7:0/3562582722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b8108dd0 0x7f45b819df80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 >> 172.21.15.7:0/3562582722 conn(0x7f45b8076b60 msgr2=0x7f45b810c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 shutdown_connections 2024-01-11T17:59:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:01.092+0000 7f45be9ff700 1 -- 172.21.15.7:0/3562582722 wait complete. 2024-01-11T17:59:01.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:01 smithi016 ceph-mon[43195]: pgmap v336: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 906 op/s 2024-01-11T17:59:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:01 smithi038 ceph-mon[43582]: pgmap v336: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 906 op/s 2024-01-11T17:59:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:01 smithi007 ceph-mon[33361]: pgmap v336: 129 pgs: 129 active+clean; 1.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 906 op/s 2024-01-11T17:59:03.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:03 smithi016 ceph-mon[43195]: pgmap v337: 129 pgs: 129 active+clean; 1.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 896 op/s 2024-01-11T17:59:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:03 smithi038 ceph-mon[43582]: pgmap v337: 129 pgs: 129 active+clean; 1.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 896 op/s 2024-01-11T17:59:04.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:03 smithi007 ceph-mon[33361]: pgmap v337: 129 pgs: 129 active+clean; 1.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 896 op/s 2024-01-11T17:59:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 35.0 s, 1961.0 tps, lat 50.993 ms stddev 67.770 2024-01-11T17:59:05.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:05 smithi016 ceph-mon[43195]: pgmap v338: 129 pgs: 129 active+clean; 1.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 814 op/s 2024-01-11T17:59:05.777 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:05 smithi038 ceph-mon[43582]: pgmap v338: 129 pgs: 129 active+clean; 1.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 814 op/s 2024-01-11T17:59:05.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:05 smithi007 ceph-mon[33361]: pgmap v338: 129 pgs: 129 active+clean; 1.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 814 op/s 2024-01-11T17:59:07.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:07 smithi016 ceph-mon[43195]: pgmap v339: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 827 op/s 2024-01-11T17:59:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:07 smithi038 ceph-mon[43582]: pgmap v339: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 827 op/s 2024-01-11T17:59:08.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:07 smithi007 ceph-mon[33361]: pgmap v339: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 827 op/s 2024-01-11T17:59:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 40.0 s, 2049.0 tps, lat 48.692 ms stddev 60.684 2024-01-11T17:59:09.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:09 smithi016 ceph-mon[43195]: pgmap v340: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 668 op/s 2024-01-11T17:59:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:09 smithi038 ceph-mon[43582]: pgmap v340: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 668 op/s 2024-01-11T17:59:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:09 smithi007 ceph-mon[33361]: pgmap v340: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 668 op/s 2024-01-11T17:59:11.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:11 smithi016 ceph-mon[43195]: pgmap v341: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 MiB/s wr, 775 op/s 2024-01-11T17:59:12.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:11 smithi038 ceph-mon[43582]: pgmap v341: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 MiB/s wr, 775 op/s 2024-01-11T17:59:12.078 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:11 smithi007 ceph-mon[33361]: pgmap v341: 129 pgs: 129 active+clean; 1.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 MiB/s wr, 775 op/s 2024-01-11T17:59:12.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-01-11T17:59:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.102+0000 7f08b47ff700 1 -- 172.21.15.7:0/1850298865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac103190 msgr2=0x7f08ac109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.102+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1850298865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac103190 0x7f08ac109a40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08a80098f0 tx=0x7f08a8009c00 comp rx=0 tx=0).stop 2024-01-11T17:59:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.103+0000 7f08b47ff700 1 -- 172.21.15.7:0/1850298865 shutdown_connections 2024-01-11T17:59:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.103+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1850298865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac103190 0x7f08ac109a40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.103+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1850298865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac101600 0x7f08ac101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.103+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1850298865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 0x7f08ac102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.103+0000 7f08b47ff700 1 -- 172.21.15.7:0/1850298865 >> 172.21.15.7:0/1850298865 conn(0x7f08ac0fcb50 msgr2=0x7f08ac0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.112+0000 7f08b47ff700 1 -- 172.21.15.7:0/1850298865 shutdown_connections 2024-01-11T17:59:13.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.112+0000 7f08b47ff700 1 -- 172.21.15.7:0/1850298865 wait complete. 2024-01-11T17:59:13.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.114+0000 7f08b47ff700 1 Processor -- start 2024-01-11T17:59:13.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.114+0000 7f08b47ff700 1 -- start start 2024-01-11T17:59:13.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.115+0000 7f08b47ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 0x7f08ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.115+0000 7f08b47ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 0x7f08ac06f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.116+0000 7f08b47ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac103190 0x7f08ac072070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.116+0000 7f08b47ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac072640 con 0x7f08ac103190 2024-01-11T17:59:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.116+0000 7f08b47ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0727b0 con 0x7f08ac1027f0 2024-01-11T17:59:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.117+0000 7f08b2d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac103190 0x7f08ac072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.117+0000 7f08b47ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac072970 con 0x7f08ac101600 2024-01-11T17:59:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.117+0000 7f08b259b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 0x7f08ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.117+0000 7f08b259b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 0x7f08ac06eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43076/0 (socket says 172.21.15.7:43076) 2024-01-11T17:59:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.117+0000 7f08b259b700 1 -- 172.21.15.7:0/1002777764 learned_addr learned my addr 172.21.15.7:0/1002777764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.118+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 0x7f08ac06f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.119+0000 7f08b259b700 1 -- 172.21.15.7:0/1002777764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 msgr2=0x7f08ac06f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.119+0000 7f08b259b700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 0x7f08ac06f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.119+0000 7f08b259b700 1 -- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac103190 msgr2=0x7f08ac072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.119+0000 7f08b259b700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac103190 0x7f08ac072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.119+0000 7f08b259b700 1 -- 172.21.15.7:0/1002777764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009c40 con 0x7f08ac101600 2024-01-11T17:59:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.119+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 0x7f08ac06f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.120+0000 7f08b259b700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 0x7f08ac06eb30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c140 tx=0x7f08a400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.120+0000 7f08a37fe700 1 -- 172.21.15.7:0/1002777764 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400e050 con 0x7f08ac101600 2024-01-11T17:59:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.120+0000 7f08a37fe700 1 -- 172.21.15.7:0/1002777764 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a400f040 con 0x7f08ac101600 2024-01-11T17:59:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.120+0000 7f08a37fe700 1 -- 172.21.15.7:0/1002777764 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4013400 con 0x7f08ac101600 2024-01-11T17:59:13.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.120+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a8009520 con 0x7f08ac101600 2024-01-11T17:59:13.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.121+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac072f60 con 0x7f08ac101600 2024-01-11T17:59:13.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.134+0000 7f08c845ab80 1 Processor -- start 2024-01-11T17:59:13.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.134+0000 7f08c845ab80 1 -- start start 2024-01-11T17:59:13.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.134+0000 7f08a37fe700 1 -- 172.21.15.7:0/1002777764 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f08a4007500 con 0x7f08ac101600 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.137+0000 7f08a37fe700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08980a1fe0 0x7f08980a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.138+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d505030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.138+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d5037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.138+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d504200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.138+0000 7f08c845ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d3f2090 con 0x558c5d504c10 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.138+0000 7f08c845ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d455b20 con 0x558c5d503d80 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.138+0000 7f08c845ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d453440 con 0x558c5d505590 2024-01-11T17:59:13.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.139+0000 7f08b2d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d504200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.139+0000 7f08b2d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d504200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44574/0 (socket says 172.21.15.7:44574) 2024-01-11T17:59:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.139+0000 7f08b2d9c700 1 -- 172.21.15.7:0/2518413029 learned_addr learned my addr 172.21.15.7:0/2518413029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.139+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d5037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.140+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08980a1fe0 0x7f08980a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.140+0000 7f08b2d9c700 1 -- 172.21.15.7:0/2518413029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 msgr2=0x558c5d5037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.140+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d5037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.140+0000 7f08b2d9c700 1 -- 172.21.15.7:0/2518413029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 msgr2=0x558c5d505030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:59:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.140+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d505030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.140+0000 7f08b2d9c700 1 -- 172.21.15.7:0/2518413029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c5d2c33b0 con 0x558c5d503d80 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.141+0000 7f08a37fe700 1 -- 172.21.15.7:0/1002777764 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f08a40cdd70 con 0x7f08ac101600 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.141+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d504200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08a80098f0 tx=0x7f08a8010ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=512dde14efe84af0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.142+0000 7f08a0ff9700 1 -- 172.21.15.7:0/2518413029 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a801c030 con 0x558c5d503d80 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.142+0000 7f08a0ff9700 1 -- 172.21.15.7:0/2518413029 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a800ba40 con 0x558c5d503d80 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.142+0000 7f08a0ff9700 1 -- 172.21.15.7:0/2518413029 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8003b20 con 0x558c5d503d80 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 -- 172.21.15.7:0/2518413029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 msgr2=0x558c5d504200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d504200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08a80098f0 tx=0x7f08a8010ed0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 -- 172.21.15.7:0/2518413029 shutdown_connections 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d504200 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d505030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2518413029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d5037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 -- 172.21.15.7:0/2518413029 >> 172.21.15.7:0/2518413029 conn(0x558c5d4707a0 msgr2=0x558c5d4f5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 -- 172.21.15.7:0/2518413029 shutdown_connections 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.143+0000 7f08c845ab80 1 -- 172.21.15.7:0/2518413029 wait complete. 2024-01-11T17:59:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.144+0000 7f08c845ab80 1 -- start start 2024-01-11T17:59:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.145+0000 7f08c845ab80 1 Processor -- start 2024-01-11T17:59:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.145+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d4fc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.146+0000 7f08b259b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d4fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.146+0000 7f08b259b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d4fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44580/0 (socket says 172.21.15.7:44580) 2024-01-11T17:59:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.146+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08980a1fe0 0x7f08980a44a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08a800a010 tx=0x7f08a8028000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.146+0000 7f08b259b700 1 -- 172.21.15.7:0/2475773729 learned_addr learned my addr 172.21.15.7:0/2475773729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.146+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d4fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.146+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d501c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.147+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d44ed30 con 0x558c5d504c10 2024-01-11T17:59:13.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.147+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d501c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.147+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d4fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.147+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d3bd3a0 con 0x558c5d503d80 2024-01-11T17:59:13.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.147+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d3bd040 con 0x558c5d505590 2024-01-11T17:59:13.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.148+0000 7f08b259b700 1 -- 172.21.15.7:0/2475773729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 msgr2=0x558c5d501c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.148+0000 7f08b259b700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d501c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.148+0000 7f08b259b700 1 -- 172.21.15.7:0/2475773729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 msgr2=0x558c5d4fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.148+0000 7f08b259b700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d4fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.148+0000 7f08b259b700 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a801f040 con 0x558c5d503d80 2024-01-11T17:59:13.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.148+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d501c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.149+0000 7f08c845ab80 10 client.0 _ll_register_callbacks cb 0x558c5d3ef720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.149+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c5d4ee010 con 0x558c5d503d80 2024-01-11T17:59:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.149+0000 7f0895ffb700 10 client.25709 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:59:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.149+0000 7f08b259b700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d4fc750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e1b0 tx=0x7f08a409a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cfa17f35a9497c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.149+0000 7f0895ffb700 1 -- 172.21.15.7:0/2475773729 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a409ae40 con 0x558c5d503d80 2024-01-11T17:59:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.149+0000 7f0895ffb700 1 -- 172.21.15.7:0/2475773729 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f08a40ba0c0 con 0x558c5d503d80 2024-01-11T17:59:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f0895ffb700 1 -- 172.21.15.7:0/2475773729 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a409a740 con 0x558c5d503d80 2024-01-11T17:59:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f08c845ab80 10 client.25709 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x558c5d26fb60 con 0x558c5d503d80 2024-01-11T17:59:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f0895ffb700 1 -- 172.21.15.7:0/2475773729 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f08a4099c30 con 0x558c5d503d80 2024-01-11T17:59:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f08c845ab80 10 client.25709 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f08c845ab80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.150+0000 7f08c845ab80 20 client.25709 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.151+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d521890 0x558c5d523d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.151+0000 7f08c845ab80 4 client.25709 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.151+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c5d447bc0 con 0x558c5d521890 2024-01-11T17:59:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.151+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d521890 0x558c5d523d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.152+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d521890 0x558c5d523d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa156d1cc10b340b server_cookie=1f724f2b158e86fc in_seq=0 out_seq=0 2024-01-11T17:59:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.152+0000 7f0895ffb700 10 client.25709 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.152+0000 7f0895ffb700 1 -- 172.21.15.7:0/2475773729 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c5d447bc0 con 0x558c5d521890 2024-01-11T17:59:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.152+0000 7f0895ffb700 10 client.25709 handle_command_reply: tid=0 2024-01-11T17:59:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 client.25709 shutdown 2024-01-11T17:59:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 msgr2=0x558c5d4fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d4fc750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08a400e1b0 tx=0x7f08a409a280 comp rx=0 tx=0).stop 2024-01-11T17:59:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 shutdown_connections 2024-01-11T17:59:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d521890 0x558c5d523d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d501c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d4fc750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 --2- 172.21.15.7:0/2475773729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d4fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 >> 172.21.15.7:0/2475773729 conn(0x558c5d4707a0 msgr2=0x558c5d507c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f0895ffb700 0 client.25709 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.153+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 shutdown_connections 2024-01-11T17:59:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.154+0000 7f08c845ab80 1 -- 172.21.15.7:0/2475773729 wait complete. 2024-01-11T17:59:13.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.154+0000 7f08c845ab80 20 client.25709 trim_cache size 0 max 16384 2024-01-11T17:59:13.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.191+0000 7f08c845ab80 1 Processor -- start 2024-01-11T17:59:13.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.191+0000 7f08c845ab80 1 -- start start 2024-01-11T17:59:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d503d80 0x558c5d50b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d504c10 0x558c5d50b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d4fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08c845ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d2c56a0 con 0x558c5d503d80 2024-01-11T17:59:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08c845ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d4fd6a0 con 0x558c5d504c10 2024-01-11T17:59:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08c845ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d502a80 con 0x558c5d505590 2024-01-11T17:59:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.192+0000 7f08b2d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d4fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b2d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d4fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43098/0 (socket says 172.21.15.7:43098) 2024-01-11T17:59:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b2d9c700 1 -- 172.21.15.7:0/4115068116 learned_addr learned my addr 172.21.15.7:0/4115068116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d504c10 0x558c5d50b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b259b700 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d503d80 0x558c5d50b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b2d9c700 1 -- 172.21.15.7:0/4115068116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d504c10 msgr2=0x558c5d50b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d504c10 0x558c5d50b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b2d9c700 1 -- 172.21.15.7:0/4115068116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d503d80 msgr2=0x558c5d50b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.193+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d503d80 0x558c5d50b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.194+0000 7f08b2d9c700 1 -- 172.21.15.7:0/4115068116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c5d410230 con 0x558c5d505590 2024-01-11T17:59:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.194+0000 7f08b259b700 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d503d80 0x558c5d50b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.194+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d4fad10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f08a800fa70 tx=0x7f08a8010ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5457974133bbe97 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.194+0000 7f0897fff700 1 -- 172.21.15.7:0/4115068116 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8034040 con 0x558c5d505590 2024-01-11T17:59:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.195+0000 7f0897fff700 1 -- 172.21.15.7:0/4115068116 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a800b840 con 0x558c5d505590 2024-01-11T17:59:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.195+0000 7f0897fff700 1 -- 172.21.15.7:0/4115068116 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8034040 con 0x558c5d505590 2024-01-11T17:59:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 -- 172.21.15.7:0/4115068116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 msgr2=0x558c5d4fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d4fad10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f08a800fa70 tx=0x7f08a8010ed0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 -- 172.21.15.7:0/4115068116 shutdown_connections 2024-01-11T17:59:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d4fad10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d503d80 0x558c5d50b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 --2- 172.21.15.7:0/4115068116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d504c10 0x558c5d50b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 -- 172.21.15.7:0/4115068116 >> 172.21.15.7:0/4115068116 conn(0x558c5d4707a0 msgr2=0x558c5d5286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 -- 172.21.15.7:0/4115068116 shutdown_connections 2024-01-11T17:59:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.196+0000 7f08c845ab80 1 -- 172.21.15.7:0/4115068116 wait complete. 2024-01-11T17:59:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.197+0000 7f08c845ab80 1 -- start start 2024-01-11T17:59:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.197+0000 7f08c845ab80 1 Processor -- start 2024-01-11T17:59:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.198+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d570070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.198+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d5705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.198+0000 7f08c845ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d576fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.199+0000 7f08b259b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d570070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.199+0000 7f08b259b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d570070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44592/0 (socket says 172.21.15.7:44592) 2024-01-11T17:59:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.199+0000 7f08b259b700 1 -- 172.21.15.7:0/3541529285 learned_addr learned my addr 172.21.15.7:0/3541529285 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.199+0000 7f08b2d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d576fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.199+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d577520 con 0x558c5d504c10 2024-01-11T17:59:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.199+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d577660 con 0x558c5d503d80 2024-01-11T17:59:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5d5777f0 con 0x558c5d505590 2024-01-11T17:59:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08c845ab80 10 client.0 _ll_register_callbacks cb 0x558c5d3ef720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:13.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d5705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08b2d9c700 1 -- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 msgr2=0x558c5d570070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d570070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08b2d9c700 1 -- 172.21.15.7:0/3541529285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 msgr2=0x558c5d5705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:13.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d5705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.200+0000 7f08b2d9c700 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c5d2c33b0 con 0x558c5d505590 2024-01-11T17:59:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.201+0000 7f08b1d9a700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d5705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.201+0000 7f08b2d9c700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d576fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005a40 tx=0x7f08a8003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9692d8aeabdfc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.201+0000 7f0895ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.201+0000 7f0895ffb700 1 -- 172.21.15.7:0/3541529285 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8034070 con 0x558c5d505590 2024-01-11T17:59:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.201+0000 7f0895ffb700 1 -- 172.21.15.7:0/3541529285 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f08a800bbb0 con 0x558c5d505590 2024-01-11T17:59:13.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.201+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08a800ba40 con 0x558c5d505590 2024-01-11T17:59:13.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.202+0000 7f0895ffb700 1 -- 172.21.15.7:0/3541529285 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a801c040 con 0x558c5d505590 2024-01-11T17:59:13.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.202+0000 7f08c845ab80 10 client.25741 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:13.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.202+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558c5d2c7cd0 con 0x558c5d505590 2024-01-11T17:59:13.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.202+0000 7f0895ffb700 1 -- 172.21.15.7:0/3541529285 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a800ba40 con 0x558c5d505590 2024-01-11T17:59:13.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.202+0000 7f0895ffb700 1 -- 172.21.15.7:0/3541529285 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f08a8030940 con 0x558c5d505590 2024-01-11T17:59:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08c845ab80 10 client.25741 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08c845ab80 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08c845ab80 20 client.25741 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08c845ab80 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d578c70 0x558c5d57b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08c845ab80 4 client.25741 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-01-11T17:59:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c5d4478e0 con 0x558c5d578c70 2024-01-11T17:59:13.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.203+0000 7f08b259b700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d578c70 0x558c5d57b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.204+0000 7f08b259b700 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d578c70 0x558c5d57b060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1b56cf7b536e984 server_cookie=16d1993e5a01a0a0 in_seq=0 out_seq=0 2024-01-11T17:59:13.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.204+0000 7f0895ffb700 10 client.25741 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:13.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f0895ffb700 1 -- 172.21.15.7:0/3541529285 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558c5d4478e0 con 0x558c5d578c70 2024-01-11T17:59:13.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f0895ffb700 10 client.25741 handle_command_reply: tid=0 2024-01-11T17:59:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 client.25741 shutdown 2024-01-11T17:59:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 msgr2=0x558c5d576fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d576fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005a40 tx=0x7f08a8003b80 comp rx=0 tx=0).stop 2024-01-11T17:59:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 shutdown_connections 2024-01-11T17:59:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558c5d578c70 0x558c5d57b060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558c5d505590 0x558c5d576fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c5d503d80 0x558c5d570070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.205+0000 7f08c845ab80 1 --2- 172.21.15.7:0/3541529285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c5d504c10 0x558c5d5705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.206+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 >> 172.21.15.7:0/3541529285 conn(0x558c5d4707a0 msgr2=0x558c5d50b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.206+0000 7f0895ffb700 0 client.25741 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.206+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 shutdown_connections 2024-01-11T17:59:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.206+0000 7f08c845ab80 1 -- 172.21.15.7:0/3541529285 wait complete. 2024-01-11T17:59:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.206+0000 7f08c845ab80 20 client.25741 trim_cache size 0 max 16384 2024-01-11T17:59:13.217 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T17:59:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-01-11T17:59:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-01-11T17:59:13.218 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T17:59:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08980a1fe0 msgr2=0x7f08980a44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08980a1fe0 0x7f08980a44a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08a800a010 tx=0x7f08a8028000 comp rx=0 tx=0).stop 2024-01-11T17:59:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 msgr2=0x7f08ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 0x7f08ac06eb30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c140 tx=0x7f08a400c450 comp rx=0 tx=0).stop 2024-01-11T17:59:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 shutdown_connections 2024-01-11T17:59:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f08980a1fe0 0x7f08980a44a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08ac101600 0x7f08ac06eb30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac103190 0x7f08ac072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 --2- 172.21.15.7:0/1002777764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08ac1027f0 0x7f08ac06f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 >> 172.21.15.7:0/1002777764 conn(0x7f08ac0fcb50 msgr2=0x7f08ac0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 shutdown_connections 2024-01-11T17:59:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.207+0000 7f08b47ff700 1 -- 172.21.15.7:0/1002777764 wait complete. 2024-01-11T17:59:13.239 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-01-11T17:59:13.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T17:59:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.533+0000 7f84badff700 1 -- 172.21.15.7:0/3513060698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41016e0 msgr2=0x7f84b4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.533+0000 7f84badff700 1 --2- 172.21.15.7:0/3513060698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41016e0 0x7f84b4101b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009990 tx=0x7f84a8009ca0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 -- 172.21.15.7:0/3513060698 shutdown_connections 2024-01-11T17:59:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 --2- 172.21.15.7:0/3513060698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b4102080 0x7f84b4108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 --2- 172.21.15.7:0/3513060698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b41004f0 0x7f84b4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 --2- 172.21.15.7:0/3513060698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41016e0 0x7f84b4101b40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 -- 172.21.15.7:0/3513060698 >> 172.21.15.7:0/3513060698 conn(0x7f84b40fba60 msgr2=0x7f84b40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 -- 172.21.15.7:0/3513060698 shutdown_connections 2024-01-11T17:59:13.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.534+0000 7f84badff700 1 -- 172.21.15.7:0/3513060698 wait complete. 2024-01-11T17:59:13.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.536+0000 7f84badff700 1 Processor -- start 2024-01-11T17:59:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.536+0000 7f84badff700 1 -- start start 2024-01-11T17:59:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84badff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41004f0 0x7f84b419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84badff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b41016e0 0x7f84b419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84badff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b4102080 0x7f84b41a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84badff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41a5640 con 0x7f84b4102080 2024-01-11T17:59:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84badff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41a5780 con 0x7f84b41004f0 2024-01-11T17:59:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84badff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41a5910 con 0x7f84b41016e0 2024-01-11T17:59:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84b8b9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41004f0 0x7f84b419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84b939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b4102080 0x7f84b41a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84b3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b41016e0 0x7f84b419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84b8b9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41004f0 0x7f84b419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44606/0 (socket says 172.21.15.7:44606) 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.537+0000 7f84b8b9b700 1 -- 172.21.15.7:0/2344596185 learned_addr learned my addr 172.21.15.7:0/2344596185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.538+0000 7f84b3fff700 1 -- 172.21.15.7:0/2344596185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41004f0 msgr2=0x7f84b419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.538+0000 7f84b3fff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41004f0 0x7f84b419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.538+0000 7f84b3fff700 1 -- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b4102080 msgr2=0x7f84b41a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.538+0000 7f84b3fff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b4102080 0x7f84b41a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.538+0000 7f84b3fff700 1 -- 172.21.15.7:0/2344596185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80095f0 con 0x7f84b41016e0 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.538+0000 7f84b939c700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b4102080 0x7f84b41a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.539+0000 7f84b3fff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b41016e0 0x7f84b419e000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006010 tx=0x7f84a800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.540+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a801d070 con 0x7f84b41016e0 2024-01-11T17:59:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.540+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b41a5b90 con 0x7f84b41016e0 2024-01-11T17:59:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.540+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a8005510 con 0x7f84b41016e0 2024-01-11T17:59:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.540+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a801d070 con 0x7f84b41016e0 2024-01-11T17:59:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.540+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b41a6010 con 0x7f84b41016e0 2024-01-11T17:59:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.542+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b41078a0 con 0x7f84b41016e0 2024-01-11T17:59:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.545+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f84a800fb90 con 0x7f84b41016e0 2024-01-11T17:59:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.547+0000 7f84b1ffb700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f849c0a1fa0 0x7f849c0a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.547+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f84a80ceaa0 con 0x7f84b41016e0 2024-01-11T17:59:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.549+0000 7f84b8b9b700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f849c0a1fa0 0x7f849c0a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.552+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f84a80979c0 con 0x7f84b41016e0 2024-01-11T17:59:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:13.552+0000 7f84b8b9b700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f849c0a1fa0 0x7f849c0a4460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f84a4007cf0 tx=0x7f84a4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:13.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:13 smithi016 ceph-mon[43195]: pgmap v342: 129 pgs: 129 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 797 op/s 2024-01-11T17:59:14.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:13 smithi038 ceph-mon[43582]: pgmap v342: 129 pgs: 129 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 797 op/s 2024-01-11T17:59:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.066+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84b40683f0 con 0x7f84b41016e0 2024-01-11T17:59:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.067+0000 7f84b1ffb700 1 -- 172.21.15.7:0/2344596185 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11316 (secure 0 0 0) 0x7f84a8026090 con 0x7f84b41016e0 2024-01-11T17:59:14.075 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:13 smithi007 ceph-mon[33361]: pgmap v342: 129 pgs: 129 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 797 op/s 2024-01-11T17:59:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T17:59:14.075 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T17:58:51.313874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/2512926203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2512926203},{"type":"v1","addr":"172.21.15.16:6837","nonce":2512926203}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":45,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":43,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T17:59:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f849c0a1fa0 msgr2=0x7f849c0a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f849c0a1fa0 0x7f849c0a4460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f84a4007cf0 tx=0x7f84a4007480 comp rx=0 tx=0).stop 2024-01-11T17:59:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b41016e0 msgr2=0x7f84b419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b41016e0 0x7f84b419e000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006010 tx=0x7f84a800f690 comp rx=0 tx=0).stop 2024-01-11T17:59:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 shutdown_connections 2024-01-11T17:59:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f849c0a1fa0 0x7f849c0a4460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b4102080 0x7f84b41a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84b41004f0 0x7f84b419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 --2- 172.21.15.7:0/2344596185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84b41016e0 0x7f84b419e000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 >> 172.21.15.7:0/2344596185 conn(0x7f84b40fba60 msgr2=0x7f84b41049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 shutdown_connections 2024-01-11T17:59:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.075+0000 7f84badff700 1 -- 172.21.15.7:0/2344596185 wait complete. 2024-01-11T17:59:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-01-11T17:59:14.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-01-11T17:59:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 45.0 s, 1140.4 tps, lat 87.047 ms stddev 113.246 2024-01-11T17:59:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 -- 172.21.15.7:0/400772015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc881049e0 msgr2=0x7efc88104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 --2- 172.21.15.7:0/400772015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc881049e0 0x7efc88104e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b0d0 tx=0x7efc7800b3e0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 -- 172.21.15.7:0/400772015 shutdown_connections 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 --2- 172.21.15.7:0/400772015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc88105380 0x7efc8810bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 --2- 172.21.15.7:0/400772015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc881037f0 0x7efc88103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 --2- 172.21.15.7:0/400772015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc881049e0 0x7efc88104e40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.406+0000 7efc8ddff700 1 -- 172.21.15.7:0/400772015 >> 172.21.15.7:0/400772015 conn(0x7efc880fed60 msgr2=0x7efc881011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.407+0000 7efc8ddff700 1 -- 172.21.15.7:0/400772015 shutdown_connections 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.407+0000 7efc8ddff700 1 -- 172.21.15.7:0/400772015 wait complete. 2024-01-11T17:59:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.408+0000 7efc8ddff700 1 Processor -- start 2024-01-11T17:59:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.409+0000 7efc8ddff700 1 -- start start 2024-01-11T17:59:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.410+0000 7efc8ddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc881037f0 0x7efc88070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.410+0000 7efc8ddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc881049e0 0x7efc88070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.410+0000 7efc8ddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc88105380 0x7efc881a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.410+0000 7efc8ddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc88070bc0 con 0x7efc88105380 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.410+0000 7efc8ddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc881a9f40 con 0x7efc881049e0 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.410+0000 7efc8ddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc881aa080 con 0x7efc881037f0 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.411+0000 7efc877fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc881037f0 0x7efc88070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.411+0000 7efc877fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc881037f0 0x7efc88070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43142/0 (socket says 172.21.15.7:43142) 2024-01-11T17:59:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.411+0000 7efc877fe700 1 -- 172.21.15.7:0/1595153175 learned_addr learned my addr 172.21.15.7:0/1595153175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.412+0000 7efc87fff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc88105380 0x7efc881a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.412+0000 7efc7effd700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc881049e0 0x7efc88070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.413+0000 7efc87fff700 1 -- 172.21.15.7:0/1595153175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc881037f0 msgr2=0x7efc88070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.413+0000 7efc87fff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc881037f0 0x7efc88070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.413+0000 7efc87fff700 1 -- 172.21.15.7:0/1595153175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc881049e0 msgr2=0x7efc88070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.413+0000 7efc87fff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc881049e0 0x7efc88070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.413+0000 7efc87fff700 1 -- 172.21.15.7:0/1595153175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78009d00 con 0x7efc88105380 2024-01-11T17:59:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.414+0000 7efc87fff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc88105380 0x7efc881a9a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efc8000b3f0 tx=0x7efc8000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.415+0000 7efc857fa700 1 -- 172.21.15.7:0/1595153175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80011830 con 0x7efc88105380 2024-01-11T17:59:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.415+0000 7efc857fa700 1 -- 172.21.15.7:0/1595153175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8000be50 con 0x7efc88105380 2024-01-11T17:59:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.415+0000 7efc857fa700 1 -- 172.21.15.7:0/1595153175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80010d50 con 0x7efc88105380 2024-01-11T17:59:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.415+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc88075ab0 con 0x7efc88105380 2024-01-11T17:59:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.415+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc88075ff0 con 0x7efc88105380 2024-01-11T17:59:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.424+0000 7efca1c1fb80 1 Processor -- start 2024-01-11T17:59:14.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.424+0000 7efc857fa700 1 -- 172.21.15.7:0/1595153175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7efc80010380 con 0x7efc88105380 2024-01-11T17:59:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 -- start start 2024-01-11T17:59:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c375f20 0x55bd2c376340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c377090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c37de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c26d030 con 0x55bd2c375f20 2024-01-11T17:59:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c2d0ac0 con 0x55bd2c376c10 2024-01-11T17:59:14.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.426+0000 7efca1c1fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c2ce3e0 con 0x55bd2c3775d0 2024-01-11T17:59:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.427+0000 7efc7effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c375f20 0x55bd2c376340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.427+0000 7efc7effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c37de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.427+0000 7efc7effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c37de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43158/0 (socket says 172.21.15.7:43158) 2024-01-11T17:59:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.427+0000 7efc7effd700 1 -- 172.21.15.7:0/4246804470 learned_addr learned my addr 172.21.15.7:0/4246804470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc877fe700 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c377090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc7effd700 1 -- 172.21.15.7:0/4246804470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 msgr2=0x55bd2c377090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc7effd700 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c377090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc7effd700 1 -- 172.21.15.7:0/4246804470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c375f20 msgr2=0x55bd2c376340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc7effd700 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c375f20 0x55bd2c376340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc7effd700 1 -- 172.21.15.7:0/4246804470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd2c13e350 con 0x55bd2c3775d0 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc877fe700 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c377090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc857fa700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc740a2020 0x7efc740a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.430+0000 7efc7effd700 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c37de60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efc780060b0 tx=0x7efc7800b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce37049dc360b54 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.431+0000 7efc857fa700 1 -- 172.21.15.7:0/1595153175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efc800cd550 con 0x7efc88105380 2024-01-11T17:59:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.431+0000 7efc877fe700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc740a2020 0x7efc740a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.432+0000 7efc7dffb700 1 -- 172.21.15.7:0/4246804470 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78027070 con 0x55bd2c3775d0 2024-01-11T17:59:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.432+0000 7efc7dffb700 1 -- 172.21.15.7:0/4246804470 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc78022e10 con 0x55bd2c3775d0 2024-01-11T17:59:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.432+0000 7efc7dffb700 1 -- 172.21.15.7:0/4246804470 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7800e040 con 0x55bd2c3775d0 2024-01-11T17:59:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.433+0000 7efca1c1fb80 1 -- 172.21.15.7:0/4246804470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 msgr2=0x55bd2c37de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.433+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c37de60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efc780060b0 tx=0x7efc7800b3e0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.434+0000 7efca1c1fb80 1 -- 172.21.15.7:0/4246804470 shutdown_connections 2024-01-11T17:59:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.434+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c37de60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.434+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c375f20 0x55bd2c376340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.434+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/4246804470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c377090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.434+0000 7efca1c1fb80 1 -- 172.21.15.7:0/4246804470 >> 172.21.15.7:0/4246804470 conn(0x55bd2c2eb740 msgr2=0x55bd2c370950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.435+0000 7efca1c1fb80 1 -- 172.21.15.7:0/4246804470 shutdown_connections 2024-01-11T17:59:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.435+0000 7efc877fe700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc740a2020 0x7efc740a44e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55bd2c3727f0 tx=0x7efc70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.435+0000 7efca1c1fb80 1 -- 172.21.15.7:0/4246804470 wait complete. 2024-01-11T17:59:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.436+0000 7efca1c1fb80 1 -- start start 2024-01-11T17:59:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.436+0000 7efca1c1fb80 1 Processor -- start 2024-01-11T17:59:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c37f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c376c10 0x55bd2c37fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c386e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c2c9cd0 con 0x55bd2c3775d0 2024-01-11T17:59:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c238340 con 0x55bd2c375f20 2024-01-11T17:59:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c237fe0 con 0x55bd2c376c10 2024-01-11T17:59:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efca1c1fb80 10 client.0 _ll_register_callbacks cb 0x55bd2c26a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efc877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c37f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efc877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c37f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44640/0 (socket says 172.21.15.7:44640) 2024-01-11T17:59:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.438+0000 7efc877fe700 1 -- 172.21.15.7:0/3788101394 learned_addr learned my addr 172.21.15.7:0/3788101394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc7effd700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c376c10 0x55bd2c37fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc87fff700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c386e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc877fe700 1 -- 172.21.15.7:0/3788101394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c376c10 msgr2=0x55bd2c37fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc877fe700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c376c10 0x55bd2c37fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc877fe700 1 -- 172.21.15.7:0/3788101394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 msgr2=0x55bd2c386e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc877fe700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c386e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.439+0000 7efc877fe700 1 -- 172.21.15.7:0/3788101394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8810c580 con 0x55bd2c375f20 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.440+0000 7efc877fe700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c37f930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc700156f0 tx=0x7efc70004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73791b2759e8ce9c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.440+0000 7efc66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:59:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.441+0000 7efc66ffd700 1 -- 172.21.15.7:0/3788101394 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70016830 con 0x55bd2c375f20 2024-01-11T17:59:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.441+0000 7efc66ffd700 1 -- 172.21.15.7:0/3788101394 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efc70016990 con 0x55bd2c375f20 2024-01-11T17:59:14.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.442+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd2c368d60 con 0x55bd2c375f20 2024-01-11T17:59:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.443+0000 7efc66ffd700 1 -- 172.21.15.7:0/3788101394 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc70016b50 con 0x55bd2c375f20 2024-01-11T17:59:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.444+0000 7efca1c1fb80 10 client.25727 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.444+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bd2c0eab00 con 0x55bd2c375f20 2024-01-11T17:59:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.444+0000 7efc66ffd700 1 -- 172.21.15.7:0/3788101394 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd2c368d60 con 0x55bd2c375f20 2024-01-11T17:59:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efc66ffd700 1 -- 172.21.15.7:0/3788101394 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efc700143d0 con 0x55bd2c375f20 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efca1c1fb80 10 client.25727 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efca1c1fb80 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efca1c1fb80 20 client.25727 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c38b7c0 0x55bd2c38dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efca1c1fb80 4 client.25727 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.445+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd2c2c2b60 con 0x55bd2c38b7c0 2024-01-11T17:59:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.446+0000 7efc7effd700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c38b7c0 0x55bd2c38dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.448+0000 7efc66ffd700 10 client.25727 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.448+0000 7efc7effd700 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c38b7c0 0x55bd2c38dbe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49cf1333ccb039b1 server_cookie=27e82bced788016c in_seq=0 out_seq=0 2024-01-11T17:59:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.449+0000 7efc66ffd700 1 -- 172.21.15.7:0/3788101394 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd2c2c2b60 con 0x55bd2c38b7c0 2024-01-11T17:59:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.449+0000 7efc66ffd700 10 client.25727 handle_command_reply: tid=0 2024-01-11T17:59:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.449+0000 7efca1c1fb80 1 client.25727 shutdown 2024-01-11T17:59:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 msgr2=0x55bd2c37f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c37f930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc700156f0 tx=0x7efc70004750 comp rx=0 tx=0).stop 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 shutdown_connections 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c38b7c0 0x55bd2c38dbe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c386e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c37f930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/3788101394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c376c10 0x55bd2c37fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 >> 172.21.15.7:0/3788101394 conn(0x55bd2c2eb740 msgr2=0x55bd2c36ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efc66ffd700 0 client.25727 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.450+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 shutdown_connections 2024-01-11T17:59:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.453+0000 7efca1c1fb80 1 -- 172.21.15.7:0/3788101394 wait complete. 2024-01-11T17:59:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.453+0000 7efca1c1fb80 20 client.25727 trim_cache size 0 max 16384 2024-01-11T17:59:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.489+0000 7efca1c1fb80 1 Processor -- start 2024-01-11T17:59:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.489+0000 7efca1c1fb80 1 -- start start 2024-01-11T17:59:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.490+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c375f20 0x55bd2c38dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.490+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c386df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.490+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c37eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.490+0000 7efc7effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c386df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.490+0000 7efc7effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c386df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44644/0 (socket says 172.21.15.7:44644) 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.490+0000 7efc87fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c37eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.491+0000 7efc87fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c37eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43622/0 (socket says 172.21.15.7:43622) 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.491+0000 7efc87fff700 1 -- 172.21.15.7:0/1594184493 learned_addr learned my addr 172.21.15.7:0/1594184493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.491+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c140640 con 0x55bd2c3775d0 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.491+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c388ae0 con 0x55bd2c376c10 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.491+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c36d560 con 0x55bd2c375f20 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.491+0000 7efc877fe700 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c375f20 0x55bd2c38dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc87fff700 1 -- 172.21.15.7:0/1594184493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c375f20 msgr2=0x55bd2c38dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc87fff700 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c375f20 0x55bd2c38dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc87fff700 1 -- 172.21.15.7:0/1594184493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 msgr2=0x55bd2c386df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc87fff700 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c386df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc87fff700 1 -- 172.21.15.7:0/1594184493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd2c28b1d0 con 0x55bd2c3775d0 2024-01-11T17:59:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc877fe700 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c375f20 0x55bd2c38dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.492+0000 7efc7effd700 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c386df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.493+0000 7efc87fff700 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c37eeb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efc800cad70 tx=0x7efc80004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a4ff16ae45b24 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.493+0000 7efc7d7fa700 1 -- 172.21.15.7:0/1594184493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80009180 con 0x55bd2c3775d0 2024-01-11T17:59:14.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.493+0000 7efc7d7fa700 1 -- 172.21.15.7:0/1594184493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc80099f80 con 0x55bd2c3775d0 2024-01-11T17:59:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.493+0000 7efc7d7fa700 1 -- 172.21.15.7:0/1594184493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8009a1e0 con 0x55bd2c3775d0 2024-01-11T17:59:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 msgr2=0x55bd2c37eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c37eeb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efc800cad70 tx=0x7efc80004860 comp rx=0 tx=0).stop 2024-01-11T17:59:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 shutdown_connections 2024-01-11T17:59:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c375f20 0x55bd2c38dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c3775d0 0x55bd2c37eeb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/1594184493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c376c10 0x55bd2c386df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 >> 172.21.15.7:0/1594184493 conn(0x55bd2c2eb740 msgr2=0x55bd2c36c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.494+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 shutdown_connections 2024-01-11T17:59:14.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.495+0000 7efca1c1fb80 1 -- 172.21.15.7:0/1594184493 wait complete. 2024-01-11T17:59:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.495+0000 7efca1c1fb80 1 -- start start 2024-01-11T17:59:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.496+0000 7efca1c1fb80 1 Processor -- start 2024-01-11T17:59:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.498+0000 7efca1c1fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c384620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.499+0000 7efc877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c384620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.499+0000 7efc877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c384620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44646/0 (socket says 172.21.15.7:44646) 2024-01-11T17:59:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.499+0000 7efc877fe700 1 -- 172.21.15.7:0/2125247579 learned_addr learned my addr 172.21.15.7:0/2125247579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.500+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c376c10 0x55bd2c384b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.500+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c385b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.500+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c386060 con 0x55bd2c376c10 2024-01-11T17:59:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.500+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c3861d0 con 0x55bd2c375f20 2024-01-11T17:59:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.500+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd2c386390 con 0x55bd2c3775d0 2024-01-11T17:59:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.500+0000 7efca1c1fb80 10 client.0 _ll_register_callbacks cb 0x55bd2c26a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.501+0000 7efc87fff700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c385b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.502+0000 7efc877fe700 1 -- 172.21.15.7:0/2125247579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 msgr2=0x55bd2c385b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.502+0000 7efc877fe700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c385b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.502+0000 7efc877fe700 1 -- 172.21.15.7:0/2125247579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c376c10 msgr2=0x55bd2c384b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.502+0000 7efc877fe700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c376c10 0x55bd2c384b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.502+0000 7efc877fe700 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd2c0eab00 con 0x55bd2c375f20 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.504+0000 7efc87fff700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c385b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.504+0000 7efc877fe700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c384620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efc700048a0 tx=0x7efc70014d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27fd92ca516a78b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.506+0000 7efc66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.506+0000 7efc66ffd700 1 -- 172.21.15.7:0/2125247579 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70016a40 con 0x55bd2c375f20 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.506+0000 7efc66ffd700 1 -- 172.21.15.7:0/2125247579 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efc70016ba0 con 0x55bd2c375f20 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.506+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc78022e10 con 0x55bd2c375f20 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.507+0000 7efc66ffd700 1 -- 172.21.15.7:0/2125247579 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc70016d60 con 0x55bd2c375f20 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.507+0000 7efca1c1fb80 10 client.25733 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:14.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.507+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bd2c13e350 con 0x55bd2c375f20 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.508+0000 7efc66ffd700 1 -- 172.21.15.7:0/2125247579 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc78022e10 con 0x55bd2c375f20 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efc66ffd700 1 -- 172.21.15.7:0/2125247579 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efc70021440 con 0x55bd2c375f20 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efca1c1fb80 10 client.25733 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efca1c1fb80 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efca1c1fb80 20 client.25733 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c3e2750 0x55bd2c3e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efca1c1fb80 4 client.25733 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.509+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bd2c2c2880 con 0x55bd2c3e2750 2024-01-11T17:59:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.510+0000 7efc7effd700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c3e2750 0x55bd2c3e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.510+0000 7efc7effd700 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c3e2750 0x55bd2c3e4b70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8cd0db9995538f server_cookie=10b87191376e59b7 in_seq=0 out_seq=0 2024-01-11T17:59:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.510+0000 7efc66ffd700 10 client.25733 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.542+0000 7efc66ffd700 1 -- 172.21.15.7:0/2125247579 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bd2c2c2880 con 0x55bd2c3e2750 2024-01-11T17:59:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.542+0000 7efc66ffd700 10 client.25733 handle_command_reply: tid=0 2024-01-11T17:59:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.542+0000 7efca1c1fb80 1 client.25733 shutdown 2024-01-11T17:59:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.542+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 msgr2=0x55bd2c384620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.542+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c384620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efc700048a0 tx=0x7efc70014d00 comp rx=0 tx=0).stop 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 shutdown_connections 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bd2c3e2750 0x55bd2c3e4b70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bd2c3775d0 0x55bd2c385b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2c375f20 0x55bd2c384620 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 --2- 172.21.15.7:0/2125247579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd2c376c10 0x55bd2c384b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 >> 172.21.15.7:0/2125247579 conn(0x55bd2c2eb740 msgr2=0x55bd2c37eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efc66ffd700 0 client.25733 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 shutdown_connections 2024-01-11T17:59:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.543+0000 7efca1c1fb80 1 -- 172.21.15.7:0/2125247579 wait complete. 2024-01-11T17:59:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.544+0000 7efca1c1fb80 20 client.25733 trim_cache size 0 max 16384 2024-01-11T17:59:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.546+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc740a2020 msgr2=0x7efc740a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.546+0000 7efc8ddff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc740a2020 0x7efc740a44e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55bd2c3727f0 tx=0x7efc70008040 comp rx=0 tx=0).stop 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.546+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc88105380 msgr2=0x7efc881a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.546+0000 7efc8ddff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc88105380 0x7efc881a9a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efc8000b3f0 tx=0x7efc8000b700 comp rx=0 tx=0).stop 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.547+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 shutdown_connections 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.547+0000 7efc8ddff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc740a2020 0x7efc740a44e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.547+0000 7efc8ddff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc881037f0 0x7efc88070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.547+0000 7efc8ddff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc88105380 0x7efc881a9a00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.547+0000 7efc8ddff700 1 --2- 172.21.15.7:0/1595153175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc881049e0 0x7efc88070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.547+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 >> 172.21.15.7:0/1595153175 conn(0x7efc880fed60 msgr2=0x7efc88107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.548+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 shutdown_connections 2024-01-11T17:59:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:14.548+0000 7efc8ddff700 1 -- 172.21.15.7:0/1595153175 wait complete. 2024-01-11T17:59:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.581+0000 7f6348fff700 1 -- 172.21.15.7:0/1587341922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 msgr2=0x7f634410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.581+0000 7f6348fff700 1 --2- 172.21.15.7:0/1587341922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634410abe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6334009990 tx=0x7f6334009ca0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 -- 172.21.15.7:0/1587341922 shutdown_connections 2024-01-11T17:59:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 --2- 172.21.15.7:0/1587341922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634410abe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 --2- 172.21.15.7:0/1587341922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634410dcb0 0x7f6344110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 --2- 172.21.15.7:0/1587341922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634410b1b0 0x7f634410d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 -- 172.21.15.7:0/1587341922 >> 172.21.15.7:0/1587341922 conn(0x7f634406e530 msgr2=0x7f6344070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 -- 172.21.15.7:0/1587341922 shutdown_connections 2024-01-11T17:59:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.582+0000 7f6348fff700 1 -- 172.21.15.7:0/1587341922 wait complete. 2024-01-11T17:59:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.584+0000 7f6348fff700 1 Processor -- start 2024-01-11T17:59:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.585+0000 7f6348fff700 1 -- start start 2024-01-11T17:59:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.586+0000 7f6348fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.586+0000 7f6348fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634410b1b0 0x7f634419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.586+0000 7f6348fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634410dcb0 0x7f63441a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f6348fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a9ca0 con 0x7f634410dcb0 2024-01-11T17:59:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f6348fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a9de0 con 0x7f634410b1b0 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f634259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f634259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43178/0 (socket says 172.21.15.7:43178) 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f634259c700 1 -- 172.21.15.7:0/3973789819 learned_addr learned my addr 172.21.15.7:0/3973789819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f6339d9b700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634410b1b0 0x7f634419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441a9f20 con 0x7f63441087f0 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.587+0000 7f6342d9d700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634410dcb0 0x7f63441a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.588+0000 7f634259c700 1 -- 172.21.15.7:0/3973789819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634410b1b0 msgr2=0x7f634419e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.588+0000 7f634259c700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634410b1b0 0x7f634419e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.588+0000 7f634259c700 1 -- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634410dcb0 msgr2=0x7f63441a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.588+0000 7f634259c700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634410dcb0 0x7f63441a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.588+0000 7f634259c700 1 -- 172.21.15.7:0/3973789819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340095f0 con 0x7f63441087f0 2024-01-11T17:59:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.590+0000 7f634259c700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634419db60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63340038c0 tx=0x7f6334003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.591+0000 7f633bfff700 1 -- 172.21.15.7:0/3973789819 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633401d070 con 0x7f63441087f0 2024-01-11T17:59:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.592+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63441aa120 con 0x7f63441087f0 2024-01-11T17:59:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.592+0000 7f633bfff700 1 -- 172.21.15.7:0/3973789819 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6334005580 con 0x7f63441087f0 2024-01-11T17:59:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.592+0000 7f633bfff700 1 -- 172.21.15.7:0/3973789819 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400fe60 con 0x7f63441087f0 2024-01-11T17:59:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.593+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63441aa540 con 0x7f63441087f0 2024-01-11T17:59:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.600+0000 7f633bfff700 1 -- 172.21.15.7:0/3973789819 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f63340050c0 con 0x7f63441087f0 2024-01-11T17:59:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.608+0000 7f635cc3db80 1 Processor -- start 2024-01-11T17:59:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.608+0000 7f635cc3db80 1 -- start start 2024-01-11T17:59:14.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.608+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b01030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.609+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 0x565416aff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.609+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b00200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.607+0000 7f633bfff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63240a2010 0x7f63240a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.609+0000 7f6339d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 0x565416aff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.609+0000 7f633bfff700 1 -- 172.21.15.7:0/3973789819 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f63340d0030 con 0x7f63441087f0 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.609+0000 7f6339d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 0x565416aff7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44674/0 (socket says 172.21.15.7:44674) 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.609+0000 7f634259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b01030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.610+0000 7f634259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b01030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43648/0 (socket says 172.21.15.7:43648) 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.610+0000 7f635cc3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654169ee090 con 0x565416b00c10 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.610+0000 7f635cc3db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416a51b20 con 0x565416b01590 2024-01-11T17:59:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.610+0000 7f635cc3db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416a4f440 con 0x565416affd80 2024-01-11T17:59:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.611+0000 7f6339d9b700 1 -- 172.21.15.7:0/3792874317 learned_addr learned my addr 172.21.15.7:0/3792874317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.611+0000 7f6339d9b700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63240a2010 0x7f63240a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.612+0000 7f6342d9d700 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b00200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.613+0000 7f634259c700 1 -- 172.21.15.7:0/3792874317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 msgr2=0x565416b00200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.613+0000 7f634259c700 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b00200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.613+0000 7f634259c700 1 -- 172.21.15.7:0/3792874317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 msgr2=0x565416aff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.613+0000 7f634259c700 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 0x565416aff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.613+0000 7f634259c700 1 -- 172.21.15.7:0/3792874317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654168bf3b0 con 0x565416b00c10 2024-01-11T17:59:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.614+0000 7f6339d9b700 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 0x565416aff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:14.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.615+0000 7f634259c700 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b01030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6344110840 tx=0x7f6334009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6887b63aa97c4fe0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.616+0000 7f6339d9b700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63240a2010 0x7f63240a44d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f632c0155a0 tx=0x7f632c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.617+0000 7f6338d99700 1 -- 172.21.15.7:0/3792874317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340b2070 con 0x565416b00c10 2024-01-11T17:59:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.617+0000 7f6338d99700 1 -- 172.21.15.7:0/3792874317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63340cf020 con 0x565416b00c10 2024-01-11T17:59:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.617+0000 7f6338d99700 1 -- 172.21.15.7:0/3792874317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340a32f0 con 0x565416b00c10 2024-01-11T17:59:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.618+0000 7f635cc3db80 1 -- 172.21.15.7:0/3792874317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 msgr2=0x565416b01030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.618+0000 7f635cc3db80 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b01030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6344110840 tx=0x7f6334009ca0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.620+0000 7f635cc3db80 1 -- 172.21.15.7:0/3792874317 shutdown_connections 2024-01-11T17:59:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.620+0000 7f635cc3db80 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b00200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.620+0000 7f635cc3db80 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b01030 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.620+0000 7f635cc3db80 1 --2- 172.21.15.7:0/3792874317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b01590 0x565416aff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.621+0000 7f635cc3db80 1 -- 172.21.15.7:0/3792874317 >> 172.21.15.7:0/3792874317 conn(0x565416a6c7a0 msgr2=0x565416af1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.621+0000 7f635cc3db80 1 -- 172.21.15.7:0/3792874317 shutdown_connections 2024-01-11T17:59:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.622+0000 7f635cc3db80 1 -- 172.21.15.7:0/3792874317 wait complete. 2024-01-11T17:59:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.623+0000 7f635cc3db80 1 -- start start 2024-01-11T17:59:14.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.624+0000 7f635cc3db80 1 Processor -- start 2024-01-11T17:59:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.626+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b09330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.626+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b09870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.627+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f635cc3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416a4ad30 con 0x565416b01590 2024-01-11T17:59:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f634259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b09330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f635cc3db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654169b93a0 con 0x565416b00c10 2024-01-11T17:59:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f635cc3db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654169b9040 con 0x565416affd80 2024-01-11T17:59:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f635cc3db80 10 client.0 _ll_register_callbacks cb 0x5654169eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f634259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b09330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43196/0 (socket says 172.21.15.7:43196) 2024-01-11T17:59:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f634259c700 1 -- 172.21.15.7:0/2164602885 learned_addr learned my addr 172.21.15.7:0/2164602885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.628+0000 7f6342d9d700 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.629+0000 7f634259c700 1 -- 172.21.15.7:0/2164602885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 msgr2=0x565416b09870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.629+0000 7f634259c700 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b09870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.630+0000 7f634259c700 1 -- 172.21.15.7:0/2164602885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 msgr2=0x565416b108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.630+0000 7f634259c700 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.630+0000 7f634259c700 1 -- 172.21.15.7:0/2164602885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63441aa540 con 0x565416affd80 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.630+0000 7f634259c700 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b09330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6334097a60 tx=0x7f633400f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83879a88694895c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.630+0000 7f6331ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.631+0000 7f6331ffb700 1 -- 172.21.15.7:0/2164602885 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340b2030 con 0x565416affd80 2024-01-11T17:59:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.631+0000 7f6331ffb700 1 -- 172.21.15.7:0/2164602885 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f63340b3070 con 0x565416affd80 2024-01-11T17:59:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.631+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565416aea010 con 0x565416affd80 2024-01-11T17:59:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.632+0000 7f6331ffb700 1 -- 172.21.15.7:0/2164602885 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63340a32f0 con 0x565416affd80 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.632+0000 7f635cc3db80 10 client.25774 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.632+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56541686bb60 con 0x565416affd80 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.632+0000 7f6331ffb700 1 -- 172.21.15.7:0/2164602885 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565416aea010 con 0x565416affd80 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.632+0000 7f6331ffb700 1 -- 172.21.15.7:0/2164602885 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f63340282b0 con 0x565416affd80 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.633+0000 7f635cc3db80 10 client.25774 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.633+0000 7f635cc3db80 10 client.25774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-01-11T17:59:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.633+0000 7f635cc3db80 20 client.25774 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.634+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b14d20 0x565416b17110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.634+0000 7f635cc3db80 4 client.25774 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.634+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565416a43bc0 con 0x565416b14d20 2024-01-11T17:59:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.634+0000 7f6339d9b700 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b14d20 0x565416b17110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.636+0000 7f6339d9b700 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b14d20 0x565416b17110 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f78c69f868dbc99 server_cookie=89510c0b2a997493 in_seq=0 out_seq=0 2024-01-11T17:59:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.636+0000 7f6331ffb700 10 client.25774 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:59:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.637+0000 7f6331ffb700 1 -- 172.21.15.7:0/2164602885 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565416a43bc0 con 0x565416b14d20 2024-01-11T17:59:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.637+0000 7f6331ffb700 10 client.25774 handle_command_reply: tid=0 2024-01-11T17:59:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.637+0000 7f635cc3db80 1 client.25774 shutdown 2024-01-11T17:59:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.639+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 msgr2=0x565416b09330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.639+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b09330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6334097a60 tx=0x7f633400f460 comp rx=0 tx=0).stop 2024-01-11T17:59:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.640+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 shutdown_connections 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.640+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b14d20 0x565416b17110 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.640+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b09330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.640+0000 7f6331ffb700 0 client.25774 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.641+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.641+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2164602885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b09870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.641+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 >> 172.21.15.7:0/2164602885 conn(0x565416a6c7a0 msgr2=0x565416afe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.641+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 shutdown_connections 2024-01-11T17:59:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.641+0000 7f635cc3db80 1 -- 172.21.15.7:0/2164602885 wait complete. 2024-01-11T17:59:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.641+0000 7f635cc3db80 20 client.25774 trim_cache size 0 max 16384 2024-01-11T17:59:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.688+0000 7f635cc3db80 1 Processor -- start 2024-01-11T17:59:14.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.688+0000 7f635cc3db80 1 -- start start 2024-01-11T17:59:14.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.689+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 0x565416b07660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.689+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b10870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.689+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416b01590 0x565416aff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.689+0000 7f635cc3db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654168c16a0 con 0x565416b00c10 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.689+0000 7f635cc3db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654169ee090 con 0x565416affd80 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.689+0000 7f635cc3db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416a43bc0 con 0x565416b01590 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.690+0000 7f634259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 0x565416b07660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.690+0000 7f634259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 0x565416b07660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44696/0 (socket says 172.21.15.7:44696) 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.690+0000 7f634259c700 1 -- 172.21.15.7:0/2170340656 learned_addr learned my addr 172.21.15.7:0/2170340656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.690+0000 7f6342d9d700 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416b01590 0x565416aff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.691+0000 7f6339d9b700 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b10870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.691+0000 7f634259c700 1 -- 172.21.15.7:0/2170340656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416b01590 msgr2=0x565416aff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.691+0000 7f634259c700 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416b01590 0x565416aff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.691+0000 7f634259c700 1 -- 172.21.15.7:0/2170340656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 msgr2=0x565416b10870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.691+0000 7f634259c700 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b10870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.691+0000 7f634259c700 1 -- 172.21.15.7:0/2170340656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565416a0c230 con 0x565416affd80 2024-01-11T17:59:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.692+0000 7f6342d9d700 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416b01590 0x565416aff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.692+0000 7f634259c700 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 0x565416b07660 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63340036f0 tx=0x7f6334096cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99d3cf09da5bc40 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.693+0000 7f6333fff700 1 -- 172.21.15.7:0/2170340656 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633409f140 con 0x565416affd80 2024-01-11T17:59:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.693+0000 7f6333fff700 1 -- 172.21.15.7:0/2170340656 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63340a32f0 con 0x565416affd80 2024-01-11T17:59:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.693+0000 7f6333fff700 1 -- 172.21.15.7:0/2170340656 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340283b0 con 0x565416affd80 2024-01-11T17:59:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.694+0000 7f635cc3db80 1 -- 172.21.15.7:0/2170340656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 msgr2=0x565416b07660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.695+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 0x565416b07660 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63340036f0 tx=0x7f6334096cc0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.695+0000 7f635cc3db80 1 -- 172.21.15.7:0/2170340656 shutdown_connections 2024-01-11T17:59:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.695+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416b01590 0x565416aff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.695+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416affd80 0x565416b07660 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.695+0000 7f635cc3db80 1 --2- 172.21.15.7:0/2170340656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b00c10 0x565416b10870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.695+0000 7f635cc3db80 1 -- 172.21.15.7:0/2170340656 >> 172.21.15.7:0/2170340656 conn(0x565416a6c7a0 msgr2=0x565416af1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.696+0000 7f635cc3db80 1 -- 172.21.15.7:0/2170340656 shutdown_connections 2024-01-11T17:59:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.696+0000 7f635cc3db80 1 -- 172.21.15.7:0/2170340656 wait complete. 2024-01-11T17:59:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.697+0000 7f635cc3db80 1 -- start start 2024-01-11T17:59:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.697+0000 7f635cc3db80 1 Processor -- start 2024-01-11T17:59:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.699+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b0e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.700+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.700+0000 7f635cc3db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b0f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f634259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b0e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f634259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b0e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43214/0 (socket says 172.21.15.7:43214) 2024-01-11T17:59:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f634259c700 1 -- 172.21.15.7:0/297746740 learned_addr learned my addr 172.21.15.7:0/297746740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416b0fb30 con 0x565416b01590 2024-01-11T17:59:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416b0fca0 con 0x565416b00c10 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x565416b0fe60 con 0x565416affd80 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.703+0000 7f635cc3db80 10 client.0 _ll_register_callbacks cb 0x5654169eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.701+0000 7f6339d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.703+0000 7f634259c700 1 -- 172.21.15.7:0/297746740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 msgr2=0x565416b0e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.703+0000 7f634259c700 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b0e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.703+0000 7f634259c700 1 -- 172.21.15.7:0/297746740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 msgr2=0x565416b0f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.703+0000 7f634259c700 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b0f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.703+0000 7f634259c700 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56541686bb60 con 0x565416affd80 2024-01-11T17:59:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.704+0000 7f634259c700 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b0e0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6334009fd0 tx=0x7f6334009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451ded7a4b2f6e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.704+0000 7f6331ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.705+0000 7f6331ffb700 1 -- 172.21.15.7:0/297746740 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340b2070 con 0x565416affd80 2024-01-11T17:59:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.705+0000 7f6331ffb700 1 -- 172.21.15.7:0/297746740 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6334028390 con 0x565416affd80 2024-01-11T17:59:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.705+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63340cf020 con 0x565416affd80 2024-01-11T17:59:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.706+0000 7f6331ffb700 1 -- 172.21.15.7:0/297746740 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633409f140 con 0x565416affd80 2024-01-11T17:59:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.706+0000 7f635cc3db80 10 client.25780 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.706+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5654168c3cd0 con 0x565416affd80 2024-01-11T17:59:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.706+0000 7f6331ffb700 1 -- 172.21.15.7:0/297746740 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63340cf020 con 0x565416affd80 2024-01-11T17:59:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.706+0000 7f6331ffb700 1 -- 172.21.15.7:0/297746740 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6334080050 con 0x565416affd80 2024-01-11T17:59:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f635cc3db80 10 client.25780 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f635cc3db80 10 client.25780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-01-11T17:59:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f635cc3db80 20 client.25780 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f635cc3db80 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b6b4e0 0x565416b6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f635cc3db80 4 client.25780 mds_command: new command op to 24577 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-01-11T17:59:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 --> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565416a438e0 con 0x565416b6b4e0 2024-01-11T17:59:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.707+0000 7f6339d9b700 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b6b4e0 0x565416b6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.708+0000 7f6339d9b700 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b6b4e0 0x565416b6d8d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b36e67c65d07581f server_cookie=cfec4a9c847f0c81 in_seq=0 out_seq=0 2024-01-11T17:59:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.708+0000 7f6331ffb700 10 client.25780 ms_handle_connect on v2:172.21.15.38:6838/443773776 2024-01-11T17:59:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.711+0000 7f6331ffb700 1 -- 172.21.15.7:0/297746740 <== mds.2 v2:172.21.15.38:6838/443773776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+83907 (crc 0 0 0) 0x565416a438e0 con 0x565416b6b4e0 2024-01-11T17:59:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.711+0000 7f6331ffb700 10 client.25780 handle_command_reply: tid=0 2024-01-11T17:59:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.712+0000 7f635cc3db80 1 client.25780 shutdown 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 msgr2=0x565416b0e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b0e0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6334009fd0 tx=0x7f6334009ca0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 shutdown_connections 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:6838/443773776,v1:172.21.15.38:6839/443773776] conn(0x565416b6b4e0 0x565416b6d8d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x565416affd80 0x565416b0e0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565416b01590 0x565416b0f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 --2- 172.21.15.7:0/297746740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565416b00c10 0x565416b0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f6331ffb700 0 client.25780 ms_handle_reset on v2:172.21.15.38:6838/443773776 2024-01-11T17:59:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.713+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 >> 172.21.15.7:0/297746740 conn(0x565416a6c7a0 msgr2=0x565416af1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.714+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 shutdown_connections 2024-01-11T17:59:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.714+0000 7f635cc3db80 1 -- 172.21.15.7:0/297746740 wait complete. 2024-01-11T17:59:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.714+0000 7f635cc3db80 20 client.25780 trim_cache size 0 max 16384 2024-01-11T17:59:14.718 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3643266, 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628646, 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2620_vm" 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6616366, 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628515, 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_fsm" 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 23425221, 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628193, 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2679" 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 48905692, 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628163, 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187_vm" 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.720 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 51148974, 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628083, 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3501_vm" 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 63338969, 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628015, 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/113" 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.721 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 63469085, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628717, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3501_vm" 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 76230806, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628774, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2693" 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 76594487, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627835, 2024-01-11T17:59:14.722 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603" 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 80079893, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628034, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6111" 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 94073137, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628709, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2702" 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.723 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 107741487, 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627826, 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2617_fsm" 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 113759963, 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628712, 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3712" 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 122592901, 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628411, 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3712" 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 163493667, 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628183, 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3381_vm" 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 166134662, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628014, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/548" 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 169034256, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628658, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_fsm" 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 177781948, 2024-01-11T17:59:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628227, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/828" 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 194694069, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628425, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2663" 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 208606117, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628767, 2024-01-11T17:59:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6102_vm" 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 225336144, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628080, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394_vm" 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 243690877, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628225, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2658" 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 253386336, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628503, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2679" 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 262784309, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628505, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456" 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 268448468, 2024-01-11T17:59:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628736, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12199" 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 303539022, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628490, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12204" 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 316393637, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628486, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_vm" 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 371956139, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627840, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2613" 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 385371400, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628663, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3542" 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 404878219, 2024-01-11T17:59:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628660, 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2703" 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 435279081, 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628439, 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12207" 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 440376252, 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628434, 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2754" 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 440812189, 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628749, 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3997" 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 441493767, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628511, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753" 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 443999891, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628687, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_vm" 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 451406843, 2024-01-11T17:59:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628509, 2024-01-11T17:59:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2683" 2024-01-11T17:59:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 452471510, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628700, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_vm" 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 469473076, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628483, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6113" 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 492468802, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628190, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3598" 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 497866865, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628019, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3119" 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 503094467, 2024-01-11T17:59:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628648, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2678" 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 511440547, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628254, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2689" 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 513874433, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628695, 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2841" 2024-01-11T17:59:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 527421325, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628680, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418" 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 536918127, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628693, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3502" 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 550333391, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628724, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3440" 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 551650563, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628214, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605" 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 556161663, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628713, 2024-01-11T17:59:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608" 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 569450842, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628506, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12209" 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 585473257, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628659, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_fsm" 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 599569413, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628664, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2701" 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 670327423, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628427, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2836" 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 675767846, 2024-01-11T17:59:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628504, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12212" 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 679125301, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628766, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830" 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 694636603, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628232, 2024-01-11T17:59:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2666" 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 698124668, 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628488, 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2757" 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 717386936, 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628005, 2024-01-11T17:59:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3604" 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 728190896, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628497, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/5002" 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 728376754, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628202, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192" 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 737148641, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628733, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3080" 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 739966232, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627905, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2834" 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 742173821, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628642, 2024-01-11T17:59:14.746 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_fsm" 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 768426018, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628415, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3164" 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 777218947, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628655, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_fsm" 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 794473550, 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628710, 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2655" 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 795987420, 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628729, 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/548" 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.748 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 803788762, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628661, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12189" 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 805958258, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628400, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3575" 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 819203965, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628407, 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6110" 2024-01-11T17:59:14.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 868123421, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628221, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3503" 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 879247571, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627820, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_fsm" 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.750 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 880025598, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627907, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2836" 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 894033472, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628716, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3164" 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 902868030, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628650, 2024-01-11T17:59:14.751 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2187" 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 905770953, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628755, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118" 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 914221559, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628172, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192_vm" 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 917181823, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628152, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12216" 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 925200231, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628147, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12207" 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 936289951, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628401, 2024-01-11T17:59:14.753 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3467" 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 938704475, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628750, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2832_vm" 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 964032453, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628510, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12196" 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 965306624, 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628423, 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3440" 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 966171076, 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628239, 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2688" 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.755 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 968890158, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628757, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2659" 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 975118179, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627817, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1249" 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1010674203, 2024-01-11T17:59:14.756 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628053, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418_vm" 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1012577206, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628499, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3596_vm" 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1021820536, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628765, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2328" 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.757 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1025295469, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628419, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/113" 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1030378544, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628489, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609_fsm" 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1061947899, 2024-01-11T17:59:14.758 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628142, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12199" 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1065596077, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628722, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2691" 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1079205860, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628430, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603" 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1098245588, 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628735, 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2754" 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1103055243, 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628669, 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12184" 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.760 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1105503051, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628675, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2692" 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1117141767, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628433, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3394_vm" 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1145029591, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628212, 2024-01-11T17:59:14.761 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202" 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1147297856, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628756, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_vm" 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1161227362, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628769, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_vm" 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.762 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1165732288, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628763, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255_fsm" 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1169077229, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628771, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826_vm" 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1194975045, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628746, 2024-01-11T17:59:14.763 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_vm" 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1211964771, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628715, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_vm" 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1211982081, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628714, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2831" 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1228364621, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628484, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601" 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1238749202, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628654, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417_vm" 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1243990664, 2024-01-11T17:59:14.765 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628684, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12206" 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1245307131, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628681, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12216" 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1251495837, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628408, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2702" 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1260766627, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628215, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764_fsm" 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1265376361, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628196, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12209" 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1265835929, 2024-01-11T17:59:14.767 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628041, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6106_vm" 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1273768444, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628201, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753" 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1282915226, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628518, 2024-01-11T17:59:14.768 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_fsm" 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1299322739, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628402, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_fsm" 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1308912122, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628440, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610" 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1313912280, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628217, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3767" 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1314965179, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628242, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609" 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1315056761, 2024-01-11T17:59:14.770 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628406, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_fsm" 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1315114831, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628247, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2834_vm" 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1318102715, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628679, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_fsm" 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1320807295, 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628730, 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247" 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1321867367, 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628743, 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613" 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.772 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1332208912, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628210, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2615" 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1348833670, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627913, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2840" 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1357148041, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628517, 2024-01-11T17:59:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576_vm" 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1359767959, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628634, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2681" 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1379041814, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628023, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3597" 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.774 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1401348317, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628493, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381_vm" 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1416419892, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627870, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603" 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1430047358, 2024-01-11T17:59:14.775 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628762, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256_vm" 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1432287299, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628725, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2685" 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1449934765, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628410, 2024-01-11T17:59:14.776 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3597" 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1459201622, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628207, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3576_vm" 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1461871515, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628027, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3542" 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.777 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1470309833, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628428, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/548" 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1483914278, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628185, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3118_vm" 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1505757901, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628670, 2024-01-11T17:59:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3350_vm" 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1507941452, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628682, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1249" 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1516758190, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628414, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187_vm" 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1548321680, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628198, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610_vm" 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1549865120, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628513, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_fsm" 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1556523848, 2024-01-11T17:59:14.780 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628003, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3712" 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1571876485, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628438, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259" 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1620345850, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628752, 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2653" 2024-01-11T17:59:14.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1620783891, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628778, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_fsm" 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1623914658, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628404, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2674" 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.782 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1625588359, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628133, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12184" 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1630957746, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628491, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2835" 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1640972268, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627998, 2024-01-11T17:59:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2701" 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1643161079, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628224, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2699" 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1644201209, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628175, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753_vm" 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.784 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1645950112, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628187, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/5002" 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1649698364, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628773, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2337" 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1649997235, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628251, 2024-01-11T17:59:14.785 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/174" 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1650913905, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628727, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838" 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1670518901, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628645, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2680" 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.786 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1681298557, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628784, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387" 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1688363840, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627886, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394" 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1709520332, 2024-01-11T17:59:14.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628732, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764" 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1742041305, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628170, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2662" 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1745899669, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628580, 2024-01-11T17:59:14.788 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3766" 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1752562762, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628035, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6112" 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1758592023, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628178, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2757" 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.789 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1766701464, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628417, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6111" 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1774093008, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628197, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3379" 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1778900681, 2024-01-11T17:59:14.790 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628666, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603" 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1817512796, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628635, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_vm" 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1827786267, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628061, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3602_vm" 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1840558772, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628422, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605_vm" 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1847569520, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628770, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079" 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1852356824, 2024-01-11T17:59:14.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628246, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2602" 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1858461751, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628785, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16390" 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1871608372, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628092, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_vm" 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1875985222, 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628416, 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3501_vm" 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1879466832, 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628176, 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259_vm" 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.794 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1909714880, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627819, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259" 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1911684701, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628249, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2675" 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1925137813, 2024-01-11T17:59:14.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628656, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106" 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1933790748, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628737, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3455" 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1935781910, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628738, 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_fsm" 2024-01-11T17:59:14.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1947942291, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628432, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3080" 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1957826169, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628171, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3257" 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1962630437, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628077, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2995_vm" 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1978699513, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628498, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2660" 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1996791271, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628739, 2024-01-11T17:59:14.798 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259" 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2008752665, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628226, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612_vm" 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2012982859, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628500, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3598" 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2018867550, 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628218, 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_vm" 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2022650170, 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627832, 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601_fsm" 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.800 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2023140864, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628516, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_fsm" 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2024786969, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628495, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118_vm" 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2024894157, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628704, 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3119" 2024-01-11T17:59:14.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2027951135, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628209, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2656" 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2031922899, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628696, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_fsm" 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2034337116, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628787, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387_fsm" 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2034774760, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627970, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2682" 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2045369515, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628697, 2024-01-11T17:59:14.803 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_vm" 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2049225970, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628219, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2669" 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2071734784, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628253, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2224" 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2085398637, 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628694, 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394" 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2153366964, 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628772, 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2686" 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.805 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2164493889, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627916, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3440" 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2170393857, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628399, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3603_vm" 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2182896016, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628220, 2024-01-11T17:59:14.806 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2995" 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2188848228, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628237, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3541" 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2196740223, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627926, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2651" 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2201615890, 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627958, 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2674" 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2209787314, 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628673, 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3466" 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2219648698, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628657, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3085" 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2221168928, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628726, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2663" 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2226188796, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628667, 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_vm" 2024-01-11T17:59:14.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2232947558, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628699, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2673" 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2254199173, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628668, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3601_vm" 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.810 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2262043659, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628759, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3534" 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2262849222, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628186, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187" 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2277854901, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628208, 2024-01-11T17:59:14.811 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_fsm" 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2291467877, 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628644, 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2684" 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.812 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2308559615, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628248, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3395" 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2326098286, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628181, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2835" 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2349542776, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627847, 2024-01-11T17:59:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3466" 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2363434583, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628437, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12182_fsm" 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2363740024, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628671, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840" 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2364941292, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628177, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607" 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2366543704, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628025, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3080" 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2384033025, 2024-01-11T17:59:14.815 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627996, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3575" 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2388297303, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628403, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3119" 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2397895923, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628638, 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613_vm" 2024-01-11T17:59:14.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2401225021, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628651, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381" 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2455170063, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628677, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995_vm" 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.817 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2463035736, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628081, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2611_vm" 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2481528301, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628228, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202_vm" 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2492447014, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628194, 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12212" 2024-01-11T17:59:14.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2494798673, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628481, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3257" 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2503687347, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627987, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2692" 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.819 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2507973901, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628496, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187" 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2514986690, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628412, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608" 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2531562116, 2024-01-11T17:59:14.820 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628146, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12206" 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2542185188, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628703, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_fsm" 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2547879719, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628833, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3766" 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2558264724, 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628238, 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3600" 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2560242791, 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628686, 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576" 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.822 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2571935418, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628098, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_vm" 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2587332909, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628640, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618" 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2589855535, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628701, 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3575" 2024-01-11T17:59:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2603933160, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628222, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12201" 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2622496735, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627927, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2652" 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.824 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2630710603, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627977, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2685" 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2654684102, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628707, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_fsm" 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2656705970, 2024-01-11T17:59:14.825 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628512, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192" 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2661332487, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628637, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12191" 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2685614622, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628199, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2683" 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2700807570, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627883, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3256" 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2703645616, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628708, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6110" 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2711156148, 2024-01-11T17:59:14.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628243, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1255" 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2721927821, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628753, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2755" 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2727735432, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627834, 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_fsm" 2024-01-11T17:59:14.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2728153296, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628429, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247" 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2739380529, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627982, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2754" 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2752520494, 2024-01-11T17:59:14.829 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628245, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2650" 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2754781259, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628636, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_vm" 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2758280198, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628216, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_vm" 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2765093291, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628206, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_fsm" 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2788812987, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628502, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2665" 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2797010344, 2024-01-11T17:59:14.831 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628741, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610" 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2798084428, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628641, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_fsm" 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2799503804, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628514, 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_vm" 2024-01-11T17:59:14.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2803920126, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628016, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/549" 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2812760492, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627999, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2702" 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.833 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2814411488, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627836, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603_fsm" 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2828253269, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628421, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2691" 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2833202741, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627931, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2655" 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2838160220, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627957, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2673" 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2848405361, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628742, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3606" 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2852304724, 2024-01-11T17:59:14.835 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628250, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12186" 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2866392511, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628000, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3467" 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2867813089, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628718, 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6111" 2024-01-11T17:59:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2884588478, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628229, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3605" 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2884951297, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628519, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2656" 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.837 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2900456686, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628046, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3541_vm" 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2901868457, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628683, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/PG_VERSION" 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2908368162, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628436, 2024-01-11T17:59:14.838 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3455" 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2909523238, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628768, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6104_vm" 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2922936693, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628174, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601" 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.839 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2954975424, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628649, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_fsm" 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2959498862, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628405, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2668" 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2981972181, 2024-01-11T17:59:14.840 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628734, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394_vm" 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3002444755, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628413, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2831" 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3030521368, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628485, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_vm" 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3065635104, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511629091, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_internal.init" 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3069623974, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628652, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2654" 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3074539296, 2024-01-11T17:59:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627946, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3164" 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3084818936, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628240, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2604" 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3090056056, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628492, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2837" 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3094787180, 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628192, 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2665" 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3105048892, 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628200, 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12196" 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3121239627, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628233, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_vm" 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3140492725, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627824, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610" 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3144202012, 2024-01-11T17:59:14.845 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628184, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1417" 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3150071307, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628189, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3596_vm" 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3161488743, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628231, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3609" 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3165076704, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628775, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830_vm" 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3171663739, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627944, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2663" 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3176361022, 2024-01-11T17:59:14.847 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628191, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_fsm" 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3187537605, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628179, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609_fsm" 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3204873079, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628431, 2024-01-11T17:59:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764" 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3206471999, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628751, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606_vm" 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3222949632, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628689, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106_vm" 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3223009150, 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628236, 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601_vm" 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3227562465, 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628507, 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3379" 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.850 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3239582100, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628748, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840_fsm" 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3261909279, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628720, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/113" 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3280528016, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628647, 2024-01-11T17:59:14.851 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2696" 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3293791023, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628723, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_vm" 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3295108097, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628435, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12199" 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3297414514, 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627909, 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838" 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3298243610, 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628487, 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607" 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3305209574, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628783, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16384" 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3316849728, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628758, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_fsm" 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3323838526, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628653, 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_fsm" 2024-01-11T17:59:14.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3325126560, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628230, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_vm" 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3341171733, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628662, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_fsm" 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.855 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3359263420, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628760, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439" 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3383123929, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627850, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_fsm" 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3397441125, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628685, 2024-01-11T17:59:14.856 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418_vm" 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3401083054, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628420, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3604" 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3405993011, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628173, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6113" 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3425970270, 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628482, 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_vm" 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3440528607, 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628223, 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607_fsm" 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3447887878, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628235, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_vm" 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3456875240, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627914, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2841" 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3457523978, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628728, 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2836" 2024-01-11T17:59:14.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3468389698, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628205, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2619_fsm" 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3495048706, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628060, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603_vm" 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.860 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3503343611, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627876, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764" 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3508910513, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628241, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3439_vm" 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3516383265, 2024-01-11T17:59:14.861 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628639, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3607" 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3546969727, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628745, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2600_fsm" 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3557021465, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628043, 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3350_vm" 2024-01-11T17:59:14.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3560763107, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628182, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2837" 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3562763990, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628786, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16393" 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.863 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3585212493, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628731, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603" 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3589020454, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628441, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3606" 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3620324691, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627855, 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608" 2024-01-11T17:59:14.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3623193608, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628777, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2704" 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3645829725, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628706, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2668" 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3651919775, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628494, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417" 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3652026829, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628195, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456" 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3667312790, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628702, 2024-01-11T17:59:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3467" 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3690456415, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628678, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/549" 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3706018960, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628409, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2655" 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.867 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3739088131, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628776, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3574" 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3770626818, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627973, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3380" 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3780927971, 2024-01-11T17:59:14.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628211, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_fsm" 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3791404421, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628676, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6112" 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3793019167, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628692, 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_vm" 2024-01-11T17:59:14.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3795083720, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627881, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418" 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3795331581, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628705, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2674" 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.870 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3802600617, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628203, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_fsm" 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3810761205, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627952, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2668" 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3829925249, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628058, 2024-01-11T17:59:14.871 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3601_vm" 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3831323693, 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628688, 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834" 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3835202360, 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628442, 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2613" 2024-01-11T17:59:14.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3838543516, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628721, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3604" 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3861512174, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628719, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2682" 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.873 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3869389656, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627815, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247" 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3876956954, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628665, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_fsm" 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3880932869, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628057, 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764_vm" 2024-01-11T17:59:14.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3894381177, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628204, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_vm" 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3894384184, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628674, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3380" 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.875 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3901403539, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628690, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256" 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3901580503, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628244, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612" 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3904303507, 2024-01-11T17:59:14.876 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628747, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_filenode.map" 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3904475441, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627901, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2831" 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3925748932, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628418, 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2682" 2024-01-11T17:59:14.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3957443822, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628033, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6110" 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3975443541, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628761, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3081" 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3976270643, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627961, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3502" 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3983961767, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628740, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207" 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3985588093, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628754, 2024-01-11T17:59:14.879 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/175" 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3998894244, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628764, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606" 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4006727467, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628252, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3602" 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.880 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4023642885, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628234, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/826" 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4026296629, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628691, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2652" 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4055829176, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628188, 2024-01-11T17:59:14.881 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2660" 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4059070680, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628213, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_vm" 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4059202085, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628426, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838" 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4062741227, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628744, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079_vm" 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4071904133, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628508, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610_vm" 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4075824678, 2024-01-11T17:59:14.883 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628115, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/PG_VERSION" 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4086295425, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628424, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2685" 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4097773482, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628672, 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2611_vm" 2024-01-11T17:59:14.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4145056279, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627945, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3455" 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4157057928, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627986, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2691" 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.885 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4173522724, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628698, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2651" 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4198690260, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628480, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2662" 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4208012639, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628153, 2024-01-11T17:59:14.886 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12182_fsm" 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4212301730, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628180, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12204" 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4216884729, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628643, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2690" 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.887 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4235147401, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628633, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224_vm" 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4240216921, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628501, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_fsm" 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4243086697, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628711, 2024-01-11T17:59:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3597" 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4271947166, 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628007, 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3606" 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4286865045, 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627894, 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576" 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:59:14.889 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63240a2010 msgr2=0x7f63240a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63240a2010 0x7f63240a44d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f632c0155a0 tx=0x7f632c009190 comp rx=0 tx=0).stop 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 msgr2=0x7f634419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634419db60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63340038c0 tx=0x7f6334003c00 comp rx=0 tx=0).stop 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 shutdown_connections 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f63240a2010 0x7f63240a44d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63441087f0 0x7f634419db60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634410dcb0 0x7f63441a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 --2- 172.21.15.7:0/3973789819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634410b1b0 0x7f634419e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 >> 172.21.15.7:0/3973789819 conn(0x7f634406e530 msgr2=0x7f6344109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 shutdown_connections 2024-01-11T17:59:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:14.717+0000 7f6348fff700 1 -- 172.21.15.7:0/3973789819 wait complete. 2024-01-11T17:59:14.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-01-11T17:59:14.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:14 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2344596185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:59:15.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:14 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2344596185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:59:15.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:14 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2344596185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T17:59:15.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.294+0000 7fa4a27ff700 1 -- 172.21.15.7:0/2590010163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c101120 msgr2=0x7fa49c1015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/2590010163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c101120 0x7fa49c1015a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa49400b0a0 tx=0x7fa49400b3b0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 -- 172.21.15.7:0/2590010163 shutdown_connections 2024-01-11T17:59:15.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/2590010163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c101120 0x7fa49c1015a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/2590010163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa49c0ffe00 0x7fa49c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/2590010163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c100760 0x7fa49c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 -- 172.21.15.7:0/2590010163 >> 172.21.15.7:0/2590010163 conn(0x7fa49c0fba00 msgr2=0x7fa49c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 -- 172.21.15.7:0/2590010163 shutdown_connections 2024-01-11T17:59:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.295+0000 7fa4a27ff700 1 -- 172.21.15.7:0/2590010163 wait complete. 2024-01-11T17:59:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.297+0000 7fa4a27ff700 1 Processor -- start 2024-01-11T17:59:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.297+0000 7fa4a27ff700 1 -- start start 2024-01-11T17:59:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.297+0000 7fa4a27ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 0x7fa49c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.298+0000 7fa4a27ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c100760 0x7fa49c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.298+0000 7fa4a27ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa49c101120 0x7fa49c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.298+0000 7fa4a27ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a1320 con 0x7fa49c101120 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.298+0000 7fa4a27ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a1460 con 0x7fa49c100760 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.298+0000 7fa4a27ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a15f0 con 0x7fa49c0ffe00 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.299+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 0x7fa49c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.299+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 0x7fa49c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43232/0 (socket says 172.21.15.7:43232) 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.299+0000 7fa49bfff700 1 -- 172.21.15.7:0/313473143 learned_addr learned my addr 172.21.15.7:0/313473143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.299+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa49c101120 0x7fa49c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.300+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c100760 0x7fa49c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.300+0000 7fa49bfff700 1 -- 172.21.15.7:0/313473143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c100760 msgr2=0x7fa49c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.300+0000 7fa49bfff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c100760 0x7fa49c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.300+0000 7fa49bfff700 1 -- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa49c101120 msgr2=0x7fa49c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.300+0000 7fa49bfff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa49c101120 0x7fa49c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.300+0000 7fa49bfff700 1 -- 172.21.15.7:0/313473143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009d00 con 0x7fa49c0ffe00 2024-01-11T17:59:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.301+0000 7fa49bfff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 0x7fa49c1997a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00c920 tx=0x7fa48c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.301+0000 7fa4997fa700 1 -- 172.21.15.7:0/313473143 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c007b00 con 0x7fa49c0ffe00 2024-01-11T17:59:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.301+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c1a18d0 con 0x7fa49c0ffe00 2024-01-11T17:59:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.301+0000 7fa4997fa700 1 -- 172.21.15.7:0/313473143 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa48c00ce70 con 0x7fa49c0ffe00 2024-01-11T17:59:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.301+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1a1dd0 con 0x7fa49c0ffe00 2024-01-11T17:59:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.302+0000 7fa4997fa700 1 -- 172.21.15.7:0/313473143 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c005460 con 0x7fa49c0ffe00 2024-01-11T17:59:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.310+0000 7fa4997fa700 1 -- 172.21.15.7:0/313473143 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa48c00f450 con 0x7fa49c0ffe00 2024-01-11T17:59:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.314+0000 7fa4997fa700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4840a1f60 0x7fa4840a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.314+0000 7fa4997fa700 1 -- 172.21.15.7:0/313473143 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa48c0ce760 con 0x7fa49c0ffe00 2024-01-11T17:59:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.317+0000 7fa4b65e7b80 1 Processor -- start 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 -- start start 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bf400 0x5653da9bf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 0x5653da9be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da8b5090 con 0x5653da9bf400 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da918b20 con 0x5653da9be570 2024-01-11T17:59:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.318+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da916440 con 0x5653da9bfd80 2024-01-11T17:59:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.319+0000 7fa4a0d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 0x5653da9be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.319+0000 7fa4a0d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 0x5653da9be9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44724/0 (socket says 172.21.15.7:44724) 2024-01-11T17:59:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.319+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3171085583 learned_addr learned my addr 172.21.15.7:0/3171085583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.320+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4840a1f60 0x7fa4840a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.320+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.320+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3171085583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 msgr2=0x5653da9bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.320+0000 7fa49bfff700 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bf400 0x5653da9bf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.321+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9bdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.321+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3171085583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bf400 msgr2=0x5653da9bf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.321+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bf400 0x5653da9bf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.321+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3171085583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653da7863b0 con 0x5653da9be570 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.321+0000 7fa49bfff700 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bf400 0x5653da9bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.322+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 0x5653da9be9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa494015000 tx=0x7fa494009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1b771c17e57a41 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.322+0000 7fa4827fc700 1 -- 172.21.15.7:0/3171085583 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400e040 con 0x5653da9be570 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.322+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4840a1f60 0x7fa4840a4420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa490012690 tx=0x7fa490009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.322+0000 7fa4827fc700 1 -- 172.21.15.7:0/3171085583 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa494007760 con 0x5653da9be570 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.322+0000 7fa4827fc700 1 -- 172.21.15.7:0/3171085583 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49401c7a0 con 0x5653da9be570 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.324+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3171085583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 msgr2=0x5653da9be9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.324+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 0x5653da9be9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa494015000 tx=0x7fa494009390 comp rx=0 tx=0).stop 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.325+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3171085583 shutdown_connections 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.325+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9be570 0x5653da9be9f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.325+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bf400 0x5653da9bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.325+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3171085583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.325+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3171085583 >> 172.21.15.7:0/3171085583 conn(0x5653da9337a0 msgr2=0x5653da9b8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.326+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3171085583 shutdown_connections 2024-01-11T17:59:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.326+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3171085583 wait complete. 2024-01-11T17:59:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.327+0000 7fa4b65e7b80 1 -- start start 2024-01-11T17:59:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.330+0000 7fa4b65e7b80 1 Processor -- start 2024-01-11T17:59:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.331+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9e1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.331+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.331+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9e8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da911d30 con 0x5653da9be570 2024-01-11T17:59:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa4a0d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa4a0d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9e8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43258/0 (socket says 172.21.15.7:43258) 2024-01-11T17:59:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/1119696036 learned_addr learned my addr 172.21.15.7:0/1119696036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa49bfff700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da8803a0 con 0x5653da9bf400 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.332+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da880040 con 0x5653da9bfd80 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa4b65e7b80 10 client.0 _ll_register_callbacks cb 0x5653da8b2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/1119696036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 msgr2=0x5653da9e1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9e1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 msgr2=0x5653da9e1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9e1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/1119696036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c10c590 con 0x5653da9bfd80 2024-01-11T17:59:15.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.333+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:15.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.334+0000 7fa47b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:15.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.334+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9e8990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4940094b0 tx=0x7fa4940052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fca726f2916401f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.334+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1119696036 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400e040 con 0x5653da9bfd80 2024-01-11T17:59:15.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.334+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1119696036 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa494003a00 con 0x5653da9bfd80 2024-01-11T17:59:15.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.334+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653da9b1010 con 0x5653da9bfd80 2024-01-11T17:59:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.335+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1119696036 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa494003d00 con 0x5653da9bfd80 2024-01-11T17:59:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.335+0000 7fa4b65e7b80 10 client.25792 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.335+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5653da732b60 con 0x5653da9bfd80 2024-01-11T17:59:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.335+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1119696036 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653da9b1010 con 0x5653da9bfd80 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1119696036 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa494019030 con 0x5653da9bfd80 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa4b65e7b80 10 client.25792 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa4b65e7b80 10 client.25792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa4b65e7b80 20 client.25792 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9ecd80 0x5653da9ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa4b65e7b80 4 client.25792 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.336+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653da90abc0 con 0x5653da9ecd80 2024-01-11T17:59:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.337+0000 7fa49bfff700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9ecd80 0x5653da9ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.338+0000 7fa49bfff700 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9ecd80 0x5653da9ef1a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6eacbcda62c6e6e server_cookie=416b6b6061007373 in_seq=0 out_seq=0 2024-01-11T17:59:15.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.338+0000 7fa47b7fe700 10 client.25792 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:59:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.339+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1119696036 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653da90abc0 con 0x5653da9ecd80 2024-01-11T17:59:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.339+0000 7fa47b7fe700 10 client.25792 handle_command_reply: tid=0 2024-01-11T17:59:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.339+0000 7fa4b65e7b80 1 client.25792 shutdown 2024-01-11T17:59:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 msgr2=0x5653da9e8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9e8990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4940094b0 tx=0x7fa4940052f0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 shutdown_connections 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9ecd80 0x5653da9ef1a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bfd80 0x5653da9e8990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa47b7fe700 0 client.25792 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1119696036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.340+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 >> 172.21.15.7:0/1119696036 conn(0x5653da9337a0 msgr2=0x5653da9bce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.341+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 shutdown_connections 2024-01-11T17:59:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.341+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1119696036 wait complete. 2024-01-11T17:59:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.341+0000 7fa4b65e7b80 20 client.25792 trim_cache size 0 max 16384 2024-01-11T17:59:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.367+0000 7fa4b65e7b80 1 Processor -- start 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 -- start start 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9be970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 0x5653da9e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 0x5653da9ba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da7886a0 con 0x5653da9be570 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da9b0b50 con 0x5653da9bfd80 2024-01-11T17:59:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.369+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da9e9af0 con 0x5653da9bf400 2024-01-11T17:59:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.371+0000 7fa4a0d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 0x5653da9ba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.371+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 0x5653da9e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.372+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 0x5653da9e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43272/0 (socket says 172.21.15.7:43272) 2024-01-11T17:59:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.371+0000 7fa4a0d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 0x5653da9ba520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44736/0 (socket says 172.21.15.7:44736) 2024-01-11T17:59:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.372+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3764633913 learned_addr learned my addr 172.21.15.7:0/3764633913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa49bfff700 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3764633913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 msgr2=0x5653da9e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 0x5653da9e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3764633913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 msgr2=0x5653da9be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9be970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa4a0d9c700 1 -- 172.21.15.7:0/3764633913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653da8d3230 con 0x5653da9bfd80 2024-01-11T17:59:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.373+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 0x5653da9e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.374+0000 7fa49bfff700 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.374+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 0x5653da9ba520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa494009cd0 tx=0x7fa494012470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc9e6bb617103bc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.378+0000 7fa481ffb700 1 -- 172.21.15.7:0/3764633913 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400e040 con 0x5653da9bfd80 2024-01-11T17:59:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.379+0000 7fa481ffb700 1 -- 172.21.15.7:0/3764633913 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa494006160 con 0x5653da9bfd80 2024-01-11T17:59:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.379+0000 7fa481ffb700 1 -- 172.21.15.7:0/3764633913 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494005750 con 0x5653da9bfd80 2024-01-11T17:59:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.380+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3764633913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 msgr2=0x5653da9ba520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.380+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 0x5653da9ba520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa494009cd0 tx=0x7fa494012470 comp rx=0 tx=0).stop 2024-01-11T17:59:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3764633913 shutdown_connections 2024-01-11T17:59:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bfd80 0x5653da9ba520 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9be570 0x5653da9be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/3764633913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9bf400 0x5653da9e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3764633913 >> 172.21.15.7:0/3764633913 conn(0x5653da9337a0 msgr2=0x5653da9bce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3764633913 shutdown_connections 2024-01-11T17:59:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.381+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/3764633913 wait complete. 2024-01-11T17:59:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.382+0000 7fa4b65e7b80 1 -- start start 2024-01-11T17:59:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.382+0000 7fa4b65e7b80 1 Processor -- start 2024-01-11T17:59:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.385+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 0x5653da9c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.386+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.386+0000 7fa4b65e7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bfd80 0x5653da9e7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.386+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da9e76b0 con 0x5653da9bfd80 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.386+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da9e7820 con 0x5653da9bf400 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.386+0000 7fa4b65e7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653da9e79e0 con 0x5653da9be570 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.386+0000 7fa4b65e7b80 10 client.0 _ll_register_callbacks cb 0x5653da8b2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.387+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 0x5653da9c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.387+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 0x5653da9c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43278/0 (socket says 172.21.15.7:43278) 2024-01-11T17:59:15.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.387+0000 7fa49bfff700 1 -- 172.21.15.7:0/1974514390 learned_addr learned my addr 172.21.15.7:0/1974514390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.388+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.388+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bfd80 0x5653da9e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.389+0000 7fa49bfff700 1 -- 172.21.15.7:0/1974514390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 msgr2=0x5653da9c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.389+0000 7fa49bfff700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.389+0000 7fa49bfff700 1 -- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bfd80 msgr2=0x5653da9e7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.389+0000 7fa49bfff700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bfd80 0x5653da9e7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.390+0000 7fa49bfff700 1 -- 172.21.15.7:0/1974514390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa490009e30 con 0x5653da9be570 2024-01-11T17:59:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.390+0000 7fa4a0d9c700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bfd80 0x5653da9e7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.391+0000 7fa49bfff700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 0x5653da9c4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa48c097690 tx=0x7fa48c09f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d17d8103c74fbe88 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.391+0000 7fa47b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.391+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1974514390 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c09f2e0 con 0x5653da9be570 2024-01-11T17:59:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.392+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1974514390 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa48c0a3520 con 0x5653da9be570 2024-01-11T17:59:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.392+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa494007760 con 0x5653da9be570 2024-01-11T17:59:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.396+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1974514390 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c09f2e0 con 0x5653da9be570 2024-01-11T17:59:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.397+0000 7fa4b65e7b80 10 client.25798 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.397+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5653da732b60 con 0x5653da9be570 2024-01-11T17:59:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1974514390 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa494007760 con 0x5653da9be570 2024-01-11T17:59:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1974514390 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa48c09b3f0 con 0x5653da9be570 2024-01-11T17:59:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa4b65e7b80 10 client.25798 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa4b65e7b80 10 client.25798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-01-11T17:59:15.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa4b65e7b80 20 client.25798 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:15.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9dbfd0 0x5653da9de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa4b65e7b80 4 client.25798 mds_command: new command op to 24601 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-01-11T17:59:15.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.398+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 --> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5653da90a8e0 con 0x5653da9dbfd0 2024-01-11T17:59:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.400+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9dbfd0 0x5653da9de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.402+0000 7fa49b7fe700 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9dbfd0 0x5653da9de3f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30d1d1aafaf1d1ce server_cookie=54b04e83b3259ba0 in_seq=0 out_seq=0 2024-01-11T17:59:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.402+0000 7fa47b7fe700 10 client.25798 ms_handle_connect on v2:172.21.15.7:6840/898637958 2024-01-11T17:59:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.405+0000 7fa47b7fe700 1 -- 172.21.15.7:0/1974514390 <== mds.1 v2:172.21.15.7:6840/898637958 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18464 (crc 0 0 0) 0x5653da90a8e0 con 0x5653da9dbfd0 2024-01-11T17:59:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.405+0000 7fa47b7fe700 10 client.25798 handle_command_reply: tid=0 2024-01-11T17:59:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.406+0000 7fa4b65e7b80 1 client.25798 shutdown 2024-01-11T17:59:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.407+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 msgr2=0x5653da9c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.407+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 0x5653da9c4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa48c097690 tx=0x7fa48c09f210 comp rx=0 tx=0).stop 2024-01-11T17:59:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 shutdown_connections 2024-01-11T17:59:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:6840/898637958,v1:172.21.15.7:6841/898637958] conn(0x5653da9dbfd0 0x5653da9de3f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653da9be570 0x5653da9c4720 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa48c097690 tx=0x7fa48c09f210 comp rx=0 tx=0).stop 2024-01-11T17:59:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653da9bfd80 0x5653da9e7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 --2- 172.21.15.7:0/1974514390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653da9bf400 0x5653da9c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa47b7fe700 0 client.25798 ms_handle_reset on v2:172.21.15.7:6840/898637958 2024-01-11T17:59:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 >> 172.21.15.7:0/1974514390 conn(0x5653da9337a0 msgr2=0x5653da9c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.408+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 shutdown_connections 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.409+0000 7fa4b65e7b80 1 -- 172.21.15.7:0/1974514390 wait complete. 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.409+0000 7fa4b65e7b80 20 client.25798 trim_cache size 0 max 16384 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 13790604, 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628523, 2024-01-11T17:59:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_vm" 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 50184632, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628521, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_fsm" 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 134275798, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628558, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3395" 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 167486833, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628126, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2609_fsm" 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 255320999, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628551, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439_vm" 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 289285072, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628529, 2024-01-11T17:59:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2669" 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 309086444, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628788, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16396" 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 378764530, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628532, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12201" 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 382487609, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628138, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192" 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 431520127, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628531, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3503" 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 531646387, 2024-01-11T17:59:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627831, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601" 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 566540848, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628009, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3766" 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 581650556, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628076, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3381_vm" 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 587602298, 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628542, 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2666" 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 639098914, 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627941, 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2660" 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 642531777, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628534, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2699" 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 646008570, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628537, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/828" 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 684328946, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627943, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2662" 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 687912895, 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628563, 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224" 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 689771969, 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627975, 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3379" 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 703001723, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627965, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2679" 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 875859433, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628522, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202" 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 976285531, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511629088, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/log/postgresql-Thu.log" 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 977131440, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628082, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610_vm" 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1094230645, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627932, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2757" 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1302451726, 2024-01-11T17:59:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628553, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255" 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1345347512, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628555, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2650" 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1385673230, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628530, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995" 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1388181362, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628539, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3605" 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1425251485, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628036, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6113" 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1490579165, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627989, 2024-01-11T17:59:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/5002" 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1589182223, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627880, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1417" 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1599249175, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628536, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612_vm" 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1604476727, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628154, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2619_fsm" 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1607042801, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628561, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/174" 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1786667704, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628557, 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834_vm" 2024-01-11T17:59:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1860098694, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628120, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_fsm" 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1966666233, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628550, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2604" 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1993518878, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627889, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3456" 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2072975109, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628119, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608_fsm" 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2109526528, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627816, 2024-01-11T17:59:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247_fsm" 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2132076108, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628072, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753_vm" 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2182274743, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627854, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2607" 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2213166050, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628540, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_vm" 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2228223670, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628140, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12196" 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2386011949, 2024-01-11T17:59:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628520, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2615" 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2400355414, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627827, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753" 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2416396558, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627830, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2616_fsm" 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2601818157, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628108, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_vm" 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2640576164, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628528, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_vm" 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2699289062, 2024-01-11T17:59:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627971, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2683" 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2728168173, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628560, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12186" 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2734939086, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628545, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_vm" 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2758908716, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627780, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_0/.meta" 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2759375477, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627779, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d" 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2819278711, 2024-01-11T17:59:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628538, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202_vm" 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2862224175, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628543, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_vm" 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2881511371, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628541, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3609" 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2888287150, 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627911, 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3598" 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2920705272, 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628535, 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2658" 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2929132444, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628064, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576_vm" 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2932108642, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628546, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_vm" 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2953941199, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628554, 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612" 2024-01-11T17:59:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2954013507, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628548, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3600" 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2973161045, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628547, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541" 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3031313559, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627949, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2665" 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3040576706, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628160, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192_vm" 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3053890291, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628562, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602" 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3166002378, 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628148, 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12209" 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3226040233, 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628564, 2024-01-11T17:59:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2689" 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3273250530, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628527, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3767" 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3287256761, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628091, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_vm" 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3350161054, 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628524, 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605" 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3510757208, 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628533, 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607_fsm" 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3542858592, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628526, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_vm" 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3593799248, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628549, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2688" 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3601079468, 2024-01-11T17:59:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628135, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187" 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3624953545, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628049, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3596_vm" 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3634818380, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628556, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602" 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3653723863, 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627908, 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2837" 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3735461535, 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628052, 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3118_vm" 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3820707759, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627933, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2656" 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3870955621, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628525, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_fsm" 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3909676804, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628028, 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3257" 2024-01-11T17:59:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3912427587, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628150, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12212" 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3942882063, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627906, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2835" 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4046950217, 2024-01-11T17:59:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628552, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609" 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4087564760, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628145, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12204" 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4134929423, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628559, 2024-01-11T17:59:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2675" 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4198055850, 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511628544, 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826" 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T17:59:15.449 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.411+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4840a1f60 msgr2=0x7fa4840a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.411+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4840a1f60 0x7fa4840a4420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa490012690 tx=0x7fa490009610 comp rx=0 tx=0).stop 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.411+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 msgr2=0x7fa49c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.411+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 0x7fa49c1997a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00c920 tx=0x7fa48c00cc30 comp rx=0 tx=0).stop 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.412+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 shutdown_connections 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.412+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa4840a1f60 0x7fa4840a4420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.412+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa49c0ffe00 0x7fa49c1997a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.412+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa49c101120 0x7fa49c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.412+0000 7fa4a27ff700 1 --2- 172.21.15.7:0/313473143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49c100760 0x7fa49c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.412+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 >> 172.21.15.7:0/313473143 conn(0x7fa49c0fba00 msgr2=0x7fa49c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.413+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 shutdown_connections 2024-01-11T17:59:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.413+0000 7fa4a27ff700 1 -- 172.21.15.7:0/313473143 wait complete. 2024-01-11T17:59:15.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-01-11T17:59:15.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 -- 172.21.15.7:0/4122092246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038074010 msgr2=0x7f4038074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 --2- 172.21.15.7:0/4122092246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038074010 0x7f4038074490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4028009990 tx=0x7f4028009ca0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 -- 172.21.15.7:0/4122092246 shutdown_connections 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 --2- 172.21.15.7:0/4122092246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4038109490 0x7f403810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 --2- 172.21.15.7:0/4122092246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40380755f0 0x7f4038073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 --2- 172.21.15.7:0/4122092246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038074010 0x7f4038074490 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.847+0000 7f403f9ff700 1 -- 172.21.15.7:0/4122092246 >> 172.21.15.7:0/4122092246 conn(0x7f40380fcb30 msgr2=0x7f40380fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.848+0000 7f403f9ff700 1 -- 172.21.15.7:0/4122092246 shutdown_connections 2024-01-11T17:59:15.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.848+0000 7f403f9ff700 1 -- 172.21.15.7:0/4122092246 wait complete. 2024-01-11T17:59:15.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.849+0000 7f403f9ff700 1 Processor -- start 2024-01-11T17:59:15.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.850+0000 7f403f9ff700 1 -- start start 2024-01-11T17:59:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.850+0000 7f403f9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4038074010 0x7f4038195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.850+0000 7f403f9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380755f0 0x7f4038195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403f9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038109490 0x7f403819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403d79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4038074010 0x7f4038195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403d79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4038074010 0x7f4038195250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43746/0 (socket says 172.21.15.7:43746) 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403d79b700 1 -- 172.21.15.7:0/1217178723 learned_addr learned my addr 172.21.15.7:0/1217178723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380755f0 0x7f4038195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819cdd0 con 0x7f4038074010 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40381ae030 con 0x7f4038109490 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40381ae170 con 0x7f40380755f0 2024-01-11T17:59:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.851+0000 7f403df9c700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038109490 0x7f403819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f403cf9a700 1 -- 172.21.15.7:0/1217178723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038109490 msgr2=0x7f403819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038109490 0x7f403819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f403cf9a700 1 -- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4038074010 msgr2=0x7f4038195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4038074010 0x7f4038195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f403cf9a700 1 -- 172.21.15.7:0/1217178723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034009520 con 0x7f40380755f0 2024-01-11T17:59:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380755f0 0x7f4038195790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f402800b790 tx=0x7f4028004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.852+0000 7f402e7fc700 1 -- 172.21.15.7:0/1217178723 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402801d070 con 0x7f40380755f0 2024-01-11T17:59:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.853+0000 7f402e7fc700 1 -- 172.21.15.7:0/1217178723 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4028022470 con 0x7f40380755f0 2024-01-11T17:59:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.853+0000 7f402e7fc700 1 -- 172.21.15.7:0/1217178723 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028017650 con 0x7f40380755f0 2024-01-11T17:59:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.853+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40280095f0 con 0x7f40380755f0 2024-01-11T17:59:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.853+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40381ae550 con 0x7f40380755f0 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.861+0000 7f402e7fc700 1 -- 172.21.15.7:0/1217178723 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f40280177b0 con 0x7f40380755f0 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.862+0000 7f405365ab80 1 Processor -- start 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 -- start start 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a6d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 0x56022a6d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a5cd090 con 0x56022a6d7d80 2024-01-11T17:59:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a630b20 con 0x56022a6d7400 2024-01-11T17:59:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.863+0000 7f405365ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a62e440 con 0x56022a6d6570 2024-01-11T17:59:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.864+0000 7f403df9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.864+0000 7f403cf9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 0x56022a6d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.864+0000 7f403cf9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 0x56022a6d69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43304/0 (socket says 172.21.15.7:43304) 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.864+0000 7f403cf9a700 1 -- 172.21.15.7:0/2022731185 learned_addr learned my addr 172.21.15.7:0/2022731185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.864+0000 7f403d79b700 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a6d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.865+0000 7f403cf9a700 1 -- 172.21.15.7:0/2022731185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 msgr2=0x56022a6d7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.865+0000 7f403cf9a700 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6d7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.865+0000 7f403cf9a700 1 -- 172.21.15.7:0/2022731185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 msgr2=0x56022a6d5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.865+0000 7f403cf9a700 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a6d5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.865+0000 7f403cf9a700 1 -- 172.21.15.7:0/2022731185 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56022a49e3b0 con 0x56022a6d6570 2024-01-11T17:59:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.865+0000 7f403df9c700 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.866+0000 7f402e7fc700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40240a1f60 0x7f40240a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.867+0000 7f402e7fc700 1 -- 172.21.15.7:0/1217178723 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40280ced60 con 0x7f40380755f0 2024-01-11T17:59:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.867+0000 7f403df9c700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40240a1f60 0x7f40240a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.868+0000 7f403cf9a700 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 0x56022a6d69f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40280d2d30 tx=0x7f40280d7700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad6dc6974e87fd6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.868+0000 7f403df9c700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40240a1f60 0x7f40240a4420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56022a6d5f20 tx=0x7f403000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.869+0000 7f40237fe700 1 -- 172.21.15.7:0/2022731185 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280d2e80 con 0x56022a6d6570 2024-01-11T17:59:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.869+0000 7f40237fe700 1 -- 172.21.15.7:0/2022731185 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4028097b60 con 0x56022a6d6570 2024-01-11T17:59:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.869+0000 7f40237fe700 1 -- 172.21.15.7:0/2022731185 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280d2e80 con 0x56022a6d6570 2024-01-11T17:59:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 -- 172.21.15.7:0/2022731185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 msgr2=0x56022a6d69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 0x56022a6d69f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40280d2d30 tx=0x7f40280d7700 comp rx=0 tx=0).stop 2024-01-11T17:59:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 -- 172.21.15.7:0/2022731185 shutdown_connections 2024-01-11T17:59:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d6570 0x56022a6d69f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 --2- 172.21.15.7:0/2022731185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a6d5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.871+0000 7f405365ab80 1 -- 172.21.15.7:0/2022731185 >> 172.21.15.7:0/2022731185 conn(0x56022a64b7a0 msgr2=0x56022a6d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.872+0000 7f405365ab80 1 -- 172.21.15.7:0/2022731185 shutdown_connections 2024-01-11T17:59:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.872+0000 7f405365ab80 1 -- 172.21.15.7:0/2022731185 wait complete. 2024-01-11T17:59:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.872+0000 7f405365ab80 1 -- start start 2024-01-11T17:59:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.873+0000 7f405365ab80 1 Processor -- start 2024-01-11T17:59:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 0x56022a6dfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a6e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a629d30 con 0x56022a6d7400 2024-01-11T17:59:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a5983a0 con 0x56022a6e5320 2024-01-11T17:59:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a598040 con 0x56022a6d7d80 2024-01-11T17:59:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f405365ab80 10 client.0 _ll_register_callbacks cb 0x56022a5ca720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.874+0000 7f403d79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 0x56022a6dfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.875+0000 7f403cf9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.875+0000 7f403d79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 0x56022a6dfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43758/0 (socket says 172.21.15.7:43758) 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.875+0000 7f403d79b700 1 -- 172.21.15.7:0/4199229178 learned_addr learned my addr 172.21.15.7:0/4199229178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.875+0000 7f403df9c700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a6e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.876+0000 7f403d79b700 1 -- 172.21.15.7:0/4199229178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 msgr2=0x56022a6e02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.876+0000 7f403d79b700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6e02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.876+0000 7f403d79b700 1 -- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 msgr2=0x56022a6e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.876+0000 7f403d79b700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a6e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.876+0000 7f403d79b700 1 -- 172.21.15.7:0/4199229178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4030011230 con 0x56022a6d7400 2024-01-11T17:59:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.876+0000 7f403df9c700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a6e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.877+0000 7f403d79b700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 0x56022a6dfdb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4034011890 tx=0x7f4034011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f525acf72483953 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.877+0000 7f4020ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:59:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.877+0000 7f4020ff9700 1 -- 172.21.15.7:0/4199229178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034016070 con 0x56022a6d7400 2024-01-11T17:59:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.878+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56022a6c9010 con 0x56022a6d7400 2024-01-11T17:59:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.878+0000 7f4020ff9700 1 -- 172.21.15.7:0/4199229178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f403400f770 con 0x56022a6d7400 2024-01-11T17:59:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.879+0000 7f4020ff9700 1 -- 172.21.15.7:0/4199229178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f403400f930 con 0x56022a6d7400 2024-01-11T17:59:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.879+0000 7f405365ab80 10 client.15726 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.879+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f40381ae550 con 0x56022a6d7400 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f4020ff9700 1 -- 172.21.15.7:0/4199229178 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56022a6c9010 con 0x56022a6d7400 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f4020ff9700 1 -- 172.21.15.7:0/4199229178 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f403401ac80 con 0x56022a6d7400 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f405365ab80 10 client.15726 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f405365ab80 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f405365ab80 20 client.15726 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f405365ab80 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a6fd160 0x56022a6ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f405365ab80 4 client.15726 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.880+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56022a622bc0 con 0x56022a6fd160 2024-01-11T17:59:15.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.882+0000 7f403cf9a700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a6fd160 0x56022a6ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.884+0000 7f403cf9a700 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a6fd160 0x56022a6ff580 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0b762b02caf888 server_cookie=644ad9c64b14f45a in_seq=0 out_seq=0 2024-01-11T17:59:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.884+0000 7f4020ff9700 10 client.15726 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.885+0000 7f4020ff9700 1 -- 172.21.15.7:0/4199229178 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56022a622bc0 con 0x56022a6fd160 2024-01-11T17:59:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.885+0000 7f4020ff9700 10 client.15726 handle_command_reply: tid=0 2024-01-11T17:59:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.885+0000 7f405365ab80 1 client.15726 shutdown 2024-01-11T17:59:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.885+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 msgr2=0x56022a6dfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.885+0000 7f405365ab80 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 0x56022a6dfdb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4034011890 tx=0x7f4034011ba0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 shutdown_connections 2024-01-11T17:59:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a6fd160 0x56022a6ff580 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a6e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f4020ff9700 0 client.15726 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7400 0x56022a6dfdb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 --2- 172.21.15.7:0/4199229178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 >> 172.21.15.7:0/4199229178 conn(0x56022a64b7a0 msgr2=0x56022a6d4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 shutdown_connections 2024-01-11T17:59:15.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.886+0000 7f405365ab80 1 -- 172.21.15.7:0/4199229178 wait complete. 2024-01-11T17:59:15.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.887+0000 7f405365ab80 20 client.15726 trim_cache size 0 max 16384 2024-01-11T17:59:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.922+0000 7f405365ab80 1 Processor -- start 2024-01-11T17:59:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 -- start start 2024-01-11T17:59:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6e5320 0x56022a6d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a4a06a0 con 0x56022a6e5320 2024-01-11T17:59:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a6fa440 con 0x56022a6d7400 2024-01-11T17:59:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.923+0000 7f405365ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a6e0c60 con 0x56022a6d7d80 2024-01-11T17:59:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.924+0000 7f403d79b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.924+0000 7f403d79b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44762/0 (socket says 172.21.15.7:44762) 2024-01-11T17:59:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.924+0000 7f403d79b700 1 -- 172.21.15.7:0/982304803 learned_addr learned my addr 172.21.15.7:0/982304803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.924+0000 7f403df9c700 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6e5320 0x56022a6d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.924+0000 7f403d79b700 1 -- 172.21.15.7:0/982304803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 msgr2=0x56022a6d5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.924+0000 7f403d79b700 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.925+0000 7f403d79b700 1 -- 172.21.15.7:0/982304803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6e5320 msgr2=0x56022a6d6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.925+0000 7f403d79b700 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6e5320 0x56022a6d6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.925+0000 7f403d79b700 1 -- 172.21.15.7:0/982304803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56022a44ab60 con 0x56022a6d7400 2024-01-11T17:59:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.925+0000 7f403d79b700 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6ff550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4034009b40 tx=0x7f40340056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5930a6a5ad588012 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.926+0000 7f4022ffd700 1 -- 172.21.15.7:0/982304803 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034016030 con 0x56022a6d7400 2024-01-11T17:59:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.926+0000 7f4022ffd700 1 -- 172.21.15.7:0/982304803 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40340052d0 con 0x56022a6d7400 2024-01-11T17:59:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.926+0000 7f4022ffd700 1 -- 172.21.15.7:0/982304803 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40340039e0 con 0x56022a6d7400 2024-01-11T17:59:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.928+0000 7f405365ab80 1 -- 172.21.15.7:0/982304803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 msgr2=0x56022a6ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.928+0000 7f405365ab80 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6ff550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4034009b40 tx=0x7f40340056c0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 -- 172.21.15.7:0/982304803 shutdown_connections 2024-01-11T17:59:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6e5320 0x56022a6d6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6d7400 0x56022a6ff550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 --2- 172.21.15.7:0/982304803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7d80 0x56022a6d5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 -- 172.21.15.7:0/982304803 >> 172.21.15.7:0/982304803 conn(0x56022a64b7a0 msgr2=0x56022a6de850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 -- 172.21.15.7:0/982304803 shutdown_connections 2024-01-11T17:59:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.930+0000 7f405365ab80 1 -- 172.21.15.7:0/982304803 wait complete. 2024-01-11T17:59:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.934+0000 7f405365ab80 1 -- start start 2024-01-11T17:59:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.934+0000 7f405365ab80 1 Processor -- start 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 0x56022a70c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a70cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a752100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a6db2c0 con 0x56022a6d7d80 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a6d21d0 con 0x56022a6e5320 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022a70d440 con 0x56022a6d7400 2024-01-11T17:59:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.935+0000 7f405365ab80 10 client.0 _ll_register_callbacks cb 0x56022a5ca720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.936+0000 7f403d79b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 0x56022a70c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.936+0000 7f403d79b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 0x56022a70c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43342/0 (socket says 172.21.15.7:43342) 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.936+0000 7f403d79b700 1 -- 172.21.15.7:0/1783407459 learned_addr learned my addr 172.21.15.7:0/1783407459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.936+0000 7f403cf9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a70cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.936+0000 7f403df9c700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a752100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.937+0000 7f403d79b700 1 -- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 msgr2=0x56022a752100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.937+0000 7f403d79b700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a752100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.937+0000 7f403d79b700 1 -- 172.21.15.7:0/1783407459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 msgr2=0x56022a70cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.937+0000 7f403d79b700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a70cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.937+0000 7f403d79b700 1 -- 172.21.15.7:0/1783407459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40381ae550 con 0x56022a6d7400 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.937+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a70cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.938+0000 7f403df9c700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a752100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.939+0000 7f4020ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.939+0000 7f403d79b700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 0x56022a70c800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4034019410 tx=0x7f403400fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3764ae6d885d3045 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.940+0000 7f4020ff9700 1 -- 172.21.15.7:0/1783407459 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034016040 con 0x56022a6d7400 2024-01-11T17:59:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.940+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4028097b60 con 0x56022a6d7400 2024-01-11T17:59:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.940+0000 7f4020ff9700 1 -- 172.21.15.7:0/1783407459 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4034017070 con 0x56022a6d7400 2024-01-11T17:59:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.943+0000 7f4020ff9700 1 -- 172.21.15.7:0/1783407459 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4034005150 con 0x56022a6d7400 2024-01-11T17:59:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.943+0000 7f405365ab80 10 client.25813 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.943+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56022a5eb230 con 0x56022a6d7400 2024-01-11T17:59:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f4020ff9700 1 -- 172.21.15.7:0/1783407459 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4028097b60 con 0x56022a6d7400 2024-01-11T17:59:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f4020ff9700 1 -- 172.21.15.7:0/1783407459 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f40340195b0 con 0x56022a6d7400 2024-01-11T17:59:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f405365ab80 10 client.25813 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f405365ab80 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f405365ab80 20 client.25813 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f405365ab80 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a753bb0 0x56022a755fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f405365ab80 4 client.25813 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-01-11T17:59:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.944+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56022a6228e0 con 0x56022a753bb0 2024-01-11T17:59:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.945+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a753bb0 0x56022a755fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.947+0000 7f403cf9a700 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a753bb0 0x56022a755fd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c616fc3485b7205b server_cookie=37e0debbb8fb52f6 in_seq=0 out_seq=0 2024-01-11T17:59:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.947+0000 7f4020ff9700 10 client.25813 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.948+0000 7f4020ff9700 1 -- 172.21.15.7:0/1783407459 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56022a6228e0 con 0x56022a753bb0 2024-01-11T17:59:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.948+0000 7f4020ff9700 10 client.25813 handle_command_reply: tid=0 2024-01-11T17:59:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.948+0000 7f405365ab80 1 client.25813 shutdown 2024-01-11T17:59:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.950+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 msgr2=0x56022a70c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.950+0000 7f405365ab80 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 0x56022a70c800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4034019410 tx=0x7f403400fd40 comp rx=0 tx=0).stop 2024-01-11T17:59:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 shutdown_connections 2024-01-11T17:59:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56022a753bb0 0x56022a755fd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56022a6d7400 0x56022a70c800 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56022a6e5320 0x56022a752100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 --2- 172.21.15.7:0/1783407459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56022a6d7d80 0x56022a70cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 >> 172.21.15.7:0/1783407459 conn(0x56022a64b7a0 msgr2=0x56022a6df330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f4020ff9700 0 client.25813 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 shutdown_connections 2024-01-11T17:59:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 1 -- 172.21.15.7:0/1783407459 wait complete. 2024-01-11T17:59:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.951+0000 7f405365ab80 20 client.25813 trim_cache size 0 max 16384 2024-01-11T17:59:15.954 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40240a1f60 msgr2=0x7f40240a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40240a1f60 0x7f40240a4420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56022a6d5f20 tx=0x7f403000b040 comp rx=0 tx=0).stop 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380755f0 msgr2=0x7f4038195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380755f0 0x7f4038195790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f402800b790 tx=0x7f4028004710 comp rx=0 tx=0).stop 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 shutdown_connections 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40240a1f60 0x7f40240a4420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.953+0000 7f403f9ff700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4038109490 0x7f403819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.954+0000 7f403f9ff700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4038074010 0x7f4038195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.954+0000 7f403f9ff700 1 --2- 172.21.15.7:0/1217178723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40380755f0 0x7f4038195790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.954+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 >> 172.21.15.7:0/1217178723 conn(0x7f40380fcb30 msgr2=0x7f4038100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.957+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 shutdown_connections 2024-01-11T17:59:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T17:59:15.957+0000 7f403f9ff700 1 -- 172.21.15.7:0/1217178723 wait complete. 2024-01-11T17:59:15.989 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:15 smithi007 ceph-mon[33361]: pgmap v343: 129 pgs: 129 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 799 op/s 2024-01-11T17:59:15.993 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 3643266, 'ino': 1099511628646, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2620_vm'}, {'damage_type': 'backtrace', 'id': 6616366, 'ino': 1099511628515, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_fsm'}, {'damage_type': 'backtrace', 'id': 23425221, 'ino': 1099511628193, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2679'}, {'damage_type': 'backtrace', 'id': 48905692, 'ino': 1099511628163, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187_vm'}, {'damage_type': 'backtrace', 'id': 51148974, 'ino': 1099511628083, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3501_vm'}, {'damage_type': 'backtrace', 'id': 63338969, 'ino': 1099511628015, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/113'}, {'damage_type': 'backtrace', 'id': 63469085, 'ino': 1099511628717, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3501_vm'}, {'damage_type': 'backtrace', 'id': 76230806, 'ino': 1099511628774, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2693'}, {'damage_type': 'backtrace', 'id': 76594487, 'ino': 1099511627835, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603'}, {'damage_type': 'backtrace', 'id': 80079893, 'ino': 1099511628034, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6111'}, {'damage_type': 'backtrace', 'id': 94073137, 'ino': 1099511628709, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2702'}, {'damage_type': 'backtrace', 'id': 107741487, 'ino': 1099511627826, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2617_fsm'}, {'damage_type': 'backtrace', 'id': 113759963, 'ino': 1099511628712, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3712'}, {'damage_type': 'backtrace', 'id': 122592901, 'ino': 1099511628411, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3712'}, {'damage_type': 'backtrace', 'id': 163493667, 'ino': 1099511628183, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3381_vm'}, {'damage_type': 'backtrace', 'id': 166134662, 'ino': 1099511628014, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/548'}, {'damage_type': 'backtrace', 'id': 169034256, 'ino': 1099511628658, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_fsm'}, {'damage_type': 'backtrace', 'id': 177781948, 'ino': 1099511628227, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/828'}, {'damage_type': 'backtrace', 'id': 194694069, 'ino': 1099511628425, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2663'}, {'damage_type': 'backtrace', 'id': 208606117, 'ino': 1099511628767, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6102_vm'}, {'damage_type': 'backtrace', 'id': 225336144, 'ino': 1099511628080, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394_vm'}, {'damage_type': 'backtrace', 'id': 243690877, 'ino': 1099511628225, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2658'}, {'damage_type': 'backtrace', 'id': 253386336, 'ino': 1099511628503, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2679'}, {'damage_type': 'backtrace', 'id': 262784309, 'ino': 1099511628505, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456'}, {'damage_type': 'backtrace', 'id': 268448468, 'ino': 1099511628736, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12199'}, {'damage_type': 'backtrace', 'id': 303539022, 'ino': 1099511628490, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12204'}, {'damage_type': 'backtrace', 'id': 316393637, 'ino': 1099511628486, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_vm'}, {'damage_type': 'backtrace', 'id': 371956139, 'ino': 1099511627840, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2613'}, {'damage_type': 'backtrace', 'id': 385371400, 'ino': 1099511628663, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3542'}, {'damage_type': 'backtrace', 'id': 404878219, 'ino': 1099511628660, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2703'}, {'damage_type': 'backtrace', 'id': 435279081, 'ino': 1099511628439, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12207'}, {'damage_type': 'backtrace', 'id': 440376252, 'ino': 1099511628434, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2754'}, {'damage_type': 'backtrace', 'id': 440812189, 'ino': 1099511628749, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3997'}, {'damage_type': 'backtrace', 'id': 441493767, 'ino': 1099511628511, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753'}, {'damage_type': 'backtrace', 'id': 443999891, 'ino': 1099511628687, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_vm'}, {'damage_type': 'backtrace', 'id': 451406843, 'ino': 1099511628509, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2683'}, {'damage_type': 'backtrace', 'id': 452471510, 'ino': 1099511628700, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_vm'}, {'damage_type': 'backtrace', 'id': 469473076, 'ino': 1099511628483, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6113'}, {'damage_type': 'backtrace', 'id': 492468802, 'ino': 1099511628190, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3598'}, {'damage_type': 'backtrace', 'id': 497866865, 'ino': 1099511628019, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3119'}, {'damage_type': 'backtrace', 'id': 503094467, 'ino': 1099511628648, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2678'}, {'damage_type': 'backtrace', 'id': 511440547, 'ino': 1099511628254, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2689'}, {'damage_type': 'backtrace', 'id': 513874433, 'ino': 1099511628695, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2841'}, {'damage_type': 'backtrace', 'id': 527421325, 'ino': 1099511628680, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418'}, {'damage_type': 'backtrace', 'id': 536918127, 'ino': 1099511628693, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3502'}, {'damage_type': 'backtrace', 'id': 550333391, 'ino': 1099511628724, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3440'}, {'damage_type': 'backtrace', 'id': 551650563, 'ino': 1099511628214, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605'}, {'damage_type': 'backtrace', 'id': 556161663, 'ino': 1099511628713, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608'}, {'damage_type': 'backtrace', 'id': 569450842, 'ino': 1099511628506, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12209'}, {'damage_type': 'backtrace', 'id': 585473257, 'ino': 1099511628659, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_fsm'}, {'damage_type': 'backtrace', 'id': 599569413, 'ino': 1099511628664, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2701'}, {'damage_type': 'backtrace', 'id': 670327423, 'ino': 1099511628427, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2836'}, {'damage_type': 'backtrace', 'id': 675767846, 'ino': 1099511628504, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12212'}, {'damage_type': 'backtrace', 'id': 679125301, 'ino': 1099511628766, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830'}, {'damage_type': 'backtrace', 'id': 694636603, 'ino': 1099511628232, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2666'}, {'damage_type': 'backtrace', 'id': 698124668, 'ino': 1099511628488, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2757'}, {'damage_type': 'backtrace', 'id': 717386936, 'ino': 1099511628005, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3604'}, {'damage_type': 'backtrace', 'id': 728190896, 'ino': 1099511628497, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/5002'}, {'damage_type': 'backtrace', 'id': 728376754, 'ino': 1099511628202, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192'}, {'damage_type': 'backtrace', 'id': 737148641, 'ino': 1099511628733, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3080'}, {'damage_type': 'backtrace', 'id': 739966232, 'ino': 1099511627905, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2834'}, {'damage_type': 'backtrace', 'id': 742173821, 'ino': 1099511628642, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603_fsm'}, {'damage_type': 'backtrace', 'id': 768426018, 'ino': 1099511628415, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3164'}, {'damage_type': 'backtrace', 'id': 777218947, 'ino': 1099511628655, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_fsm'}, {'damage_type': 'backtrace', 'id': 794473550, 'ino': 1099511628710, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2655'}, {'damage_type': 'backtrace', 'id': 795987420, 'ino': 1099511628729, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/548'}, {'damage_type': 'backtrace', 'id': 803788762, 'ino': 1099511628661, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12189'}, {'damage_type': 'backtrace', 'id': 805958258, 'ino': 1099511628400, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3575'}, {'damage_type': 'backtrace', 'id': 819203965, 'ino': 1099511628407, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6110'}, {'damage_type': 'backtrace', 'id': 868123421, 'ino': 1099511628221, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3503'}, {'damage_type': 'backtrace', 'id': 879247571, 'ino': 1099511627820, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_fsm'}, {'damage_type': 'backtrace', 'id': 880025598, 'ino': 1099511627907, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2836'}, {'damage_type': 'backtrace', 'id': 894033472, 'ino': 1099511628716, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3164'}, {'damage_type': 'backtrace', 'id': 902868030, 'ino': 1099511628650, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2187'}, {'damage_type': 'backtrace', 'id': 905770953, 'ino': 1099511628755, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118'}, {'damage_type': 'backtrace', 'id': 914221559, 'ino': 1099511628172, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12192_vm'}, {'damage_type': 'backtrace', 'id': 917181823, 'ino': 1099511628152, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12216'}, {'damage_type': 'backtrace', 'id': 925200231, 'ino': 1099511628147, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12207'}, {'damage_type': 'backtrace', 'id': 936289951, 'ino': 1099511628401, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3467'}, {'damage_type': 'backtrace', 'id': 938704475, 'ino': 1099511628750, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2832_vm'}, {'damage_type': 'backtrace', 'id': 964032453, 'ino': 1099511628510, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12196'}, {'damage_type': 'backtrace', 'id': 965306624, 'ino': 1099511628423, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3440'}, {'damage_type': 'backtrace', 'id': 966171076, 'ino': 1099511628239, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2688'}, {'damage_type': 'backtrace', 'id': 968890158, 'ino': 1099511628757, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2659'}, {'damage_type': 'backtrace', 'id': 975118179, 'ino': 1099511627817, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1249'}, {'damage_type': 'backtrace', 'id': 1010674203, 'ino': 1099511628053, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418_vm'}, {'damage_type': 'backtrace', 'id': 1012577206, 'ino': 1099511628499, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3596_vm'}, {'damage_type': 'backtrace', 'id': 1021820536, 'ino': 1099511628765, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2328'}, {'damage_type': 'backtrace', 'id': 1025295469, 'ino': 1099511628419, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/113'}, {'damage_type': 'backtrace', 'id': 1030378544, 'ino': 1099511628489, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609_fsm'}, {'damage_type': 'backtrace', 'id': 1061947899, 'ino': 1099511628142, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12199'}, {'damage_type': 'backtrace', 'id': 1065596077, 'ino': 1099511628722, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2691'}, {'damage_type': 'backtrace', 'id': 1079205860, 'ino': 1099511628430, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603'}, {'damage_type': 'backtrace', 'id': 1098245588, 'ino': 1099511628735, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2754'}, {'damage_type': 'backtrace', 'id': 1103055243, 'ino': 1099511628669, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12184'}, {'damage_type': 'backtrace', 'id': 1105503051, 'ino': 1099511628675, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2692'}, {'damage_type': 'backtrace', 'id': 1117141767, 'ino': 1099511628433, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3394_vm'}, {'damage_type': 'backtrace', 'id': 1145029591, 'ino': 1099511628212, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202'}, {'damage_type': 'backtrace', 'id': 1147297856, 'ino': 1099511628756, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_vm'}, {'damage_type': 'backtrace', 'id': 1161227362, 'ino': 1099511628769, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2619_vm'}, {'damage_type': 'backtrace', 'id': 1165732288, 'ino': 1099511628763, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255_fsm'}, {'damage_type': 'backtrace', 'id': 1169077229, 'ino': 1099511628771, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826_vm'}, {'damage_type': 'backtrace', 'id': 1194975045, 'ino': 1099511628746, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12197_vm'}, {'damage_type': 'backtrace', 'id': 1211964771, 'ino': 1099511628715, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187_vm'}, {'damage_type': 'backtrace', 'id': 1211982081, 'ino': 1099511628714, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2831'}, {'damage_type': 'backtrace', 'id': 1228364621, 'ino': 1099511628484, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601'}, {'damage_type': 'backtrace', 'id': 1238749202, 'ino': 1099511628654, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417_vm'}, {'damage_type': 'backtrace', 'id': 1243990664, 'ino': 1099511628684, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12206'}, {'damage_type': 'backtrace', 'id': 1245307131, 'ino': 1099511628681, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12216'}, {'damage_type': 'backtrace', 'id': 1251495837, 'ino': 1099511628408, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2702'}, {'damage_type': 'backtrace', 'id': 1260766627, 'ino': 1099511628215, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764_fsm'}, {'damage_type': 'backtrace', 'id': 1265376361, 'ino': 1099511628196, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12209'}, {'damage_type': 'backtrace', 'id': 1265835929, 'ino': 1099511628041, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6106_vm'}, {'damage_type': 'backtrace', 'id': 1273768444, 'ino': 1099511628201, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753'}, {'damage_type': 'backtrace', 'id': 1282915226, 'ino': 1099511628518, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_fsm'}, {'damage_type': 'backtrace', 'id': 1299322739, 'ino': 1099511628402, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_fsm'}, {'damage_type': 'backtrace', 'id': 1308912122, 'ino': 1099511628440, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610'}, {'damage_type': 'backtrace', 'id': 1313912280, 'ino': 1099511628217, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3767'}, {'damage_type': 'backtrace', 'id': 1314965179, 'ino': 1099511628242, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609'}, {'damage_type': 'backtrace', 'id': 1315056761, 'ino': 1099511628406, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_fsm'}, {'damage_type': 'backtrace', 'id': 1315114831, 'ino': 1099511628247, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2834_vm'}, {'damage_type': 'backtrace', 'id': 1318102715, 'ino': 1099511628679, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_fsm'}, {'damage_type': 'backtrace', 'id': 1320807295, 'ino': 1099511628730, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247'}, {'damage_type': 'backtrace', 'id': 1321867367, 'ino': 1099511628743, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613'}, {'damage_type': 'backtrace', 'id': 1332208912, 'ino': 1099511628210, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2615'}, {'damage_type': 'backtrace', 'id': 1348833670, 'ino': 1099511627913, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2840'}, {'damage_type': 'backtrace', 'id': 1357148041, 'ino': 1099511628517, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576_vm'}, {'damage_type': 'backtrace', 'id': 1359767959, 'ino': 1099511628634, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2681'}, {'damage_type': 'backtrace', 'id': 1379041814, 'ino': 1099511628023, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3597'}, {'damage_type': 'backtrace', 'id': 1401348317, 'ino': 1099511628493, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381_vm'}, {'damage_type': 'backtrace', 'id': 1416419892, 'ino': 1099511627870, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603'}, {'damage_type': 'backtrace', 'id': 1430047358, 'ino': 1099511628762, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256_vm'}, {'damage_type': 'backtrace', 'id': 1432287299, 'ino': 1099511628725, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2685'}, {'damage_type': 'backtrace', 'id': 1449934765, 'ino': 1099511628410, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3597'}, {'damage_type': 'backtrace', 'id': 1459201622, 'ino': 1099511628207, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3576_vm'}, {'damage_type': 'backtrace', 'id': 1461871515, 'ino': 1099511628027, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3542'}, {'damage_type': 'backtrace', 'id': 1470309833, 'ino': 1099511628428, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/548'}, {'damage_type': 'backtrace', 'id': 1483914278, 'ino': 1099511628185, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3118_vm'}, {'damage_type': 'backtrace', 'id': 1505757901, 'ino': 1099511628670, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3350_vm'}, {'damage_type': 'backtrace', 'id': 1507941452, 'ino': 1099511628682, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1249'}, {'damage_type': 'backtrace', 'id': 1516758190, 'ino': 1099511628414, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187_vm'}, {'damage_type': 'backtrace', 'id': 1548321680, 'ino': 1099511628198, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2610_vm'}, {'damage_type': 'backtrace', 'id': 1549865120, 'ino': 1099511628513, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_fsm'}, {'damage_type': 'backtrace', 'id': 1556523848, 'ino': 1099511628003, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3712'}, {'damage_type': 'backtrace', 'id': 1571876485, 'ino': 1099511628438, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259'}, {'damage_type': 'backtrace', 'id': 1620345850, 'ino': 1099511628752, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2653'}, {'damage_type': 'backtrace', 'id': 1620783891, 'ino': 1099511628778, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_fsm'}, {'damage_type': 'backtrace', 'id': 1623914658, 'ino': 1099511628404, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2674'}, {'damage_type': 'backtrace', 'id': 1625588359, 'ino': 1099511628133, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12184'}, {'damage_type': 'backtrace', 'id': 1630957746, 'ino': 1099511628491, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2835'}, {'damage_type': 'backtrace', 'id': 1640972268, 'ino': 1099511627998, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2701'}, {'damage_type': 'backtrace', 'id': 1643161079, 'ino': 1099511628224, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2699'}, {'damage_type': 'backtrace', 'id': 1644201209, 'ino': 1099511628175, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2753_vm'}, {'damage_type': 'backtrace', 'id': 1645950112, 'ino': 1099511628187, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/5002'}, {'damage_type': 'backtrace', 'id': 1649698364, 'ino': 1099511628773, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2337'}, {'damage_type': 'backtrace', 'id': 1649997235, 'ino': 1099511628251, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/174'}, {'damage_type': 'backtrace', 'id': 1650913905, 'ino': 1099511628727, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838'}, {'damage_type': 'backtrace', 'id': 1670518901, 'ino': 1099511628645, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2680'}, {'damage_type': 'backtrace', 'id': 1681298557, 'ino': 1099511628784, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387'}, {'damage_type': 'backtrace', 'id': 1688363840, 'ino': 1099511627886, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3394'}, {'damage_type': 'backtrace', 'id': 1709520332, 'ino': 1099511628732, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764'}, {'damage_type': 'backtrace', 'id': 1742041305, 'ino': 1099511628170, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2662'}, {'damage_type': 'backtrace', 'id': 1745899669, 'ino': 1099511628580, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3766'}, {'damage_type': 'backtrace', 'id': 1752562762, 'ino': 1099511628035, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6112'}, {'damage_type': 'backtrace', 'id': 1758592023, 'ino': 1099511628178, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2757'}, {'damage_type': 'backtrace', 'id': 1766701464, 'ino': 1099511628417, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6111'}, {'damage_type': 'backtrace', 'id': 1774093008, 'ino': 1099511628197, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3379'}, {'damage_type': 'backtrace', 'id': 1778900681, 'ino': 1099511628666, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3603'}, {'damage_type': 'backtrace', 'id': 1817512796, 'ino': 1099511628635, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207_vm'}, {'damage_type': 'backtrace', 'id': 1827786267, 'ino': 1099511628061, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3602_vm'}, {'damage_type': 'backtrace', 'id': 1840558772, 'ino': 1099511628422, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2605_vm'}, {'damage_type': 'backtrace', 'id': 1847569520, 'ino': 1099511628770, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079'}, {'damage_type': 'backtrace', 'id': 1852356824, 'ino': 1099511628246, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2602'}, {'damage_type': 'backtrace', 'id': 1858461751, 'ino': 1099511628785, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16390'}, {'damage_type': 'backtrace', 'id': 1871608372, 'ino': 1099511628092, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_vm'}, {'damage_type': 'backtrace', 'id': 1875985222, 'ino': 1099511628416, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3501_vm'}, {'damage_type': 'backtrace', 'id': 1879466832, 'ino': 1099511628176, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1259_vm'}, {'damage_type': 'backtrace', 'id': 1909714880, 'ino': 1099511627819, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259'}, {'damage_type': 'backtrace', 'id': 1911684701, 'ino': 1099511628249, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2675'}, {'damage_type': 'backtrace', 'id': 1925137813, 'ino': 1099511628656, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106'}, {'damage_type': 'backtrace', 'id': 1933790748, 'ino': 1099511628737, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3455'}, {'damage_type': 'backtrace', 'id': 1935781910, 'ino': 1099511628738, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_fsm'}, {'damage_type': 'backtrace', 'id': 1947942291, 'ino': 1099511628432, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3080'}, {'damage_type': 'backtrace', 'id': 1957826169, 'ino': 1099511628171, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3257'}, {'damage_type': 'backtrace', 'id': 1962630437, 'ino': 1099511628077, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2995_vm'}, {'damage_type': 'backtrace', 'id': 1978699513, 'ino': 1099511628498, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2660'}, {'damage_type': 'backtrace', 'id': 1996791271, 'ino': 1099511628739, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259'}, {'damage_type': 'backtrace', 'id': 2008752665, 'ino': 1099511628226, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612_vm'}, {'damage_type': 'backtrace', 'id': 2012982859, 'ino': 1099511628500, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3598'}, {'damage_type': 'backtrace', 'id': 2018867550, 'ino': 1099511628218, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_vm'}, {'damage_type': 'backtrace', 'id': 2022650170, 'ino': 1099511627832, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601_fsm'}, {'damage_type': 'backtrace', 'id': 2023140864, 'ino': 1099511628516, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_fsm'}, {'damage_type': 'backtrace', 'id': 2024786969, 'ino': 1099511628495, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3118_vm'}, {'damage_type': 'backtrace', 'id': 2024894157, 'ino': 1099511628704, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3119'}, {'damage_type': 'backtrace', 'id': 2027951135, 'ino': 1099511628209, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2656'}, {'damage_type': 'backtrace', 'id': 2031922899, 'ino': 1099511628696, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1259_fsm'}, {'damage_type': 'backtrace', 'id': 2034337116, 'ino': 1099511628787, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16387_fsm'}, {'damage_type': 'backtrace', 'id': 2034774760, 'ino': 1099511627970, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2682'}, {'damage_type': 'backtrace', 'id': 2045369515, 'ino': 1099511628697, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_vm'}, {'damage_type': 'backtrace', 'id': 2049225970, 'ino': 1099511628219, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2669'}, {'damage_type': 'backtrace', 'id': 2071734784, 'ino': 1099511628253, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2224'}, {'damage_type': 'backtrace', 'id': 2085398637, 'ino': 1099511628694, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394'}, {'damage_type': 'backtrace', 'id': 2153366964, 'ino': 1099511628772, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2686'}, {'damage_type': 'backtrace', 'id': 2164493889, 'ino': 1099511627916, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3440'}, {'damage_type': 'backtrace', 'id': 2170393857, 'ino': 1099511628399, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3603_vm'}, {'damage_type': 'backtrace', 'id': 2182896016, 'ino': 1099511628220, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2995'}, {'damage_type': 'backtrace', 'id': 2188848228, 'ino': 1099511628237, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3541'}, {'damage_type': 'backtrace', 'id': 2196740223, 'ino': 1099511627926, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2651'}, {'damage_type': 'backtrace', 'id': 2201615890, 'ino': 1099511627958, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2674'}, {'damage_type': 'backtrace', 'id': 2209787314, 'ino': 1099511628673, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3466'}, {'damage_type': 'backtrace', 'id': 2219648698, 'ino': 1099511628657, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3085'}, {'damage_type': 'backtrace', 'id': 2221168928, 'ino': 1099511628726, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2663'}, {'damage_type': 'backtrace', 'id': 2226188796, 'ino': 1099511628667, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_vm'}, {'damage_type': 'backtrace', 'id': 2232947558, 'ino': 1099511628699, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2673'}, {'damage_type': 'backtrace', 'id': 2254199173, 'ino': 1099511628668, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3601_vm'}, {'damage_type': 'backtrace', 'id': 2262043659, 'ino': 1099511628759, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3534'}, {'damage_type': 'backtrace', 'id': 2262849222, 'ino': 1099511628186, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12187'}, {'damage_type': 'backtrace', 'id': 2277854901, 'ino': 1099511628208, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_fsm'}, {'damage_type': 'backtrace', 'id': 2291467877, 'ino': 1099511628644, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2684'}, {'damage_type': 'backtrace', 'id': 2308559615, 'ino': 1099511628248, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3395'}, {'damage_type': 'backtrace', 'id': 2326098286, 'ino': 1099511628181, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2835'}, {'damage_type': 'backtrace', 'id': 2349542776, 'ino': 1099511627847, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3466'}, {'damage_type': 'backtrace', 'id': 2363434583, 'ino': 1099511628437, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12182_fsm'}, {'damage_type': 'backtrace', 'id': 2363740024, 'ino': 1099511628671, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840'}, {'damage_type': 'backtrace', 'id': 2364941292, 'ino': 1099511628177, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607'}, {'damage_type': 'backtrace', 'id': 2366543704, 'ino': 1099511628025, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3080'}, {'damage_type': 'backtrace', 'id': 2384033025, 'ino': 1099511627996, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3575'}, {'damage_type': 'backtrace', 'id': 2388297303, 'ino': 1099511628403, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3119'}, {'damage_type': 'backtrace', 'id': 2397895923, 'ino': 1099511628638, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2613_vm'}, {'damage_type': 'backtrace', 'id': 2401225021, 'ino': 1099511628651, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3381'}, {'damage_type': 'backtrace', 'id': 2455170063, 'ino': 1099511628677, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995_vm'}, {'damage_type': 'backtrace', 'id': 2463035736, 'ino': 1099511628081, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2611_vm'}, {'damage_type': 'backtrace', 'id': 2481528301, 'ino': 1099511628228, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12202_vm'}, {'damage_type': 'backtrace', 'id': 2492447014, 'ino': 1099511628194, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12212'}, {'damage_type': 'backtrace', 'id': 2494798673, 'ino': 1099511628481, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3257'}, {'damage_type': 'backtrace', 'id': 2503687347, 'ino': 1099511627987, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2692'}, {'damage_type': 'backtrace', 'id': 2507973901, 'ino': 1099511628496, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12187'}, {'damage_type': 'backtrace', 'id': 2514986690, 'ino': 1099511628412, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608'}, {'damage_type': 'backtrace', 'id': 2531562116, 'ino': 1099511628146, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12206'}, {'damage_type': 'backtrace', 'id': 2542185188, 'ino': 1099511628703, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_fsm'}, {'damage_type': 'backtrace', 'id': 2547879719, 'ino': 1099511628833, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3766'}, {'damage_type': 'backtrace', 'id': 2558264724, 'ino': 1099511628238, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3600'}, {'damage_type': 'backtrace', 'id': 2560242791, 'ino': 1099511628686, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3576'}, {'damage_type': 'backtrace', 'id': 2571935418, 'ino': 1099511628098, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_vm'}, {'damage_type': 'backtrace', 'id': 2587332909, 'ino': 1099511628640, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618'}, {'damage_type': 'backtrace', 'id': 2589855535, 'ino': 1099511628701, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3575'}, {'damage_type': 'backtrace', 'id': 2603933160, 'ino': 1099511628222, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12201'}, {'damage_type': 'backtrace', 'id': 2622496735, 'ino': 1099511627927, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2652'}, {'damage_type': 'backtrace', 'id': 2630710603, 'ino': 1099511627977, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2685'}, {'damage_type': 'backtrace', 'id': 2654684102, 'ino': 1099511628707, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_fsm'}, {'damage_type': 'backtrace', 'id': 2656705970, 'ino': 1099511628512, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192'}, {'damage_type': 'backtrace', 'id': 2661332487, 'ino': 1099511628637, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12191'}, {'damage_type': 'backtrace', 'id': 2685614622, 'ino': 1099511628199, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2683'}, {'damage_type': 'backtrace', 'id': 2700807570, 'ino': 1099511627883, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3256'}, {'damage_type': 'backtrace', 'id': 2703645616, 'ino': 1099511628708, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6110'}, {'damage_type': 'backtrace', 'id': 2711156148, 'ino': 1099511628243, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1255'}, {'damage_type': 'backtrace', 'id': 2721927821, 'ino': 1099511628753, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2755'}, {'damage_type': 'backtrace', 'id': 2727735432, 'ino': 1099511627834, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2602_fsm'}, {'damage_type': 'backtrace', 'id': 2728153296, 'ino': 1099511628429, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247'}, {'damage_type': 'backtrace', 'id': 2739380529, 'ino': 1099511627982, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2754'}, {'damage_type': 'backtrace', 'id': 2752520494, 'ino': 1099511628245, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2650'}, {'damage_type': 'backtrace', 'id': 2754781259, 'ino': 1099511628636, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12182_vm'}, {'damage_type': 'backtrace', 'id': 2758280198, 'ino': 1099511628216, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2608_vm'}, {'damage_type': 'backtrace', 'id': 2765093291, 'ino': 1099511628206, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_fsm'}, {'damage_type': 'backtrace', 'id': 2788812987, 'ino': 1099511628502, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2665'}, {'damage_type': 'backtrace', 'id': 2797010344, 'ino': 1099511628741, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610'}, {'damage_type': 'backtrace', 'id': 2798084428, 'ino': 1099511628641, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2618_fsm'}, {'damage_type': 'backtrace', 'id': 2799503804, 'ino': 1099511628514, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2838_vm'}, {'damage_type': 'backtrace', 'id': 2803920126, 'ino': 1099511628016, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/549'}, {'damage_type': 'backtrace', 'id': 2812760492, 'ino': 1099511627999, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2702'}, {'damage_type': 'backtrace', 'id': 2814411488, 'ino': 1099511627836, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2603_fsm'}, {'damage_type': 'backtrace', 'id': 2828253269, 'ino': 1099511628421, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2691'}, {'damage_type': 'backtrace', 'id': 2833202741, 'ino': 1099511627931, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2655'}, {'damage_type': 'backtrace', 'id': 2838160220, 'ino': 1099511627957, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2673'}, {'damage_type': 'backtrace', 'id': 2848405361, 'ino': 1099511628742, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3606'}, {'damage_type': 'backtrace', 'id': 2852304724, 'ino': 1099511628250, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12186'}, {'damage_type': 'backtrace', 'id': 2866392511, 'ino': 1099511628000, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3467'}, {'damage_type': 'backtrace', 'id': 2867813089, 'ino': 1099511628718, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6111'}, {'damage_type': 'backtrace', 'id': 2884588478, 'ino': 1099511628229, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3605'}, {'damage_type': 'backtrace', 'id': 2884951297, 'ino': 1099511628519, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2656'}, {'damage_type': 'backtrace', 'id': 2900456686, 'ino': 1099511628046, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3541_vm'}, {'damage_type': 'backtrace', 'id': 2901868457, 'ino': 1099511628683, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/PG_VERSION'}, {'damage_type': 'backtrace', 'id': 2908368162, 'ino': 1099511628436, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3455'}, {'damage_type': 'backtrace', 'id': 2909523238, 'ino': 1099511628768, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6104_vm'}, {'damage_type': 'backtrace', 'id': 2922936693, 'ino': 1099511628174, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601'}, {'damage_type': 'backtrace', 'id': 2954975424, 'ino': 1099511628649, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_fsm'}, {'damage_type': 'backtrace', 'id': 2959498862, 'ino': 1099511628405, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2668'}, {'damage_type': 'backtrace', 'id': 2981972181, 'ino': 1099511628734, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3394_vm'}, {'damage_type': 'backtrace', 'id': 3002444755, 'ino': 1099511628413, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2831'}, {'damage_type': 'backtrace', 'id': 3030521368, 'ino': 1099511628485, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2753_vm'}, {'damage_type': 'backtrace', 'id': 3065635104, 'ino': 1099511629091, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_internal.init'}, {'damage_type': 'backtrace', 'id': 3069623974, 'ino': 1099511628652, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2654'}, {'damage_type': 'backtrace', 'id': 3074539296, 'ino': 1099511627946, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3164'}, {'damage_type': 'backtrace', 'id': 3084818936, 'ino': 1099511628240, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2604'}, {'damage_type': 'backtrace', 'id': 3090056056, 'ino': 1099511628492, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2837'}, {'damage_type': 'backtrace', 'id': 3094787180, 'ino': 1099511628192, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2665'}, {'damage_type': 'backtrace', 'id': 3105048892, 'ino': 1099511628200, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12196'}, {'damage_type': 'backtrace', 'id': 3121239627, 'ino': 1099511628233, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2603_vm'}, {'damage_type': 'backtrace', 'id': 3140492725, 'ino': 1099511627824, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610'}, {'damage_type': 'backtrace', 'id': 3144202012, 'ino': 1099511628184, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1417'}, {'damage_type': 'backtrace', 'id': 3150071307, 'ino': 1099511628189, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3596_vm'}, {'damage_type': 'backtrace', 'id': 3161488743, 'ino': 1099511628231, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3609'}, {'damage_type': 'backtrace', 'id': 3165076704, 'ino': 1099511628775, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2830_vm'}, {'damage_type': 'backtrace', 'id': 3171663739, 'ino': 1099511627944, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2663'}, {'damage_type': 'backtrace', 'id': 3176361022, 'ino': 1099511628191, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_fsm'}, {'damage_type': 'backtrace', 'id': 3187537605, 'ino': 1099511628179, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2609_fsm'}, {'damage_type': 'backtrace', 'id': 3204873079, 'ino': 1099511628431, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3764'}, {'damage_type': 'backtrace', 'id': 3206471999, 'ino': 1099511628751, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606_vm'}, {'damage_type': 'backtrace', 'id': 3222949632, 'ino': 1099511628689, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6106_vm'}, {'damage_type': 'backtrace', 'id': 3223009150, 'ino': 1099511628236, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2601_vm'}, {'damage_type': 'backtrace', 'id': 3227562465, 'ino': 1099511628507, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3379'}, {'damage_type': 'backtrace', 'id': 3239582100, 'ino': 1099511628748, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2840_fsm'}, {'damage_type': 'backtrace', 'id': 3261909279, 'ino': 1099511628720, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/113'}, {'damage_type': 'backtrace', 'id': 3280528016, 'ino': 1099511628647, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2696'}, {'damage_type': 'backtrace', 'id': 3293791023, 'ino': 1099511628723, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605_vm'}, {'damage_type': 'backtrace', 'id': 3295108097, 'ino': 1099511628435, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12199'}, {'damage_type': 'backtrace', 'id': 3297414514, 'ino': 1099511627909, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838'}, {'damage_type': 'backtrace', 'id': 3298243610, 'ino': 1099511628487, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607'}, {'damage_type': 'backtrace', 'id': 3305209574, 'ino': 1099511628783, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16384'}, {'damage_type': 'backtrace', 'id': 3316849728, 'ino': 1099511628758, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541_fsm'}, {'damage_type': 'backtrace', 'id': 3323838526, 'ino': 1099511628653, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602_fsm'}, {'damage_type': 'backtrace', 'id': 3325126560, 'ino': 1099511628230, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2616_vm'}, {'damage_type': 'backtrace', 'id': 3341171733, 'ino': 1099511628662, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_fsm'}, {'damage_type': 'backtrace', 'id': 3359263420, 'ino': 1099511628760, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439'}, {'damage_type': 'backtrace', 'id': 3383123929, 'ino': 1099511627850, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2605_fsm'}, {'damage_type': 'backtrace', 'id': 3397441125, 'ino': 1099511628685, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1418_vm'}, {'damage_type': 'backtrace', 'id': 3401083054, 'ino': 1099511628420, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3604'}, {'damage_type': 'backtrace', 'id': 3405993011, 'ino': 1099511628173, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/6113'}, {'damage_type': 'backtrace', 'id': 3425970270, 'ino': 1099511628482, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12192_vm'}, {'damage_type': 'backtrace', 'id': 3440528607, 'ino': 1099511628223, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2607_fsm'}, {'damage_type': 'backtrace', 'id': 3447887878, 'ino': 1099511628235, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2617_vm'}, {'damage_type': 'backtrace', 'id': 3456875240, 'ino': 1099511627914, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2841'}, {'damage_type': 'backtrace', 'id': 3457523978, 'ino': 1099511628728, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2836'}, {'damage_type': 'backtrace', 'id': 3468389698, 'ino': 1099511628205, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2619_fsm'}, {'damage_type': 'backtrace', 'id': 3495048706, 'ino': 1099511628060, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3603_vm'}, {'damage_type': 'backtrace', 'id': 3503343611, 'ino': 1099511627876, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764'}, {'damage_type': 'backtrace', 'id': 3508910513, 'ino': 1099511628241, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3439_vm'}, {'damage_type': 'backtrace', 'id': 3516383265, 'ino': 1099511628639, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3607'}, {'damage_type': 'backtrace', 'id': 3546969727, 'ino': 1099511628745, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2600_fsm'}, {'damage_type': 'backtrace', 'id': 3557021465, 'ino': 1099511628043, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3350_vm'}, {'damage_type': 'backtrace', 'id': 3560763107, 'ino': 1099511628182, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2837'}, {'damage_type': 'backtrace', 'id': 3562763990, 'ino': 1099511628786, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16393'}, {'damage_type': 'backtrace', 'id': 3585212493, 'ino': 1099511628731, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603'}, {'damage_type': 'backtrace', 'id': 3589020454, 'ino': 1099511628441, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3606'}, {'damage_type': 'backtrace', 'id': 3620324691, 'ino': 1099511627855, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608'}, {'damage_type': 'backtrace', 'id': 3623193608, 'ino': 1099511628777, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2704'}, {'damage_type': 'backtrace', 'id': 3645829725, 'ino': 1099511628706, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2668'}, {'damage_type': 'backtrace', 'id': 3651919775, 'ino': 1099511628494, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1417'}, {'damage_type': 'backtrace', 'id': 3652026829, 'ino': 1099511628195, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456'}, {'damage_type': 'backtrace', 'id': 3667312790, 'ino': 1099511628702, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3467'}, {'damage_type': 'backtrace', 'id': 3690456415, 'ino': 1099511628678, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/549'}, {'damage_type': 'backtrace', 'id': 3706018960, 'ino': 1099511628409, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2655'}, {'damage_type': 'backtrace', 'id': 3739088131, 'ino': 1099511628776, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3574'}, {'damage_type': 'backtrace', 'id': 3770626818, 'ino': 1099511627973, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3380'}, {'damage_type': 'backtrace', 'id': 3780927971, 'ino': 1099511628211, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3456_fsm'}, {'damage_type': 'backtrace', 'id': 3791404421, 'ino': 1099511628676, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/6112'}, {'damage_type': 'backtrace', 'id': 3793019167, 'ino': 1099511628692, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_vm'}, {'damage_type': 'backtrace', 'id': 3795083720, 'ino': 1099511627881, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1418'}, {'damage_type': 'backtrace', 'id': 3795331581, 'ino': 1099511628705, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2674'}, {'damage_type': 'backtrace', 'id': 3802600617, 'ino': 1099511628203, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_fsm'}, {'damage_type': 'backtrace', 'id': 3810761205, 'ino': 1099511627952, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2668'}, {'damage_type': 'backtrace', 'id': 3829925249, 'ino': 1099511628058, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3601_vm'}, {'damage_type': 'backtrace', 'id': 3831323693, 'ino': 1099511628688, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834'}, {'damage_type': 'backtrace', 'id': 3835202360, 'ino': 1099511628442, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2613'}, {'damage_type': 'backtrace', 'id': 3838543516, 'ino': 1099511628721, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3604'}, {'damage_type': 'backtrace', 'id': 3861512174, 'ino': 1099511628719, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2682'}, {'damage_type': 'backtrace', 'id': 3869389656, 'ino': 1099511627815, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247'}, {'damage_type': 'backtrace', 'id': 3876956954, 'ino': 1099511628665, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602_fsm'}, {'damage_type': 'backtrace', 'id': 3880932869, 'ino': 1099511628057, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3764_vm'}, {'damage_type': 'backtrace', 'id': 3894381177, 'ino': 1099511628204, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838_vm'}, {'damage_type': 'backtrace', 'id': 3894384184, 'ino': 1099511628674, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3380'}, {'damage_type': 'backtrace', 'id': 3901403539, 'ino': 1099511628690, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3256'}, {'damage_type': 'backtrace', 'id': 3901580503, 'ino': 1099511628244, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2612'}, {'damage_type': 'backtrace', 'id': 3904303507, 'ino': 1099511628747, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/pg_filenode.map'}, {'damage_type': 'backtrace', 'id': 3904475441, 'ino': 1099511627901, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2831'}, {'damage_type': 'backtrace', 'id': 3925748932, 'ino': 1099511628418, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2682'}, {'damage_type': 'backtrace', 'id': 3957443822, 'ino': 1099511628033, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6110'}, {'damage_type': 'backtrace', 'id': 3975443541, 'ino': 1099511628761, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3081'}, {'damage_type': 'backtrace', 'id': 3976270643, 'ino': 1099511627961, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3502'}, {'damage_type': 'backtrace', 'id': 3983961767, 'ino': 1099511628740, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12207'}, {'damage_type': 'backtrace', 'id': 3985588093, 'ino': 1099511628754, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/175'}, {'damage_type': 'backtrace', 'id': 3998894244, 'ino': 1099511628764, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2606'}, {'damage_type': 'backtrace', 'id': 4006727467, 'ino': 1099511628252, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/3602'}, {'damage_type': 'backtrace', 'id': 4023642885, 'ino': 1099511628234, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/826'}, {'damage_type': 'backtrace', 'id': 4026296629, 'ino': 1099511628691, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2652'}, {'damage_type': 'backtrace', 'id': 4055829176, 'ino': 1099511628188, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2660'}, {'damage_type': 'backtrace', 'id': 4059070680, 'ino': 1099511628213, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/1247_vm'}, {'damage_type': 'backtrace', 'id': 4059202085, 'ino': 1099511628426, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2838'}, {'damage_type': 'backtrace', 'id': 4062741227, 'ino': 1099511628744, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3079_vm'}, {'damage_type': 'backtrace', 'id': 4071904133, 'ino': 1099511628508, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2610_vm'}, {'damage_type': 'backtrace', 'id': 4075824678, 'ino': 1099511628115, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/PG_VERSION'}, {'damage_type': 'backtrace', 'id': 4086295425, 'ino': 1099511628424, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/2685'}, {'damage_type': 'backtrace', 'id': 4097773482, 'ino': 1099511628672, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2611_vm'}, {'damage_type': 'backtrace', 'id': 4145056279, 'ino': 1099511627945, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3455'}, {'damage_type': 'backtrace', 'id': 4157057928, 'ino': 1099511627986, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2691'}, {'damage_type': 'backtrace', 'id': 4173522724, 'ino': 1099511628698, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2651'}, {'damage_type': 'backtrace', 'id': 4198690260, 'ino': 1099511628480, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2662'}, {'damage_type': 'backtrace', 'id': 4208012639, 'ino': 1099511628153, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12182_fsm'}, {'damage_type': 'backtrace', 'id': 4212301730, 'ino': 1099511628180, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12346/12204'}, {'damage_type': 'backtrace', 'id': 4216884729, 'ino': 1099511628643, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2690'}, {'damage_type': 'backtrace', 'id': 4235147401, 'ino': 1099511628633, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224_vm'}, {'damage_type': 'backtrace', 'id': 4240216921, 'ino': 1099511628501, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_fsm'}, {'damage_type': 'backtrace', 'id': 4243086697, 'ino': 1099511628711, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3597'}, {'damage_type': 'backtrace', 'id': 4271947166, 'ino': 1099511628007, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3606'}, {'damage_type': 'backtrace', 'id': 4286865045, 'ino': 1099511627894, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576'}] 2024-01-11T17:59:15.993 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 13790604, 'ino': 1099511628523, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1247_vm'}, {'damage_type': 'backtrace', 'id': 50184632, 'ino': 1099511628521, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_fsm'}, {'damage_type': 'backtrace', 'id': 134275798, 'ino': 1099511628558, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3395'}, {'damage_type': 'backtrace', 'id': 167486833, 'ino': 1099511628126, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2609_fsm'}, {'damage_type': 'backtrace', 'id': 255320999, 'ino': 1099511628551, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3439_vm'}, {'damage_type': 'backtrace', 'id': 289285072, 'ino': 1099511628529, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2669'}, {'damage_type': 'backtrace', 'id': 309086444, 'ino': 1099511628788, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/16396'}, {'damage_type': 'backtrace', 'id': 378764530, 'ino': 1099511628532, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12201'}, {'damage_type': 'backtrace', 'id': 382487609, 'ino': 1099511628138, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192'}, {'damage_type': 'backtrace', 'id': 431520127, 'ino': 1099511628531, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3503'}, {'damage_type': 'backtrace', 'id': 531646387, 'ino': 1099511627831, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2601'}, {'damage_type': 'backtrace', 'id': 566540848, 'ino': 1099511628009, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3766'}, {'damage_type': 'backtrace', 'id': 581650556, 'ino': 1099511628076, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3381_vm'}, {'damage_type': 'backtrace', 'id': 587602298, 'ino': 1099511628542, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2666'}, {'damage_type': 'backtrace', 'id': 639098914, 'ino': 1099511627941, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2660'}, {'damage_type': 'backtrace', 'id': 642531777, 'ino': 1099511628534, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2699'}, {'damage_type': 'backtrace', 'id': 646008570, 'ino': 1099511628537, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/828'}, {'damage_type': 'backtrace', 'id': 684328946, 'ino': 1099511627943, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2662'}, {'damage_type': 'backtrace', 'id': 687912895, 'ino': 1099511628563, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2224'}, {'damage_type': 'backtrace', 'id': 689771969, 'ino': 1099511627975, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3379'}, {'damage_type': 'backtrace', 'id': 703001723, 'ino': 1099511627965, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2679'}, {'damage_type': 'backtrace', 'id': 875859433, 'ino': 1099511628522, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202'}, {'damage_type': 'backtrace', 'id': 976285531, 'ino': 1099511629088, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/log/postgresql-Thu.log'}, {'damage_type': 'backtrace', 'id': 977131440, 'ino': 1099511628082, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2610_vm'}, {'damage_type': 'backtrace', 'id': 1094230645, 'ino': 1099511627932, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2757'}, {'damage_type': 'backtrace', 'id': 1302451726, 'ino': 1099511628553, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/1255'}, {'damage_type': 'backtrace', 'id': 1345347512, 'ino': 1099511628555, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2650'}, {'damage_type': 'backtrace', 'id': 1385673230, 'ino': 1099511628530, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2995'}, {'damage_type': 'backtrace', 'id': 1388181362, 'ino': 1099511628539, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3605'}, {'damage_type': 'backtrace', 'id': 1425251485, 'ino': 1099511628036, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/6113'}, {'damage_type': 'backtrace', 'id': 1490579165, 'ino': 1099511627989, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/5002'}, {'damage_type': 'backtrace', 'id': 1589182223, 'ino': 1099511627880, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1417'}, {'damage_type': 'backtrace', 'id': 1599249175, 'ino': 1099511628536, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612_vm'}, {'damage_type': 'backtrace', 'id': 1604476727, 'ino': 1099511628154, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2619_fsm'}, {'damage_type': 'backtrace', 'id': 1607042801, 'ino': 1099511628561, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/174'}, {'damage_type': 'backtrace', 'id': 1786667704, 'ino': 1099511628557, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2834_vm'}, {'damage_type': 'backtrace', 'id': 1860098694, 'ino': 1099511628120, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_fsm'}, {'damage_type': 'backtrace', 'id': 1966666233, 'ino': 1099511628550, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2604'}, {'damage_type': 'backtrace', 'id': 1993518878, 'ino': 1099511627889, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3456'}, {'damage_type': 'backtrace', 'id': 2072975109, 'ino': 1099511628119, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2608_fsm'}, {'damage_type': 'backtrace', 'id': 2109526528, 'ino': 1099511627816, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1247_fsm'}, {'damage_type': 'backtrace', 'id': 2132076108, 'ino': 1099511628072, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753_vm'}, {'damage_type': 'backtrace', 'id': 2182274743, 'ino': 1099511627854, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2607'}, {'damage_type': 'backtrace', 'id': 2213166050, 'ino': 1099511628540, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2616_vm'}, {'damage_type': 'backtrace', 'id': 2228223670, 'ino': 1099511628140, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12196'}, {'damage_type': 'backtrace', 'id': 2386011949, 'ino': 1099511628520, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2615'}, {'damage_type': 'backtrace', 'id': 2400355414, 'ino': 1099511627827, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2753'}, {'damage_type': 'backtrace', 'id': 2416396558, 'ino': 1099511627830, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2616_fsm'}, {'damage_type': 'backtrace', 'id': 2601818157, 'ino': 1099511628108, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2838_vm'}, {'damage_type': 'backtrace', 'id': 2640576164, 'ino': 1099511628528, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3456_vm'}, {'damage_type': 'backtrace', 'id': 2699289062, 'ino': 1099511627971, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2683'}, {'damage_type': 'backtrace', 'id': 2728168173, 'ino': 1099511628560, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12186'}, {'damage_type': 'backtrace', 'id': 2734939086, 'ino': 1099511628545, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2617_vm'}, {'damage_type': 'backtrace', 'id': 2758908716, 'ino': 1099511627780, 'path': '/volumes/qa/sv_0/.meta'}, {'damage_type': 'backtrace', 'id': 2759375477, 'ino': 1099511627779, 'path': '/volumes/qa/sv_0/5ef7e487-9779-443a-aaca-b2f24334f17d'}, {'damage_type': 'backtrace', 'id': 2819278711, 'ino': 1099511628538, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/12202_vm'}, {'damage_type': 'backtrace', 'id': 2862224175, 'ino': 1099511628543, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2603_vm'}, {'damage_type': 'backtrace', 'id': 2881511371, 'ino': 1099511628541, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3609'}, {'damage_type': 'backtrace', 'id': 2888287150, 'ino': 1099511627911, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3598'}, {'damage_type': 'backtrace', 'id': 2920705272, 'ino': 1099511628535, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2658'}, {'damage_type': 'backtrace', 'id': 2929132444, 'ino': 1099511628064, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3576_vm'}, {'damage_type': 'backtrace', 'id': 2932108642, 'ino': 1099511628546, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2601_vm'}, {'damage_type': 'backtrace', 'id': 2953941199, 'ino': 1099511628554, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2612'}, {'damage_type': 'backtrace', 'id': 2954013507, 'ino': 1099511628548, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3600'}, {'damage_type': 'backtrace', 'id': 2973161045, 'ino': 1099511628547, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3541'}, {'damage_type': 'backtrace', 'id': 3031313559, 'ino': 1099511627949, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2665'}, {'damage_type': 'backtrace', 'id': 3040576706, 'ino': 1099511628160, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12192_vm'}, {'damage_type': 'backtrace', 'id': 3053890291, 'ino': 1099511628562, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3602'}, {'damage_type': 'backtrace', 'id': 3166002378, 'ino': 1099511628148, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12209'}, {'damage_type': 'backtrace', 'id': 3226040233, 'ino': 1099511628564, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2689'}, {'damage_type': 'backtrace', 'id': 3273250530, 'ino': 1099511628527, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3767'}, {'damage_type': 'backtrace', 'id': 3287256761, 'ino': 1099511628091, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/1259_vm'}, {'damage_type': 'backtrace', 'id': 3350161054, 'ino': 1099511628524, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2605'}, {'damage_type': 'backtrace', 'id': 3510757208, 'ino': 1099511628533, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2607_fsm'}, {'damage_type': 'backtrace', 'id': 3542858592, 'ino': 1099511628526, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2608_vm'}, {'damage_type': 'backtrace', 'id': 3593799248, 'ino': 1099511628549, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2688'}, {'damage_type': 'backtrace', 'id': 3601079468, 'ino': 1099511628135, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12187'}, {'damage_type': 'backtrace', 'id': 3624953545, 'ino': 1099511628049, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3596_vm'}, {'damage_type': 'backtrace', 'id': 3634818380, 'ino': 1099511628556, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2602'}, {'damage_type': 'backtrace', 'id': 3653723863, 'ino': 1099511627908, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2837'}, {'damage_type': 'backtrace', 'id': 3735461535, 'ino': 1099511628052, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3118_vm'}, {'damage_type': 'backtrace', 'id': 3820707759, 'ino': 1099511627933, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2656'}, {'damage_type': 'backtrace', 'id': 3870955621, 'ino': 1099511628525, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/3764_fsm'}, {'damage_type': 'backtrace', 'id': 3909676804, 'ino': 1099511628028, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/3257'}, {'damage_type': 'backtrace', 'id': 3912427587, 'ino': 1099511628150, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12212'}, {'damage_type': 'backtrace', 'id': 3942882063, 'ino': 1099511627906, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/2835'}, {'damage_type': 'backtrace', 'id': 4046950217, 'ino': 1099511628552, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2609'}, {'damage_type': 'backtrace', 'id': 4087564760, 'ino': 1099511628145, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/1/12204'}, {'damage_type': 'backtrace', 'id': 4134929423, 'ino': 1099511628559, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/2675'}, {'damage_type': 'backtrace', 'id': 4198055850, 'ino': 1099511628544, 'path': '/volumes/qa/sv_1/09d4c4b1-901d-4a41-b788-7617dcac494e/postgres/data/base/12347/826'}] 2024-01-11T17:59:15.993 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-01-11T17:59:15.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:15 smithi016 ceph-mon[43195]: pgmap v343: 129 pgs: 129 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 799 op/s 2024-01-11T17:59:16.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:15 smithi038 ceph-mon[43582]: pgmap v343: 129 pgs: 129 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 799 op/s 2024-01-11T17:59:17.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:17 smithi016 ceph-mon[43195]: pgmap v344: 129 pgs: 129 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 22 MiB/s wr, 878 op/s 2024-01-11T17:59:18.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:17 smithi038 ceph-mon[43582]: pgmap v344: 129 pgs: 129 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 22 MiB/s wr, 878 op/s 2024-01-11T17:59:18.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:17 smithi007 ceph-mon[33361]: pgmap v344: 129 pgs: 129 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 22 MiB/s wr, 878 op/s 2024-01-11T17:59:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 50.0 s, 1007.0 tps, lat 99.170 ms stddev 135.190 2024-01-11T17:59:20.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:19 smithi038 ceph-mon[43582]: pgmap v345: 129 pgs: 129 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 714 op/s 2024-01-11T17:59:20.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:19 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:59:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:19 smithi007 ceph-mon[33361]: pgmap v345: 129 pgs: 129 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 714 op/s 2024-01-11T17:59:20.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:19 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:59:20.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:19 smithi016 ceph-mon[43195]: pgmap v345: 129 pgs: 129 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 714 op/s 2024-01-11T17:59:20.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:19 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T17:59:22.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:21 smithi038 ceph-mon[43582]: pgmap v346: 129 pgs: 129 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 24 MiB/s wr, 850 op/s 2024-01-11T17:59:22.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:59:22.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:59:22.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:59:22.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:21 smithi007 ceph-mon[33361]: pgmap v346: 129 pgs: 129 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 24 MiB/s wr, 850 op/s 2024-01-11T17:59:22.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:59:22.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:59:22.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:59:22.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:21 smithi016 ceph-mon[43195]: pgmap v346: 129 pgs: 129 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 24 MiB/s wr, 850 op/s 2024-01-11T17:59:22.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T17:59:22.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T17:59:22.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T17:59:23.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:22 smithi007 ceph-mon[33361]: pgmap v347: 129 pgs: 129 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 913 op/s 2024-01-11T17:59:23.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:22 smithi016 ceph-mon[43195]: pgmap v347: 129 pgs: 129 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 913 op/s 2024-01-11T17:59:23.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:22 smithi038 ceph-mon[43582]: pgmap v347: 129 pgs: 129 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 26 MiB/s wr, 913 op/s 2024-01-11T17:59:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 55.0 s, 1221.4 tps, lat 82.263 ms stddev 114.504 2024-01-11T17:59:25.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:25 smithi016 ceph-mon[43195]: pgmap v348: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 28 MiB/s wr, 927 op/s 2024-01-11T17:59:25.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:25 smithi038 ceph-mon[43582]: pgmap v348: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 28 MiB/s wr, 927 op/s 2024-01-11T17:59:25.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:25 smithi007 ceph-mon[33361]: pgmap v348: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 28 MiB/s wr, 927 op/s 2024-01-11T17:59:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.824+0000 7f8ce95ff700 1 -- 172.21.15.7:0/2123901078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce41027a0 msgr2=0x7f8ce4102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.824+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/2123901078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce41027a0 0x7f8ce4102c00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800fb20 tx=0x7f8cd800fe30 comp rx=0 tx=0).stop 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 -- 172.21.15.7:0/2123901078 shutdown_connections 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/2123901078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/2123901078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce4103140 0x7f8ce41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/2123901078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce41027a0 0x7f8ce4102c00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 -- 172.21.15.7:0/2123901078 >> 172.21.15.7:0/2123901078 conn(0x7f8ce40fcb00 msgr2=0x7f8ce40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 -- 172.21.15.7:0/2123901078 shutdown_connections 2024-01-11T17:59:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.825+0000 7f8ce95ff700 1 -- 172.21.15.7:0/2123901078 wait complete. 2024-01-11T17:59:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.826+0000 7f8ce95ff700 1 Processor -- start 2024-01-11T17:59:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 -- start start 2024-01-11T17:59:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41027a0 0x7f8ce419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4103140 0x7f8ce41a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41a5600 con 0x7f8ce41027a0 2024-01-11T17:59:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce406b030 con 0x7f8ce4103140 2024-01-11T17:59:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce95ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce406b1c0 con 0x7f8ce41015b0 2024-01-11T17:59:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50822/0 (socket says 172.21.15.7:50822) 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/286511863 learned_addr learned my addr 172.21.15.7:0/286511863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.827+0000 7f8ce37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4103140 0x7f8ce41a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/286511863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4103140 msgr2=0x7f8ce41a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4103140 0x7f8ce41a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41027a0 msgr2=0x7f8ce419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41027a0 0x7f8ce419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/286511863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd800f750 con 0x7f8ce41015b0 2024-01-11T17:59:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce419da80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400c970 tx=0x7f8cd400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ccbfff700 1 -- 172.21.15.7:0/286511863 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004220 con 0x7f8ce41015b0 2024-01-11T17:59:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ccbfff700 1 -- 172.21.15.7:0/286511863 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd4007d60 con 0x7f8ce41015b0 2024-01-11T17:59:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.828+0000 7f8ccbfff700 1 -- 172.21.15.7:0/286511863 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4003c90 con 0x7f8ce41015b0 2024-01-11T17:59:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.829+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce406b4a0 con 0x7f8ce41015b0 2024-01-11T17:59:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.829+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce406b960 con 0x7f8ce41015b0 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.832+0000 7f8ccbfff700 1 -- 172.21.15.7:0/286511863 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f8cd4004380 con 0x7f8ce41015b0 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 Processor -- start 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 -- start start 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8ccbfff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8ccc0a1f50 0x7f8ccc0a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b17fd5030 con 0x559b180ddf20 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18038ac0 con 0x559b180dec10 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b180363e0 con 0x559b180df5d0 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.834+0000 7f8ccbfff700 1 -- 172.21.15.7:0/286511863 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8cd40cd800 con 0x7f8ce41015b0 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8ccc0a1f50 0x7f8ccc0a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41644/0 (socket says 172.21.15.7:41644) 2024-01-11T17:59:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 -- 172.21.15.7:0/1818795096 learned_addr learned my addr 172.21.15.7:0/1818795096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180de340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42382/0 (socket says 172.21.15.7:42382) 2024-01-11T17:59:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 -- 172.21.15.7:0/1818795096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 msgr2=0x559b180e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180e5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 -- 172.21.15.7:0/1818795096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 msgr2=0x559b180de340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180de340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 -- 172.21.15.7:0/1818795096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b17ea6350 con 0x559b180dec10 2024-01-11T17:59:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8ccc0a1f50 0x7f8ccc0a4410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009550 tx=0x7f8cdc006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.835+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180df090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006010 tx=0x7f8cd8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649d8c5d345fd955 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.836+0000 7f8cc97fa700 1 -- 172.21.15.7:0/1818795096 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8003880 con 0x559b180dec10 2024-01-11T17:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.836+0000 7f8cc97fa700 1 -- 172.21.15.7:0/1818795096 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd8004660 con 0x559b180dec10 2024-01-11T17:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.836+0000 7f8cc97fa700 1 -- 172.21.15.7:0/1818795096 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8004030 con 0x559b180dec10 2024-01-11T17:59:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/1818795096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 msgr2=0x559b180df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180df090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006010 tx=0x7f8cd8005e80 comp rx=0 tx=0).stop 2024-01-11T17:59:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/1818795096 shutdown_connections 2024-01-11T17:59:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/1818795096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180df090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/1818795096 >> 172.21.15.7:0/1818795096 conn(0x559b18053740 msgr2=0x559b180d8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/1818795096 shutdown_connections 2024-01-11T17:59:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/1818795096 wait complete. 2024-01-11T17:59:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.837+0000 7f8cfd2c6b80 1 -- start start 2024-01-11T17:59:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 Processor -- start 2024-01-11T17:59:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180ddf20 0x559b180e7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 0x559b180e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18031cd0 con 0x559b180dec10 2024-01-11T17:59:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b17fa0340 con 0x559b180ddf20 2024-01-11T17:59:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b17f9ffe0 con 0x559b180df5d0 2024-01-11T17:59:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8cfd2c6b80 10 client.0 _ll_register_callbacks cb 0x559b17fd26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8ce27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 0x559b180e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180ddf20 0x559b180e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8ce27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 0x559b180e7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42390/0 (socket says 172.21.15.7:42390) 2024-01-11T17:59:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.838+0000 7f8ce27fc700 1 -- 172.21.15.7:0/3982039208 learned_addr learned my addr 172.21.15.7:0/3982039208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce27fc700 1 -- 172.21.15.7:0/3982039208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 msgr2=0x559b180eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce27fc700 1 -- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180ddf20 msgr2=0x559b180e7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180ddf20 0x559b180e7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce27fc700 1 -- 172.21.15.7:0/3982039208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce406b960 con 0x559b180dec10 2024-01-11T17:59:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180ddf20 0x559b180e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 0x559b180e7ea0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800f430 tx=0x7f8cd8009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e3d7a63539f68b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.839+0000 7f8cc6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/3982039208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8004180 con 0x559b180dec10 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/3982039208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8cd80042e0 con 0x559b180dec10 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b180d0d60 con 0x559b180dec10 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/3982039208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cd8009eb0 con 0x559b180dec10 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cfd2c6b80 10 client.15738 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8ce410a340 con 0x559b180dec10 2024-01-11T17:59:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/3982039208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b180d0d60 con 0x559b180dec10 2024-01-11T17:59:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.840+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/3982039208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8cd801f440 con 0x559b180dec10 2024-01-11T17:59:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8cfd2c6b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8cfd2c6b80 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8cfd2c6b80 20 client.15738 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b180f33b0 0x559b180f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8cfd2c6b80 4 client.15738 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b1802ab60 con 0x559b180f33b0 2024-01-11T17:59:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.841+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b180f33b0 0x559b180f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.842+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b180f33b0 0x559b180f57d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f7e3103ad91a64d server_cookie=7697a032ded029a2 in_seq=0 out_seq=0 2024-01-11T17:59:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.842+0000 7f8cc6ff5700 10 client.15738 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.842+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/3982039208 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b1802ab60 con 0x559b180f33b0 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.842+0000 7f8cc6ff5700 10 client.15738 handle_command_reply: tid=0 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 client.15738 shutdown 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 msgr2=0x559b180e7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 0x559b180e7ea0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800f430 tx=0x7f8cd8009c50 comp rx=0 tx=0).stop 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 shutdown_connections 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b180f33b0 0x559b180f57d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180df5d0 0x559b180eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cc6ff5700 0 client.15738 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180ddf20 0x559b180e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3982039208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180dec10 0x559b180e7ea0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 >> 172.21.15.7:0/3982039208 conn(0x559b18053740 msgr2=0x559b180d7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 shutdown_connections 2024-01-11T17:59:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3982039208 wait complete. 2024-01-11T17:59:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.843+0000 7f8cfd2c6b80 20 client.15738 trim_cache size 0 max 16384 2024-01-11T17:59:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 Processor -- start 2024-01-11T17:59:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 -- start start 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180dec10 0x559b180eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 0x559b180deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b17ea8640 con 0x559b180ddf20 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b180d5560 con 0x559b180df5d0 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.859+0000 7f8cfd2c6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b180da2e0 con 0x559b180dec10 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 0x559b180deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 0x559b180deff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41660/0 (socket says 172.21.15.7:41660) 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 -- 172.21.15.7:0/3049459211 learned_addr learned my addr 172.21.15.7:0/3049459211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180dec10 0x559b180eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 -- 172.21.15.7:0/3049459211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180dec10 msgr2=0x559b180eee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180dec10 0x559b180eee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 -- 172.21.15.7:0/3049459211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 msgr2=0x559b180e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180e65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce37fe700 1 -- 172.21.15.7:0/3049459211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b17e52b00 con 0x559b180df5d0 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.860+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.861+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 0x559b180deff0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc008fa0 tx=0x7f8cdc010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f31f5a2f53673c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.863+0000 7f8cc8ff9700 1 -- 172.21.15.7:0/3049459211 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00e030 con 0x559b180df5d0 2024-01-11T17:59:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.863+0000 7f8cc8ff9700 1 -- 172.21.15.7:0/3049459211 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cdc00f040 con 0x559b180df5d0 2024-01-11T17:59:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.863+0000 7f8cc8ff9700 1 -- 172.21.15.7:0/3049459211 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc026440 con 0x559b180df5d0 2024-01-11T17:59:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.865+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3049459211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 msgr2=0x559b180deff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.865+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 0x559b180deff0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc008fa0 tx=0x7f8cdc010000 comp rx=0 tx=0).stop 2024-01-11T17:59:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3049459211 shutdown_connections 2024-01-11T17:59:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180df5d0 0x559b180deff0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180ddf20 0x559b180e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/3049459211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180dec10 0x559b180eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3049459211 >> 172.21.15.7:0/3049459211 conn(0x559b18053740 msgr2=0x559b180d49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3049459211 shutdown_connections 2024-01-11T17:59:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.867+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/3049459211 wait complete. 2024-01-11T17:59:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.869+0000 7f8cfd2c6b80 1 -- start start 2024-01-11T17:59:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.870+0000 7f8cfd2c6b80 1 Processor -- start 2024-01-11T17:59:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.872+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 0x559b180ec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.872+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.872+0000 7f8cfd2c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180df5d0 0x559b180eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.873+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 0x559b180ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.873+0000 7f8ce2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 0x559b180ec520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50864/0 (socket says 172.21.15.7:50864) 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.874+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/675371341 learned_addr learned my addr 172.21.15.7:0/675371341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.874+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b180edf60 con 0x559b180df5d0 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.874+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b180ee0d0 con 0x559b180dec10 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.874+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b180ee290 con 0x559b180ddf20 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.874+0000 7f8cfd2c6b80 10 client.0 _ll_register_callbacks cb 0x559b17fd26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce37fe700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180df5d0 0x559b180eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 msgr2=0x559b180eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180eca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/675371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180df5d0 msgr2=0x559b180eda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180df5d0 0x559b180eda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.875+0000 7f8ce2ffd700 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc01f040 con 0x559b180ddf20 2024-01-11T17:59:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.878+0000 7f8ce2ffd700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 0x559b180ec520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40074a0 tx=0x7f8cd40036a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6fd61548e464b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.880+0000 7f8cc6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.880+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/675371341 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4078ea0 con 0x559b180ddf20 2024-01-11T17:59:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.880+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/675371341 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8cd40a2560 con 0x559b180ddf20 2024-01-11T17:59:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.880+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cd8004660 con 0x559b180ddf20 2024-01-11T17:59:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.883+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/675371341 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cd4098570 con 0x559b180ddf20 2024-01-11T17:59:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.883+0000 7f8cfd2c6b80 10 client.25828 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.883+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559b17ff31d0 con 0x559b180ddf20 2024-01-11T17:59:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.884+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/675371341 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd8004660 con 0x559b180ddf20 2024-01-11T17:59:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.884+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/675371341 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8cd4095fc0 con 0x559b180ddf20 2024-01-11T17:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.885+0000 7f8cfd2c6b80 10 client.25828 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.885+0000 7f8cfd2c6b80 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.885+0000 7f8cfd2c6b80 20 client.25828 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.885+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b1814a110 0x559b1814c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.885+0000 7f8cfd2c6b80 4 client.25828 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.886+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b1802a880 con 0x559b1814a110 2024-01-11T17:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.887+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b1814a110 0x559b1814c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.888+0000 7f8ce27fc700 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b1814a110 0x559b1814c530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78fdc72ad64857c8 server_cookie=5605246b7f20e868 in_seq=0 out_seq=0 2024-01-11T17:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.888+0000 7f8cc6ff5700 10 client.25828 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.989+0000 7f8cc6ff5700 1 -- 172.21.15.7:0/675371341 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b1802a880 con 0x559b1814a110 2024-01-11T17:59:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.990+0000 7f8cc6ff5700 10 client.25828 handle_command_reply: tid=0 2024-01-11T17:59:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.990+0000 7f8cfd2c6b80 1 client.25828 shutdown 2024-01-11T17:59:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.990+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 msgr2=0x559b180ec520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.990+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 0x559b180ec520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40074a0 tx=0x7f8cd40036a0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.990+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 shutdown_connections 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.990+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559b1814a110 0x559b1814c530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b180ddf20 0x559b180ec520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cc6ff5700 0 client.25828 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b180df5d0 0x559b180eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 1 --2- 172.21.15.7:0/675371341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b180dec10 0x559b180eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 >> 172.21.15.7:0/675371341 conn(0x559b18053740 msgr2=0x559b180e6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 shutdown_connections 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 1 -- 172.21.15.7:0/675371341 wait complete. 2024-01-11T17:59:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.991+0000 7f8cfd2c6b80 20 client.25828 trim_cache size 0 max 16384 2024-01-11T17:59:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:59:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:59:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:59:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8ccc0a1f50 msgr2=0x7f8ccc0a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8ccc0a1f50 0x7f8ccc0a4410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009550 tx=0x7f8cdc006040 comp rx=0 tx=0).stop 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 msgr2=0x7f8ce419da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce419da80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400c970 tx=0x7f8cd400cc80 comp rx=0 tx=0).stop 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 shutdown_connections 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8ccc0a1f50 0x7f8ccc0a4410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ce41015b0 0x7f8ce419da80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ce4103140 0x7f8ce41a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 --2- 172.21.15.7:0/286511863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce41027a0 0x7f8ce419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.993+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 >> 172.21.15.7:0/286511863 conn(0x7f8ce40fcb00 msgr2=0x7f8ce4105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.994+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 shutdown_connections 2024-01-11T17:59:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:27.995+0000 7f8ce95ff700 1 -- 172.21.15.7:0/286511863 wait complete. 2024-01-11T17:59:27.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:27 smithi016 ceph-mon[43195]: pgmap v349: 129 pgs: 129 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 28 MiB/s wr, 934 op/s 2024-01-11T17:59:28.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:27 smithi038 ceph-mon[43582]: pgmap v349: 129 pgs: 129 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 28 MiB/s wr, 934 op/s 2024-01-11T17:59:28.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:27 smithi007 ceph-mon[33361]: pgmap v349: 129 pgs: 129 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 28 MiB/s wr, 934 op/s 2024-01-11T17:59:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 60.0 s, 1257.6 tps, lat 79.992 ms stddev 93.902 2024-01-11T17:59:29.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:29 smithi016 ceph-mon[43195]: pgmap v350: 129 pgs: 129 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 23 MiB/s wr, 737 op/s 2024-01-11T17:59:30.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:29 smithi038 ceph-mon[43582]: pgmap v350: 129 pgs: 129 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 23 MiB/s wr, 737 op/s 2024-01-11T17:59:30.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:29 smithi007 ceph-mon[33361]: pgmap v350: 129 pgs: 129 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 23 MiB/s wr, 737 op/s 2024-01-11T17:59:31.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:31 smithi016 ceph-mon[43195]: pgmap v351: 129 pgs: 129 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 25 MiB/s wr, 842 op/s 2024-01-11T17:59:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:31 smithi038 ceph-mon[43582]: pgmap v351: 129 pgs: 129 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 25 MiB/s wr, 842 op/s 2024-01-11T17:59:32.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:31 smithi007 ceph-mon[33361]: pgmap v351: 129 pgs: 129 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 25 MiB/s wr, 842 op/s 2024-01-11T17:59:33.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:33 smithi016 ceph-mon[43195]: pgmap v352: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 26 MiB/s wr, 876 op/s 2024-01-11T17:59:34.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:33 smithi038 ceph-mon[43582]: pgmap v352: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 26 MiB/s wr, 876 op/s 2024-01-11T17:59:34.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:33 smithi007 ceph-mon[33361]: pgmap v352: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 26 MiB/s wr, 876 op/s 2024-01-11T17:59:34.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 65.0 s, 870.7 tps, lat 87.710 ms stddev 141.442 2024-01-11T17:59:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:35 smithi016 ceph-mon[43195]: pgmap v353: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 29 MiB/s wr, 1.26k op/s 2024-01-11T17:59:35.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:35 smithi038 ceph-mon[43582]: pgmap v353: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 29 MiB/s wr, 1.26k op/s 2024-01-11T17:59:35.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:35 smithi007 ceph-mon[33361]: pgmap v353: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 29 MiB/s wr, 1.26k op/s 2024-01-11T17:59:37.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:37 smithi016 ceph-mon[43195]: pgmap v354: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 36 MiB/s wr, 1.93k op/s 2024-01-11T17:59:38.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:37 smithi038 ceph-mon[43582]: pgmap v354: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 36 MiB/s wr, 1.93k op/s 2024-01-11T17:59:38.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:37 smithi007 ceph-mon[33361]: pgmap v354: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 36 MiB/s wr, 1.93k op/s 2024-01-11T17:59:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 70.0 s, 1373.0 tps, lat 89.554 ms stddev 237.136 2024-01-11T17:59:39.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:39 smithi016 ceph-mon[43195]: pgmap v355: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 1.75k op/s 2024-01-11T17:59:40.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:39 smithi038 ceph-mon[43582]: pgmap v355: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 1.75k op/s 2024-01-11T17:59:40.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:39 smithi007 ceph-mon[33361]: pgmap v355: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 1.75k op/s 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 -- 172.21.15.7:0/3621870925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601027e0 msgr2=0x7fce60102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 --2- 172.21.15.7:0/3621870925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601027e0 0x7fce60102c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce540099c0 tx=0x7fce54009cd0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 -- 172.21.15.7:0/3621870925 shutdown_connections 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 --2- 172.21.15.7:0/3621870925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601015f0 0x7fce60101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 --2- 172.21.15.7:0/3621870925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 0x7fce60109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 --2- 172.21.15.7:0/3621870925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601027e0 0x7fce60102c40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 -- 172.21.15.7:0/3621870925 >> 172.21.15.7:0/3621870925 conn(0x7fce600fcb60 msgr2=0x7fce600fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 -- 172.21.15.7:0/3621870925 shutdown_connections 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.267+0000 7fce66dff700 1 -- 172.21.15.7:0/3621870925 wait complete. 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.269+0000 7fce66dff700 1 Processor -- start 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.269+0000 7fce66dff700 1 -- start start 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.270+0000 7fce66dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601015f0 0x7fce601950d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.270+0000 7fce66dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601027e0 0x7fce60195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.270+0000 7fce66dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 0x7fce6019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.270+0000 7fce66dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6019cc50 con 0x7fce60103180 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.270+0000 7fce66dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6019cd90 con 0x7fce601015f0 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.270+0000 7fce66dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6019cf20 con 0x7fce601027e0 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.271+0000 7fce6539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 0x7fce6019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.271+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601027e0 0x7fce60195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.271+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601027e0 0x7fce60195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54848/0 (socket says 172.21.15.7:54848) 2024-01-11T17:59:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.271+0000 7fce5ffff700 1 -- 172.21.15.7:0/2552510772 learned_addr learned my addr 172.21.15.7:0/2552510772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.271+0000 7fce64b9b700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601015f0 0x7fce601950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.272+0000 7fce6539c700 1 -- 172.21.15.7:0/2552510772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601027e0 msgr2=0x7fce60195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.272+0000 7fce6539c700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601027e0 0x7fce60195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.272+0000 7fce6539c700 1 -- 172.21.15.7:0/2552510772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601015f0 msgr2=0x7fce601950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.272+0000 7fce6539c700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601015f0 0x7fce601950d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.272+0000 7fce6539c700 1 -- 172.21.15.7:0/2552510772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540095f0 con 0x7fce60103180 2024-01-11T17:59:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.274+0000 7fce6539c700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 0x7fce6019c680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fce5000c7a0 tx=0x7fce5000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.275+0000 7fce5dffb700 1 -- 172.21.15.7:0/2552510772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50004a30 con 0x7fce60103180 2024-01-11T17:59:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.275+0000 7fce5dffb700 1 -- 172.21.15.7:0/2552510772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce500054e0 con 0x7fce60103180 2024-01-11T17:59:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.275+0000 7fce5dffb700 1 -- 172.21.15.7:0/2552510772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000eb70 con 0x7fce60103180 2024-01-11T17:59:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.275+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce601a5a10 con 0x7fce60103180 2024-01-11T17:59:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.275+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce601a5ed0 con 0x7fce60103180 2024-01-11T17:59:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.282+0000 7fce7abccb80 1 Processor -- start 2024-01-11T17:59:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.283+0000 7fce7abccb80 1 -- start start 2024-01-11T17:59:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d4c10 0x5629c82d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d55d0 0x5629c82dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce7abccb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c81cb030 con 0x5629c82d4c10 2024-01-11T17:59:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce7abccb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c822eac0 con 0x5629c82d55d0 2024-01-11T17:59:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce7abccb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c822c3e0 con 0x5629c82d3f20 2024-01-11T17:59:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54864/0 (socket says 172.21.15.7:54864) 2024-01-11T17:59:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce5ffff700 1 -- 172.21.15.7:0/1502072927 learned_addr learned my addr 172.21.15.7:0/1502072927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.284+0000 7fce5ffff700 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d55d0 0x5629c82dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.286+0000 7fce64b9b700 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d4c10 0x5629c82d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.286+0000 7fce5ffff700 1 -- 172.21.15.7:0/1502072927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d55d0 msgr2=0x5629c82dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.286+0000 7fce5ffff700 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d55d0 0x5629c82dbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.286+0000 7fce5ffff700 1 -- 172.21.15.7:0/1502072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d4c10 msgr2=0x5629c82d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce5ffff700 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d4c10 0x5629c82d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce5ffff700 1 -- 172.21.15.7:0/1502072927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c809c350 con 0x5629c82d3f20 2024-01-11T17:59:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce64b9b700 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d4c10 0x5629c82d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce5ffff700 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82d4340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce5400ba70 tx=0x7fce5401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a5cf986329b99a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce46ffd700 1 -- 172.21.15.7:0/1502072927 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400bce0 con 0x5629c82d3f20 2024-01-11T17:59:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce46ffd700 1 -- 172.21.15.7:0/1502072927 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5400be40 con 0x5629c82d3f20 2024-01-11T17:59:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.287+0000 7fce46ffd700 1 -- 172.21.15.7:0/1502072927 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400f8e0 con 0x5629c82d3f20 2024-01-11T17:59:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.289+0000 7fce5dffb700 1 -- 172.21.15.7:0/2552510772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fce50004b90 con 0x7fce60103180 2024-01-11T17:59:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.293+0000 7fce7abccb80 1 -- 172.21.15.7:0/1502072927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 msgr2=0x5629c82d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.293+0000 7fce7abccb80 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82d4340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce5400ba70 tx=0x7fce5401a040 comp rx=0 tx=0).stop 2024-01-11T17:59:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 -- 172.21.15.7:0/1502072927 shutdown_connections 2024-01-11T17:59:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82d4340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d55d0 0x5629c82dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 --2- 172.21.15.7:0/1502072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d4c10 0x5629c82d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 -- 172.21.15.7:0/1502072927 >> 172.21.15.7:0/1502072927 conn(0x5629c8249740 msgr2=0x5629c82ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 -- 172.21.15.7:0/1502072927 shutdown_connections 2024-01-11T17:59:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.294+0000 7fce7abccb80 1 -- 172.21.15.7:0/1502072927 wait complete. 2024-01-11T17:59:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.295+0000 7fce5dffb700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fce480a1e50 0x7fce480a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.295+0000 7fce7abccb80 1 -- start start 2024-01-11T17:59:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.295+0000 7fce7abccb80 1 Processor -- start 2024-01-11T17:59:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.296+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 0x5629c82dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.296+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d4c10 0x5629c82dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.296+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce64b9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 0x5629c82dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce7abccb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c8227cd0 con 0x5629c82d55d0 2024-01-11T17:59:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce7abccb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c8196340 con 0x5629c82d3f20 2024-01-11T17:59:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce7abccb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c8195fe0 con 0x5629c82d4c10 2024-01-11T17:59:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce6539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce64b9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 0x5629c82dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35828/0 (socket says 172.21.15.7:35828) 2024-01-11T17:59:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce64b9b700 1 -- 172.21.15.7:0/929086667 learned_addr learned my addr 172.21.15.7:0/929086667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce7abccb80 10 client.0 _ll_register_callbacks cb 0x5629c81c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce64b9b700 1 -- 172.21.15.7:0/929086667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d4c10 msgr2=0x5629c82dde60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce64b9b700 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d4c10 0x5629c82dde60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.297+0000 7fce64b9b700 1 -- 172.21.15.7:0/929086667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 msgr2=0x5629c82e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.298+0000 7fce5dffb700 1 -- 172.21.15.7:0/2552510772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fce500cd980 con 0x7fce60103180 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.298+0000 7fce64b9b700 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.298+0000 7fce64b9b700 1 -- 172.21.15.7:0/929086667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce58008cd0 con 0x5629c82d3f20 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.298+0000 7fce6539c700 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.298+0000 7fce64b9b700 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 0x5629c82dd920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce58008fc0 tx=0x7fce580104b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ce7db0cc03ab4d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.298+0000 7fce3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:59:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.299+0000 7fce5ffff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fce480a1e50 0x7fce480a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce5ffff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fce480a1e50 0x7fce480a4310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c010 tx=0x7fce54009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce3ffff700 1 -- 172.21.15.7:0/929086667 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58010cd0 con 0x5629c82d3f20 2024-01-11T17:59:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce3ffff700 1 -- 172.21.15.7:0/929086667 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fce58010e30 con 0x5629c82d3f20 2024-01-11T17:59:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629c82c6d60 con 0x5629c82d3f20 2024-01-11T17:59:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce3ffff700 1 -- 172.21.15.7:0/929086667 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce58026480 con 0x5629c82d3f20 2024-01-11T17:59:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce7abccb80 10 client.25808 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.301+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fce54020040 con 0x5629c82d3f20 2024-01-11T17:59:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.302+0000 7fce3ffff700 1 -- 172.21.15.7:0/929086667 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629c82c6d60 con 0x5629c82d3f20 2024-01-11T17:59:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.302+0000 7fce3ffff700 1 -- 172.21.15.7:0/929086667 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fce58025790 con 0x5629c82d3f20 2024-01-11T17:59:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.302+0000 7fce7abccb80 10 client.25808 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.302+0000 7fce7abccb80 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.302+0000 7fce7abccb80 20 client.25808 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.303+0000 7fce7abccb80 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c82e9870 0x5629c82ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.303+0000 7fce7abccb80 4 client.25808 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.303+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629c8220b60 con 0x5629c82e9870 2024-01-11T17:59:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.303+0000 7fce5ffff700 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c82e9870 0x5629c82ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.305+0000 7fce5ffff700 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c82e9870 0x5629c82ebc90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3aba16cc36aef4 server_cookie=58706abd971f3497 in_seq=0 out_seq=0 2024-01-11T17:59:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.305+0000 7fce3ffff700 10 client.25808 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.306+0000 7fce3ffff700 1 -- 172.21.15.7:0/929086667 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629c8220b60 con 0x5629c82e9870 2024-01-11T17:59:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.306+0000 7fce3ffff700 10 client.25808 handle_command_reply: tid=0 2024-01-11T17:59:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.306+0000 7fce7abccb80 1 client.25808 shutdown 2024-01-11T17:59:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 msgr2=0x5629c82dd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 0x5629c82dd920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce58008fc0 tx=0x7fce580104b0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 shutdown_connections 2024-01-11T17:59:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c82e9870 0x5629c82ebc90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d3f20 0x5629c82dd920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 --2- 172.21.15.7:0/929086667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d4c10 0x5629c82dde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce3ffff700 0 client.25808 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.308+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 >> 172.21.15.7:0/929086667 conn(0x5629c8249740 msgr2=0x5629c82cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.309+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 shutdown_connections 2024-01-11T17:59:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.310+0000 7fce7abccb80 1 -- 172.21.15.7:0/929086667 wait complete. 2024-01-11T17:59:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.310+0000 7fce7abccb80 20 client.25808 trim_cache size 0 max 16384 2024-01-11T17:59:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.348+0000 7fce7abccb80 1 Processor -- start 2024-01-11T17:59:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.348+0000 7fce7abccb80 1 -- start start 2024-01-11T17:59:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce7abccb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c809e640 con 0x5629c82d55d0 2024-01-11T17:59:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce7abccb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c82ec050 con 0x5629c82d4c10 2024-01-11T17:59:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce7abccb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c82d00a0 con 0x5629c82d3f20 2024-01-11T17:59:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce64b9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce64b9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82e5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54872/0 (socket says 172.21.15.7:54872) 2024-01-11T17:59:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.349+0000 7fce64b9b700 1 -- 172.21.15.7:0/2641907011 learned_addr learned my addr 172.21.15.7:0/2641907011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce6539c700 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce5ffff700 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce64b9b700 1 -- 172.21.15.7:0/2641907011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 msgr2=0x5629c82e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce64b9b700 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce64b9b700 1 -- 172.21.15.7:0/2641907011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 msgr2=0x5629c82ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce64b9b700 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.350+0000 7fce64b9b700 1 -- 172.21.15.7:0/2641907011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c8048b00 con 0x5629c82d3f20 2024-01-11T17:59:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.351+0000 7fce5ffff700 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T17:59:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.351+0000 7fce64b9b700 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82e5550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce5802e3c0 tx=0x7fce580109d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=decda52922f96f87 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.352+0000 7fce467fc700 1 -- 172.21.15.7:0/2641907011 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58011010 con 0x5629c82d3f20 2024-01-11T17:59:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.352+0000 7fce467fc700 1 -- 172.21.15.7:0/2641907011 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce580258a0 con 0x5629c82d3f20 2024-01-11T17:59:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.352+0000 7fce467fc700 1 -- 172.21.15.7:0/2641907011 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5802e740 con 0x5629c82d3f20 2024-01-11T17:59:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.354+0000 7fce7abccb80 1 -- 172.21.15.7:0/2641907011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 msgr2=0x5629c82e5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.354+0000 7fce7abccb80 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82e5550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce5802e3c0 tx=0x7fce580109d0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 -- 172.21.15.7:0/2641907011 shutdown_connections 2024-01-11T17:59:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d3f20 0x5629c82e5550 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d55d0 0x5629c82ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 --2- 172.21.15.7:0/2641907011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 -- 172.21.15.7:0/2641907011 >> 172.21.15.7:0/2641907011 conn(0x5629c8249740 msgr2=0x5629c82dbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 -- 172.21.15.7:0/2641907011 shutdown_connections 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.355+0000 7fce7abccb80 1 -- 172.21.15.7:0/2641907011 wait complete. 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.356+0000 7fce7abccb80 1 -- start start 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.356+0000 7fce7abccb80 1 Processor -- start 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d3f20 0x5629c82e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d55d0 0x5629c82e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c82e4400 con 0x5629c82d3f20 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c82e4570 con 0x5629c82d4c10 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c82e4730 con 0x5629c82d55d0 2024-01-11T17:59:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.357+0000 7fce7abccb80 10 client.0 _ll_register_callbacks cb 0x5629c81c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.358+0000 7fce64b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d3f20 0x5629c82e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.358+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.358+0000 7fce5ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35844/0 (socket says 172.21.15.7:35844) 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.358+0000 7fce5ffff700 1 -- 172.21.15.7:0/3746846895 learned_addr learned my addr 172.21.15.7:0/3746846895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.359+0000 7fce5ffff700 1 -- 172.21.15.7:0/3746846895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d55d0 msgr2=0x5629c82e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.359+0000 7fce5ffff700 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d55d0 0x5629c82e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.359+0000 7fce5ffff700 1 -- 172.21.15.7:0/3746846895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d3f20 msgr2=0x5629c82e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.359+0000 7fce5ffff700 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d3f20 0x5629c82e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.359+0000 7fce5ffff700 1 -- 172.21.15.7:0/3746846895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629c809c350 con 0x5629c82d4c10 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.359+0000 7fce64b9b700 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d3f20 0x5629c82e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.360+0000 7fce5ffff700 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e2f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce5400ba70 tx=0x7fce54003fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192c2ff1634ecbc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.360+0000 7fce3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.361+0000 7fce3ffff700 1 -- 172.21.15.7:0/3746846895 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5402d070 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.361+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce5400be40 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.361+0000 7fce3ffff700 1 -- 172.21.15.7:0/3746846895 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fce5403a040 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.362+0000 7fce3ffff700 1 -- 172.21.15.7:0/3746846895 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce54004100 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.362+0000 7fce7abccb80 10 client.25814 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.362+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fce54020040 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.362+0000 7fce3ffff700 1 -- 172.21.15.7:0/3746846895 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5400be40 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce3ffff700 1 -- 172.21.15.7:0/3746846895 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fce5403d070 con 0x5629c82d4c10 2024-01-11T17:59:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce7abccb80 10 client.25814 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce7abccb80 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce7abccb80 20 client.25814 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce7abccb80 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c83407a0 0x5629c8342bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce7abccb80 4 client.25814 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629c8220880 con 0x5629c83407a0 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.363+0000 7fce64b9b700 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c83407a0 0x5629c8342bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.364+0000 7fce3ffff700 10 client.25814 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.364+0000 7fce64b9b700 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c83407a0 0x5629c8342bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e3b52785896039 server_cookie=12aac482c7ee7a9 in_seq=0 out_seq=0 2024-01-11T17:59:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.380+0000 7fce3ffff700 1 -- 172.21.15.7:0/3746846895 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629c8220880 con 0x5629c83407a0 2024-01-11T17:59:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.380+0000 7fce3ffff700 10 client.25814 handle_command_reply: tid=0 2024-01-11T17:59:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.380+0000 7fce7abccb80 1 client.25814 shutdown 2024-01-11T17:59:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 msgr2=0x5629c82e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e2f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce5400ba70 tx=0x7fce54003fe0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 shutdown_connections 2024-01-11T17:59:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629c83407a0 0x5629c8342bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629c82d55d0 0x5629c82e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629c82d3f20 0x5629c82e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 --2- 172.21.15.7:0/3746846895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629c82d4c10 0x5629c82e2f90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce3ffff700 0 client.25814 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.381+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 >> 172.21.15.7:0/3746846895 conn(0x5629c8249740 msgr2=0x5629c82dcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.382+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 shutdown_connections 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.382+0000 7fce7abccb80 1 -- 172.21.15.7:0/3746846895 wait complete. 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.382+0000 7fce7abccb80 20 client.25814 trim_cache size 0 max 16384 2024-01-11T17:59:41.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.384+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fce480a1e50 msgr2=0x7fce480a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.384+0000 7fce66dff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fce480a1e50 0x7fce480a4310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c010 tx=0x7fce54009f10 comp rx=0 tx=0).stop 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.384+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 msgr2=0x7fce6019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.384+0000 7fce66dff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 0x7fce6019c680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fce5000c7a0 tx=0x7fce5000cab0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 shutdown_connections 2024-01-11T17:59:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fce480a1e50 0x7fce480a4310 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce60103180 0x7fce6019c680 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce601015f0 0x7fce601950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 --2- 172.21.15.7:0/2552510772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce601027e0 0x7fce60195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 >> 172.21.15.7:0/2552510772 conn(0x7fce600fcb60 msgr2=0x7fce60105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 shutdown_connections 2024-01-11T17:59:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:41.385+0000 7fce66dff700 1 -- 172.21.15.7:0/2552510772 wait complete. 2024-01-11T17:59:41.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:41 smithi016 ceph-mon[43195]: pgmap v356: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2024-01-11T17:59:42.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:41 smithi038 ceph-mon[43582]: pgmap v356: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2024-01-11T17:59:42.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:41 smithi007 ceph-mon[33361]: pgmap v356: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 1.84k op/s 2024-01-11T17:59:43.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:43 smithi016 ceph-mon[43195]: pgmap v357: 129 pgs: 129 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 32 MiB/s wr, 1.83k op/s 2024-01-11T17:59:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:43 smithi038 ceph-mon[43582]: pgmap v357: 129 pgs: 129 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 32 MiB/s wr, 1.83k op/s 2024-01-11T17:59:44.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:43 smithi007 ceph-mon[33361]: pgmap v357: 129 pgs: 129 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 32 MiB/s wr, 1.83k op/s 2024-01-11T17:59:44.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 75.0 s, 1795.9 tps, lat 55.927 ms stddev 66.931 2024-01-11T17:59:45.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:45 smithi016 ceph-mon[43195]: pgmap v358: 129 pgs: 129 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 29 MiB/s wr, 1.75k op/s 2024-01-11T17:59:45.779 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:45 smithi038 ceph-mon[43582]: pgmap v358: 129 pgs: 129 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 29 MiB/s wr, 1.75k op/s 2024-01-11T17:59:45.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:45 smithi007 ceph-mon[33361]: pgmap v358: 129 pgs: 129 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 29 MiB/s wr, 1.75k op/s 2024-01-11T17:59:47.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:47 smithi016 ceph-mon[43195]: pgmap v359: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-01-11T17:59:48.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:47 smithi038 ceph-mon[43582]: pgmap v359: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-01-11T17:59:48.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:47 smithi007 ceph-mon[33361]: pgmap v359: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-01-11T17:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 80.0 s, 1453.7 tps, lat 69.074 ms stddev 91.214 2024-01-11T17:59:49.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:49 smithi016 ceph-mon[43195]: pgmap v360: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-01-11T17:59:50.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:49 smithi038 ceph-mon[43582]: pgmap v360: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-01-11T17:59:50.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:49 smithi007 ceph-mon[33361]: pgmap v360: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-01-11T17:59:51.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:51 smithi016 ceph-mon[43195]: pgmap v361: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 616 op/s 2024-01-11T17:59:52.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:51 smithi038 ceph-mon[43582]: pgmap v361: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 616 op/s 2024-01-11T17:59:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:51 smithi007 ceph-mon[33361]: pgmap v361: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 616 op/s 2024-01-11T17:59:53.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:53 smithi016 ceph-mon[43195]: pgmap v362: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 635 op/s 2024-01-11T17:59:54.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:53 smithi038 ceph-mon[43582]: pgmap v362: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 635 op/s 2024-01-11T17:59:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:53 smithi007 ceph-mon[33361]: pgmap v362: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 635 op/s 2024-01-11T17:59:54.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 85.0 s, 1695.8 tps, lat 58.424 ms stddev 81.102 2024-01-11T17:59:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 -- 172.21.15.7:0/3186721795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f840ffe10 msgr2=0x7f5f84102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 --2- 172.21.15.7:0/3186721795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f840ffe10 0x7f5f84102240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009990 tx=0x7f5f74009ca0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 -- 172.21.15.7:0/3186721795 shutdown_connections 2024-01-11T17:59:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 --2- 172.21.15.7:0/3186721795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f840ffe10 0x7f5f84102240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 --2- 172.21.15.7:0/3186721795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f84105150 0x7f5f841055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 --2- 172.21.15.7:0/3186721795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f84104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.646+0000 7f5f88fff700 1 -- 172.21.15.7:0/3186721795 >> 172.21.15.7:0/3186721795 conn(0x7f5f840fba30 msgr2=0x7f5f840fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.647+0000 7f5f88fff700 1 -- 172.21.15.7:0/3186721795 shutdown_connections 2024-01-11T17:59:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.647+0000 7f5f88fff700 1 -- 172.21.15.7:0/3186721795 wait complete. 2024-01-11T17:59:54.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.648+0000 7f5f88fff700 1 Processor -- start 2024-01-11T17:59:54.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.649+0000 7f5f88fff700 1 -- start start 2024-01-11T17:59:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.649+0000 7f5f88fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f840ffe10 0x7f5f8419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.649+0000 7f5f88fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f8419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.649+0000 7f5f79d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f8419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.649+0000 7f5f79d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f8419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38224/0 (socket says 172.21.15.7:38224) 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f88fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f84105150 0x7f5f841a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f79d9b700 1 -- 172.21.15.7:0/2271532146 learned_addr learned my addr 172.21.15.7:0/2271532146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841a55d0 con 0x7f5f840ffe10 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841a5710 con 0x7f5f84102780 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841a58a0 con 0x7f5f84105150 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f8259c700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f840ffe10 0x7f5f8419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f79d9b700 1 -- 172.21.15.7:0/2271532146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f84105150 msgr2=0x7f5f841a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f84105150 0x7f5f841a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f79d9b700 1 -- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f840ffe10 msgr2=0x7f5f8419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f840ffe10 0x7f5f8419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f79d9b700 1 -- 172.21.15.7:0/2271532146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740095f0 con 0x7f5f84102780 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.650+0000 7f5f8259c700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f840ffe10 0x7f5f8419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.651+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f8419df90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00e9a0 tx=0x7f5f6c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.651+0000 7f5f7bfff700 1 -- 172.21.15.7:0/2271532146 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c009790 con 0x7f5f84102780 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.651+0000 7f5f7bfff700 1 -- 172.21.15.7:0/2271532146 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f6c00cbd0 con 0x7f5f84102780 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.651+0000 7f5f7bfff700 1 -- 172.21.15.7:0/2271532146 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c018680 con 0x7f5f84102780 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.651+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f841a5b80 con 0x7f5f84102780 2024-01-11T17:59:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.651+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f841a6040 con 0x7f5f84102780 2024-01-11T17:59:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.658+0000 7f5f7bfff700 1 -- 172.21.15.7:0/2271532146 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f5f6c00cd40 con 0x7f5f84102780 2024-01-11T17:59:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.660+0000 7f5f9ccf4b80 1 Processor -- start 2024-01-11T17:59:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.661+0000 7f5f9ccf4b80 1 -- start start 2024-01-11T17:59:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.661+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.661+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.661+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b3c0030 con 0x56160b4c9c10 2024-01-11T17:59:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b423ac0 con 0x56160b4ca5d0 2024-01-11T17:59:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b4213e0 con 0x56160b4c8f20 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f7bfff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5f700a1f90 0x7f5f700a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f7bfff700 1 -- 172.21.15.7:0/2271532146 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f6c014070 con 0x7f5f84102780 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f82d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f82d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59614/0 (socket says 172.21.15.7:59614) 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.662+0000 7f5f82d9d700 1 -- 172.21.15.7:0/820575836 learned_addr learned my addr 172.21.15.7:0/820575836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f82d9d700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5f700a1f90 0x7f5f700a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f8259c700 1 -- 172.21.15.7:0/820575836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 msgr2=0x56160b4c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f8259c700 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f8259c700 1 -- 172.21.15.7:0/820575836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 msgr2=0x56160b4d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f8259c700 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.663+0000 7f5f8259c700 1 -- 172.21.15.7:0/820575836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160b291350 con 0x56160b4c9c10 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.664+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.664+0000 7f5f82d9d700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5f700a1f90 0x7f5f700a4450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c010c10 tx=0x7f5f7c010160 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.665+0000 7f5f8259c700 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4ca090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400b2e0 tx=0x7f5f74009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c4fc93e433533 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.665+0000 7f5f78d99700 1 -- 172.21.15.7:0/820575836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7401e070 con 0x56160b4c9c10 2024-01-11T17:59:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.665+0000 7f5f78d99700 1 -- 172.21.15.7:0/820575836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f7400bd10 con 0x56160b4c9c10 2024-01-11T17:59:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.665+0000 7f5f78d99700 1 -- 172.21.15.7:0/820575836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74005030 con 0x56160b4c9c10 2024-01-11T17:59:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/820575836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 msgr2=0x56160b4ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4ca090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400b2e0 tx=0x7f5f74009510 comp rx=0 tx=0).stop 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/820575836 shutdown_connections 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/820575836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4ca090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/820575836 >> 172.21.15.7:0/820575836 conn(0x56160b43e740 msgr2=0x56160b4c3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/820575836 shutdown_connections 2024-01-11T17:59:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.666+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/820575836 wait complete. 2024-01-11T17:59:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.667+0000 7f5f9ccf4b80 1 -- start start 2024-01-11T17:59:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.667+0000 7f5f9ccf4b80 1 Processor -- start 2024-01-11T17:59:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4c8f20 0x56160b4d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 0x56160b4d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4ca5d0 0x56160b4d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b41ccd0 con 0x56160b4ca5d0 2024-01-11T17:59:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b38b340 con 0x56160b4c8f20 2024-01-11T17:59:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b38afe0 con 0x56160b4c9c10 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.669+0000 7f5f9ccf4b80 10 client.0 _ll_register_callbacks cb 0x56160b3bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4c8f20 0x56160b4d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4ca5d0 0x56160b4d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f79d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 0x56160b4d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4c8f20 0x56160b4d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38254/0 (socket says 172.21.15.7:38254) 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f79d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 0x56160b4d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59618/0 (socket says 172.21.15.7:59618) 2024-01-11T17:59:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f8259c700 1 -- 172.21.15.7:0/3088073108 learned_addr learned my addr 172.21.15.7:0/3088073108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f79d9b700 1 -- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4c8f20 msgr2=0x56160b4d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4c8f20 0x56160b4d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.670+0000 7f5f79d9b700 1 -- 172.21.15.7:0/3088073108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4ca5d0 msgr2=0x56160b4d9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4ca5d0 0x56160b4d9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f79d9b700 1 -- 172.21.15.7:0/3088073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f84106100 con 0x56160b4c9c10 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f82d9d700 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4ca5d0 0x56160b4d9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 0x56160b4d2df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0cb7a0 tx=0x7f5f6c09af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed63c5f22d25d397 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f69ffb700 1 -- 172.21.15.7:0/3088073108 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c09eb50 con 0x56160b4c9c10 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f69ffb700 1 -- 172.21.15.7:0/3088073108 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f6c0a2d90 con 0x56160b4c9c10 2024-01-11T17:59:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.671+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56160b4bbd60 con 0x56160b4c9c10 2024-01-11T17:59:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f69ffb700 1 -- 172.21.15.7:0/3088073108 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f6c0850e0 con 0x56160b4c9c10 2024-01-11T17:59:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f9ccf4b80 10 client.25846 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56160b23db00 con 0x56160b4c9c10 2024-01-11T17:59:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f69ffb700 1 -- 172.21.15.7:0/3088073108 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f6c0b7030 con 0x56160b4c9c10 2024-01-11T17:59:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f69ffb700 1 -- 172.21.15.7:0/3088073108 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5f6c0ba140 con 0x56160b4c9c10 2024-01-11T17:59:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f9ccf4b80 10 client.25846 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f9ccf4b80 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.672+0000 7f5f9ccf4b80 20 client.25846 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.673+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b4de710 0x56160b4e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.673+0000 7f5f9ccf4b80 4 client.25846 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T17:59:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.673+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56160b415b60 con 0x56160b4de710 2024-01-11T17:59:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.674+0000 7f5f8259c700 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b4de710 0x56160b4e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.675+0000 7f5f8259c700 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b4de710 0x56160b4e0b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e07c018cd8a71083 server_cookie=41a1ed3e553697d7 in_seq=0 out_seq=0 2024-01-11T17:59:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.675+0000 7f5f69ffb700 10 client.25846 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.675+0000 7f5f69ffb700 1 -- 172.21.15.7:0/3088073108 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56160b415b60 con 0x56160b4de710 2024-01-11T17:59:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.675+0000 7f5f69ffb700 10 client.25846 handle_command_reply: tid=0 2024-01-11T17:59:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 client.25846 shutdown 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 msgr2=0x56160b4d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 0x56160b4d2df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0cb7a0 tx=0x7f5f6c09af40 comp rx=0 tx=0).stop 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 shutdown_connections 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b4de710 0x56160b4e0b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4ca5d0 0x56160b4d9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4c8f20 0x56160b4d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3088073108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c9c10 0x56160b4d2df0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 >> 172.21.15.7:0/3088073108 conn(0x56160b43e740 msgr2=0x56160b4c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f69ffb700 0 client.25846 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 shutdown_connections 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.676+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3088073108 wait complete. 2024-01-11T17:59:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.677+0000 7f5f9ccf4b80 20 client.25846 trim_cache size 0 max 16384 2024-01-11T17:59:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 Processor -- start 2024-01-11T17:59:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 -- start start 2024-01-11T17:59:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b293640 con 0x56160b4c9c10 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b4d08a0 con 0x56160b4ca5d0 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.704+0000 7f5f9ccf4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b4c5270 con 0x56160b4c8f20 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4dac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59622/0 (socket says 172.21.15.7:59622) 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 -- 172.21.15.7:0/3146797295 learned_addr learned my addr 172.21.15.7:0/3146797295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f82d9d700 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 -- 172.21.15.7:0/3146797295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 msgr2=0x56160b4d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 -- 172.21.15.7:0/3146797295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 msgr2=0x56160b4d9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.705+0000 7f5f8259c700 1 -- 172.21.15.7:0/3146797295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160b3de1d0 con 0x56160b4c8f20 2024-01-11T17:59:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.706+0000 7f5f82d9d700 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T17:59:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.706+0000 7f5f8259c700 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4dac10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009990 tx=0x7f5f7400b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6adb0c70faead47 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.706+0000 7f5f6bfff700 1 -- 172.21.15.7:0/3146797295 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7401e040 con 0x56160b4c8f20 2024-01-11T17:59:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.706+0000 7f5f6bfff700 1 -- 172.21.15.7:0/3146797295 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f7401f040 con 0x56160b4c8f20 2024-01-11T17:59:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.706+0000 7f5f6bfff700 1 -- 172.21.15.7:0/3146797295 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400f750 con 0x56160b4c8f20 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.707+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3146797295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 msgr2=0x56160b4dac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4dac10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009990 tx=0x7f5f7400b750 comp rx=0 tx=0).stop 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3146797295 shutdown_connections 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4dac10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b4d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/3146797295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3146797295 >> 172.21.15.7:0/3146797295 conn(0x56160b43e740 msgr2=0x56160b4bf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.708+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3146797295 shutdown_connections 2024-01-11T17:59:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.709+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/3146797295 wait complete. 2024-01-11T17:59:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.709+0000 7f5f9ccf4b80 1 -- start start 2024-01-11T17:59:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.709+0000 7f5f9ccf4b80 1 Processor -- start 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f9ccf4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b554c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f8259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4d4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59632/0 (socket says 172.21.15.7:59632) 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f8259c700 1 -- 172.21.15.7:0/842800396 learned_addr learned my addr 172.21.15.7:0/842800396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b555150 con 0x56160b4c9c10 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b555290 con 0x56160b4ca5d0 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.710+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160b555420 con 0x56160b4c8f20 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f82d9d700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b554c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f9ccf4b80 10 client.0 _ll_register_callbacks cb 0x56160b3bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f8259c700 1 -- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 msgr2=0x56160b554c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f8259c700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b554c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f8259c700 1 -- 172.21.15.7:0/842800396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 msgr2=0x56160b4d5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f8259c700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f8259c700 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56160b291350 con 0x56160b4c8f20 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.711+0000 7f5f8259c700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4d4fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f7402fa60 tx=0x7f5f7400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8320702880c77741 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f69ffb700 1 -- 172.21.15.7:0/842800396 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7401e070 con 0x56160b4c8f20 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f69ffb700 1 -- 172.21.15.7:0/842800396 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5f7402f2c0 con 0x56160b4c8f20 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f7400bd10 con 0x56160b4c8f20 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f69ffb700 1 -- 172.21.15.7:0/842800396 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f7402f480 con 0x56160b4c8f20 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f9ccf4b80 10 client.25852 fetch_fsmap learned FSMap version 29 2024-01-11T17:59:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.712+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56160b295c70 con 0x56160b4c8f20 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.713+0000 7f5f69ffb700 1 -- 172.21.15.7:0/842800396 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f7400bd10 con 0x56160b4c8f20 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.713+0000 7f5f69ffb700 1 -- 172.21.15.7:0/842800396 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5f74037060 con 0x56160b4c8f20 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.713+0000 7f5f9ccf4b80 10 client.25852 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.713+0000 7f5f9ccf4b80 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.713+0000 7f5f9ccf4b80 20 client.25852 populate_metadata read hostname 'smithi007' 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.713+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b556860 0x56160b558c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T17:59:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.714+0000 7f5f9ccf4b80 4 client.25852 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T17:59:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.714+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56160b415880 con 0x56160b556860 2024-01-11T17:59:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.715+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b556860 0x56160b558c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T17:59:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.716+0000 7f5f79d9b700 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b556860 0x56160b558c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e112758784d608 server_cookie=7800bb4f1011552c in_seq=0 out_seq=0 2024-01-11T17:59:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.716+0000 7f5f69ffb700 10 client.25852 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f69ffb700 1 -- 172.21.15.7:0/842800396 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56160b415880 con 0x56160b556860 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f69ffb700 10 client.25852 handle_command_reply: tid=0 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 client.25852 shutdown 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 msgr2=0x56160b4d4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4d4fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f7402fa60 tx=0x7f5f7400bb70 comp rx=0 tx=0).stop 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 shutdown_connections 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56160b556860 0x56160b558c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56160b4c8f20 0x56160b4d4fc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56160b4ca5d0 0x56160b554c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f69ffb700 0 client.25852 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 --2- 172.21.15.7:0/842800396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56160b4c9c10 0x56160b4d5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 >> 172.21.15.7:0/842800396 conn(0x56160b43e740 msgr2=0x56160b4d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 shutdown_connections 2024-01-11T17:59:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.790+0000 7f5f9ccf4b80 1 -- 172.21.15.7:0/842800396 wait complete. 2024-01-11T17:59:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.791+0000 7f5f9ccf4b80 20 client.25852 trim_cache size 0 max 16384 2024-01-11T17:59:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T17:59:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T17:59:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T17:59:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.791+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5f700a1f90 msgr2=0x7f5f700a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.791+0000 7f5f88fff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5f700a1f90 0x7f5f700a4450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c010c10 tx=0x7f5f7c010160 comp rx=0 tx=0).stop 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.792+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 msgr2=0x7f5f8419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.792+0000 7f5f88fff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f8419df90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00e9a0 tx=0x7f5f6c00ed60 comp rx=0 tx=0).stop 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 shutdown_connections 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5f700a1f90 0x7f5f700a4450 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f84105150 0x7f5f841a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f840ffe10 0x7f5f8419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 --2- 172.21.15.7:0/2271532146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f84102780 0x7f5f8419df90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T17:59:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 >> 172.21.15.7:0/2271532146 conn(0x7f5f840fba30 msgr2=0x7f5f84103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T17:59:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.793+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 shutdown_connections 2024-01-11T17:59:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T17:59:54.794+0000 7f5f88fff700 1 -- 172.21.15.7:0/2271532146 wait complete. 2024-01-11T17:59:55.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:55 smithi016 ceph-mon[43195]: pgmap v363: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 645 op/s 2024-01-11T17:59:55.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:55 smithi038 ceph-mon[43582]: pgmap v363: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 645 op/s 2024-01-11T17:59:55.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:55 smithi007 ceph-mon[33361]: pgmap v363: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 645 op/s 2024-01-11T17:59:57.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:57 smithi016 ceph-mon[43195]: pgmap v364: 129 pgs: 129 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 709 op/s 2024-01-11T17:59:58.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:57 smithi038 ceph-mon[43582]: pgmap v364: 129 pgs: 129 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 709 op/s 2024-01-11T17:59:58.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:57 smithi007 ceph-mon[33361]: pgmap v364: 129 pgs: 129 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 709 op/s 2024-01-11T17:59:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 90.0 s, 1401.8 tps, lat 70.349 ms stddev 79.680 2024-01-11T17:59:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 17:59:59 smithi016 ceph-mon[43195]: pgmap v365: 129 pgs: 129 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.6 MiB/s wr, 583 op/s 2024-01-11T17:59:59.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 17:59:59 smithi007 ceph-mon[33361]: pgmap v365: 129 pgs: 129 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.6 MiB/s wr, 583 op/s 2024-01-11T18:00:00.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 17:59:59 smithi038 ceph-mon[43582]: pgmap v365: 129 pgs: 129 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.6 MiB/s wr, 583 op/s 2024-01-11T18:00:00.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T17:59:59.999+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:00:00.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T17:59:59.999+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-01-11T18:00:00.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T17:59:59.999+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : mds.i(mds.2): Metadata damage detected 2024-01-11T18:00:00.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T17:59:59.999+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : mds.a(mds.1): Metadata damage detected 2024-01-11T18:00:00.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:00 smithi016 ceph-mon[43195]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:00:00.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:00 smithi016 ceph-mon[43195]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-01-11T18:00:00.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:00 smithi016 ceph-mon[43195]: mds.i(mds.2): Metadata damage detected 2024-01-11T18:00:00.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:00 smithi016 ceph-mon[43195]: mds.a(mds.1): Metadata damage detected 2024-01-11T18:00:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:00 smithi038 ceph-mon[43582]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:00:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:00 smithi038 ceph-mon[43582]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-01-11T18:00:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:00 smithi038 ceph-mon[43582]: mds.i(mds.2): Metadata damage detected 2024-01-11T18:00:01.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:00 smithi038 ceph-mon[43582]: mds.a(mds.1): Metadata damage detected 2024-01-11T18:00:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-mon[33361]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:00:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-mon[33361]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-01-11T18:00:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-mon[33361]: mds.i(mds.2): Metadata damage detected 2024-01-11T18:00:01.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:00 smithi007 ceph-mon[33361]: mds.a(mds.1): Metadata damage detected 2024-01-11T18:00:01.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:01 smithi016 ceph-mon[43195]: pgmap v366: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-01-11T18:00:02.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:01 smithi038 ceph-mon[43582]: pgmap v366: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-01-11T18:00:02.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:01 smithi007 ceph-mon[33361]: pgmap v366: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-01-11T18:00:04.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:03 smithi038 ceph-mon[43582]: pgmap v367: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 695 op/s 2024-01-11T18:00:04.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:03 smithi007 ceph-mon[33361]: pgmap v367: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 695 op/s 2024-01-11T18:00:04.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 95.0 s, 1510.4 tps, lat 67.170 ms stddev 82.072 2024-01-11T18:00:04.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:03 smithi016 ceph-mon[43195]: pgmap v367: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 695 op/s 2024-01-11T18:00:05.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:05 smithi016 ceph-mon[43195]: pgmap v368: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-01-11T18:00:05.783 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:05 smithi038 ceph-mon[43582]: pgmap v368: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-01-11T18:00:05.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:05 smithi007 ceph-mon[33361]: pgmap v368: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-01-11T18:00:07.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:07 smithi016 ceph-mon[43195]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 718 op/s 2024-01-11T18:00:08.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:07 smithi038 ceph-mon[43582]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 718 op/s 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.062+0000 7f792b9ff700 1 -- 172.21.15.7:0/453219484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 msgr2=0x7f7924102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.062+0000 7f792b9ff700 1 --2- 172.21.15.7:0/453219484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924102c20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79200099c0 tx=0x7f7920009cd0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 -- 172.21.15.7:0/453219484 shutdown_connections 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 --2- 172.21.15.7:0/453219484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7924103160 0x7f7924109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 --2- 172.21.15.7:0/453219484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79241015d0 0x7f79241019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 --2- 172.21.15.7:0/453219484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924102c20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 -- 172.21.15.7:0/453219484 >> 172.21.15.7:0/453219484 conn(0x7f79240fcb60 msgr2=0x7f79240fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 -- 172.21.15.7:0/453219484 shutdown_connections 2024-01-11T18:00:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.063+0000 7f792b9ff700 1 -- 172.21.15.7:0/453219484 wait complete. 2024-01-11T18:00:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.065+0000 7f792b9ff700 1 Processor -- start 2024-01-11T18:00:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.066+0000 7f792b9ff700 1 -- start start 2024-01-11T18:00:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.067+0000 7f792b9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79241015d0 0x7f79240753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.067+0000 7f792b9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.067+0000 7f792b9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7924103160 0x7f7924073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.067+0000 7f792b9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79240746e0 con 0x7f7924103160 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.067+0000 7f792b9ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924074850 con 0x7f79241027c0 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.067+0000 7f792b9ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79241a1540 con 0x7f79241015d0 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7929f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7924103160 0x7f7924073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7929f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7924103160 0x7f7924073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52310/0 (socket says 172.21.15.7:52310) 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46622/0 (socket says 172.21.15.7:46622) 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7929f9c700 1 -- 172.21.15.7:0/3076138928 learned_addr learned my addr 172.21.15.7:0/3076138928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f792979b700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79241015d0 0x7f79240753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 -- 172.21.15.7:0/3076138928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79241015d0 msgr2=0x7f79240753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79241015d0 0x7f79240753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 -- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7924103160 msgr2=0x7f7924073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7924103160 0x7f7924073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.068+0000 7f7928f9a700 1 -- 172.21.15.7:0/3076138928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200095f0 con 0x7f79241027c0 2024-01-11T18:00:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.069+0000 7f7928f9a700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924073a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7920009f50 tx=0x7f7920005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.069+0000 7f791a7fc700 1 -- 172.21.15.7:0/3076138928 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792001d070 con 0x7f79241027c0 2024-01-11T18:00:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.069+0000 7f791a7fc700 1 -- 172.21.15.7:0/3076138928 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7920003f90 con 0x7f79241027c0 2024-01-11T18:00:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.069+0000 7f791a7fc700 1 -- 172.21.15.7:0/3076138928 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920022bc0 con 0x7f79241027c0 2024-01-11T18:00:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.069+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7924197100 con 0x7f79241027c0 2024-01-11T18:00:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.070+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7924197530 con 0x7f79241027c0 2024-01-11T18:00:08.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:07 smithi007 ceph-mon[33361]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 718 op/s 2024-01-11T18:00:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.077+0000 7f791a7fc700 1 -- 172.21.15.7:0/3076138928 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f7920003ad0 con 0x7f79241027c0 2024-01-11T18:00:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.080+0000 7f793f81db80 1 Processor -- start 2024-01-11T18:00:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.080+0000 7f793f81db80 1 -- start start 2024-01-11T18:00:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.081+0000 7f793f81db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e4b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.081+0000 7f793f81db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.081+0000 7f793f81db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e52e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.081+0000 7f793f81db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132d42030 con 0x55b132e4c5d0 2024-01-11T18:00:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.081+0000 7f793f81db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132da5ac0 con 0x55b132e4bc10 2024-01-11T18:00:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.081+0000 7f793f81db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132da33e0 con 0x55b132e4af20 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7929f9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e4b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f792979b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7929f9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e4b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45038/0 (socket says 172.21.15.7:45038) 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7929f9c700 1 -- 172.21.15.7:0/2825890444 learned_addr learned my addr 172.21.15.7:0/2825890444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f792979b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e4c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46630/0 (socket says 172.21.15.7:46630) 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f791a7fc700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f79100a1f50 0x7f79100a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7928f9a700 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e52e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7929f9c700 1 -- 172.21.15.7:0/2825890444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 msgr2=0x55b132e4c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7929f9c700 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e4c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.082+0000 7f7929f9c700 1 -- 172.21.15.7:0/2825890444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 msgr2=0x55b132e52e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.083+0000 7f791a7fc700 1 -- 172.21.15.7:0/3076138928 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f79200cf050 con 0x7f79241027c0 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.083+0000 7f7929f9c700 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e52e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.083+0000 7f7929f9c700 1 -- 172.21.15.7:0/2825890444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b132c13350 con 0x55b132e4af20 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.083+0000 7f7928f9a700 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e52e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.083+0000 7f7929f9c700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f79100a1f50 0x7f79100a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.083+0000 7f7929f9c700 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e4b340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f791c009fb0 tx=0x7f791c0077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82a2a042fb3f16f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.084+0000 7f790f7fe700 1 -- 172.21.15.7:0/2825890444 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c016b30 con 0x55b132e4af20 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.084+0000 7f790f7fe700 1 -- 172.21.15.7:0/2825890444 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f791c00f040 con 0x55b132e4af20 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.084+0000 7f790f7fe700 1 -- 172.21.15.7:0/2825890444 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0155a0 con 0x55b132e4af20 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 -- 172.21.15.7:0/2825890444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 msgr2=0x55b132e4b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f7929f9c700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f79100a1f50 0x7f79100a4410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f791c01f000 tx=0x7f791c015bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e4b340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f791c009fb0 tx=0x7f791c0077e0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 -- 172.21.15.7:0/2825890444 shutdown_connections 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e4b340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e52e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 --2- 172.21.15.7:0/2825890444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 -- 172.21.15.7:0/2825890444 >> 172.21.15.7:0/2825890444 conn(0x55b132dc0740 msgr2=0x55b132e45950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.085+0000 7f793f81db80 1 -- 172.21.15.7:0/2825890444 shutdown_connections 2024-01-11T18:00:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.086+0000 7f793f81db80 1 -- 172.21.15.7:0/2825890444 wait complete. 2024-01-11T18:00:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.087+0000 7f793f81db80 1 -- start start 2024-01-11T18:00:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.087+0000 7f793f81db80 1 Processor -- start 2024-01-11T18:00:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.089+0000 7f793f81db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e54930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.089+0000 7f793f81db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e54e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.089+0000 7f793f81db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e5beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.089+0000 7f793f81db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132d9ecd0 con 0x55b132e4c5d0 2024-01-11T18:00:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f793f81db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132d0d340 con 0x55b132e4bc10 2024-01-11T18:00:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f793f81db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132d0cfe0 con 0x55b132e4af20 2024-01-11T18:00:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f793f81db80 10 client.0 _ll_register_callbacks cb 0x55b132d3f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7928f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e54e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7929f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e5beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7928f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e54e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46638/0 (socket says 172.21.15.7:46638) 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7928f9a700 1 -- 172.21.15.7:0/2394072661 learned_addr learned my addr 172.21.15.7:0/2394072661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f792979b700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e54930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7929f9c700 1 -- 172.21.15.7:0/2394072661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 msgr2=0x55b132e54930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7929f9c700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e54930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.090+0000 7f7929f9c700 1 -- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 msgr2=0x55b132e54e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f7929f9c700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e54e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f7929f9c700 1 -- 172.21.15.7:0/2394072661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c009ba0 con 0x55b132e4c5d0 2024-01-11T18:00:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f7928f9a700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e54e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:00:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f7929f9c700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e5beb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f791c009f80 tx=0x7f791c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6b4cb5f3481f1f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f790cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:00:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f790cff9700 1 -- 172.21.15.7:0/2394072661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c004c10 con 0x55b132e4c5d0 2024-01-11T18:00:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f790cff9700 1 -- 172.21.15.7:0/2394072661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f791c004d70 con 0x55b132e4c5d0 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.091+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b132e3dd60 con 0x55b132e4c5d0 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.092+0000 7f790cff9700 1 -- 172.21.15.7:0/2394072661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f791c020b80 con 0x55b132e4c5d0 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.092+0000 7f793f81db80 10 client.15771 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.092+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b132bbfb00 con 0x55b132e4c5d0 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.092+0000 7f790cff9700 1 -- 172.21.15.7:0/2394072661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b132e3dd60 con 0x55b132e4c5d0 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f790cff9700 1 -- 172.21.15.7:0/2394072661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f791c01e480 con 0x55b132e4c5d0 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f793f81db80 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f793f81db80 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f793f81db80 20 client.15771 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f793f81db80 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132e60750 0x55b132e62b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f793f81db80 4 client.15771 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:00:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.093+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b132d97b60 con 0x55b132e60750 2024-01-11T18:00:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.094+0000 7f792979b700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132e60750 0x55b132e62b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.095+0000 7f790cff9700 10 client.15771 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.095+0000 7f792979b700 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132e60750 0x55b132e62b70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2e090011ea3d48 server_cookie=8868c9a70d7d16bd in_seq=0 out_seq=0 2024-01-11T18:00:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.097+0000 7f790cff9700 1 -- 172.21.15.7:0/2394072661 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b132d97b60 con 0x55b132e60750 2024-01-11T18:00:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.097+0000 7f790cff9700 10 client.15771 handle_command_reply: tid=0 2024-01-11T18:00:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.097+0000 7f793f81db80 1 client.15771 shutdown 2024-01-11T18:00:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 msgr2=0x55b132e5beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e5beb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f791c009f80 tx=0x7f791c017000 comp rx=0 tx=0).stop 2024-01-11T18:00:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 shutdown_connections 2024-01-11T18:00:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132e60750 0x55b132e62b70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4af20 0x55b132e54930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4c5d0 0x55b132e5beb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 --2- 172.21.15.7:0/2394072661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4bc10 0x55b132e54e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f790cff9700 0 client.15771 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 >> 172.21.15.7:0/2394072661 conn(0x55b132dc0740 msgr2=0x55b132e44f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 shutdown_connections 2024-01-11T18:00:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 1 -- 172.21.15.7:0/2394072661 wait complete. 2024-01-11T18:00:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.098+0000 7f793f81db80 20 client.15771 trim_cache size 0 max 16384 2024-01-11T18:00:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.127+0000 7f793f81db80 1 Processor -- start 2024-01-11T18:00:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 -- start start 2024-01-11T18:00:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 0x55b132e5be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 0x55b132e53eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132c15640 con 0x55b132e4af20 2024-01-11T18:00:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132e47690 con 0x55b132e4c5d0 2024-01-11T18:00:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.128+0000 7f793f81db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132e4ca70 con 0x55b132e4bc10 2024-01-11T18:00:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.129+0000 7f792979b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.129+0000 7f792979b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e4bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52338/0 (socket says 172.21.15.7:52338) 2024-01-11T18:00:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.129+0000 7f792979b700 1 -- 172.21.15.7:0/3998646583 learned_addr learned my addr 172.21.15.7:0/3998646583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.130+0000 7f792979b700 1 -- 172.21.15.7:0/3998646583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 msgr2=0x55b132e5be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.130+0000 7f792979b700 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 0x55b132e5be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.130+0000 7f792979b700 1 -- 172.21.15.7:0/3998646583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 msgr2=0x55b132e53eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.130+0000 7f792979b700 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 0x55b132e53eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.130+0000 7f792979b700 1 -- 172.21.15.7:0/3998646583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b132d601d0 con 0x55b132e4af20 2024-01-11T18:00:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.131+0000 7f792979b700 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e4bff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7914000c00 tx=0x7f7914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287ba03276b6d2ec server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.132+0000 7f790effd700 1 -- 172.21.15.7:0/3998646583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914016070 con 0x55b132e4af20 2024-01-11T18:00:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.132+0000 7f790effd700 1 -- 172.21.15.7:0/3998646583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7914017040 con 0x55b132e4af20 2024-01-11T18:00:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.132+0000 7f790effd700 1 -- 172.21.15.7:0/3998646583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914005440 con 0x55b132e4af20 2024-01-11T18:00:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 -- 172.21.15.7:0/3998646583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 msgr2=0x55b132e4bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e4bff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7914000c00 tx=0x7f7914004750 comp rx=0 tx=0).stop 2024-01-11T18:00:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 -- 172.21.15.7:0/3998646583 shutdown_connections 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 0x55b132e53eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e4bff0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 --2- 172.21.15.7:0/3998646583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 0x55b132e5be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 -- 172.21.15.7:0/3998646583 >> 172.21.15.7:0/3998646583 conn(0x55b132dc0740 msgr2=0x55b132e419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 -- 172.21.15.7:0/3998646583 shutdown_connections 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.133+0000 7f793f81db80 1 -- 172.21.15.7:0/3998646583 wait complete. 2024-01-11T18:00:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.134+0000 7f793f81db80 1 -- start start 2024-01-11T18:00:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.134+0000 7f793f81db80 1 Processor -- start 2024-01-11T18:00:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.135+0000 7f793f81db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.135+0000 7f793f81db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 0x55b132e59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.135+0000 7f793f81db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 0x55b132e5ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.136+0000 7f793f81db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132e5b270 con 0x55b132e4af20 2024-01-11T18:00:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.136+0000 7f793f81db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132e5b3e0 con 0x55b132e4c5d0 2024-01-11T18:00:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.136+0000 7f793f81db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b132e5b5a0 con 0x55b132e4bc10 2024-01-11T18:00:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.136+0000 7f793f81db80 10 client.0 _ll_register_callbacks cb 0x55b132d3f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.138+0000 7f792979b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.138+0000 7f792979b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52348/0 (socket says 172.21.15.7:52348) 2024-01-11T18:00:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.138+0000 7f792979b700 1 -- 172.21.15.7:0/3896004586 learned_addr learned my addr 172.21.15.7:0/3896004586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.140+0000 7f792979b700 1 -- 172.21.15.7:0/3896004586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 msgr2=0x55b132e59d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.140+0000 7f792979b700 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 0x55b132e59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.140+0000 7f792979b700 1 -- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 msgr2=0x55b132e5ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.140+0000 7f792979b700 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 0x55b132e5ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.140+0000 7f792979b700 1 -- 172.21.15.7:0/3896004586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791400bc70 con 0x55b132e4af20 2024-01-11T18:00:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.143+0000 7f792979b700 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e597e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7914020040 tx=0x7f7914003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9313393fd9dcaee8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.143+0000 7f790cff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:00:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.144+0000 7f790cff9700 1 -- 172.21.15.7:0/3896004586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914016070 con 0x55b132e4af20 2024-01-11T18:00:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.144+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f791c00f040 con 0x55b132e4af20 2024-01-11T18:00:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.145+0000 7f790cff9700 1 -- 172.21.15.7:0/3896004586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f79140055d0 con 0x55b132e4af20 2024-01-11T18:00:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.145+0000 7f790cff9700 1 -- 172.21.15.7:0/3896004586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79140047c0 con 0x55b132e4af20 2024-01-11T18:00:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.145+0000 7f793f81db80 10 client.15777 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.145+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b132c17c70 con 0x55b132e4af20 2024-01-11T18:00:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.147+0000 7f790cff9700 1 -- 172.21.15.7:0/3896004586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f791c00f040 con 0x55b132e4af20 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f790cff9700 1 -- 172.21.15.7:0/3896004586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7914026040 con 0x55b132e4af20 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f793f81db80 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f793f81db80 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f793f81db80 20 client.15777 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f793f81db80 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132ec86f0 0x55b132ecab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f793f81db80 4 client.15777 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b132d97880 con 0x55b132ec86f0 2024-01-11T18:00:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.148+0000 7f7928f9a700 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132ec86f0 0x55b132ecab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.150+0000 7f7928f9a700 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132ec86f0 0x55b132ecab10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38cde141c5c8a0a5 server_cookie=2e0610a0e843e40c in_seq=0 out_seq=0 2024-01-11T18:00:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.150+0000 7f790cff9700 10 client.15777 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f790cff9700 1 -- 172.21.15.7:0/3896004586 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b132d97880 con 0x55b132ec86f0 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f790cff9700 10 client.15777 handle_command_reply: tid=0 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 client.15777 shutdown 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 msgr2=0x55b132e597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e597e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7914020040 tx=0x7f7914003cd0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 shutdown_connections 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b132ec86f0 0x55b132ecab10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b132e4c5d0 0x55b132e5ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b132e4af20 0x55b132e597e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f790cff9700 0 client.15777 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 --2- 172.21.15.7:0/3896004586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b132e4bc10 0x55b132e59d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.248+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 >> 172.21.15.7:0/3896004586 conn(0x55b132dc0740 msgr2=0x55b132e53dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.249+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 shutdown_connections 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.249+0000 7f793f81db80 1 -- 172.21.15.7:0/3896004586 wait complete. 2024-01-11T18:00:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.249+0000 7f793f81db80 20 client.15777 trim_cache size 0 max 16384 2024-01-11T18:00:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:00:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:00:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:00:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:00:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f79100a1f50 msgr2=0x7f79100a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f79100a1f50 0x7f79100a4410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f791c01f000 tx=0x7f791c015bd0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 msgr2=0x7f7924073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924073a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7920009f50 tx=0x7f7920005740 comp rx=0 tx=0).stop 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 shutdown_connections 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f79100a1f50 0x7f79100a4410 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79241015d0 0x7f79240753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7924103160 0x7f7924073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 --2- 172.21.15.7:0/3076138928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79241027c0 0x7f7924073a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 >> 172.21.15.7:0/3076138928 conn(0x7f79240fcb60 msgr2=0x7f7924105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 shutdown_connections 2024-01-11T18:00:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:08.251+0000 7f792b9ff700 1 -- 172.21.15.7:0/3076138928 wait complete. 2024-01-11T18:00:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 100.0 s, 1464.1 tps, lat 67.441 ms stddev 81.191 2024-01-11T18:00:09.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:09 smithi016 ceph-mon[43195]: pgmap v370: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.8 MiB/s wr, 569 op/s 2024-01-11T18:00:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:09 smithi038 ceph-mon[43582]: pgmap v370: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.8 MiB/s wr, 569 op/s 2024-01-11T18:00:10.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:09 smithi007 ceph-mon[33361]: pgmap v370: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.8 MiB/s wr, 569 op/s 2024-01-11T18:00:12.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:11 smithi038 ceph-mon[43582]: pgmap v371: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 686 op/s 2024-01-11T18:00:12.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:11 smithi007 ceph-mon[33361]: pgmap v371: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 686 op/s 2024-01-11T18:00:12.251 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:11 smithi016 ceph-mon[43195]: pgmap v371: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 686 op/s 2024-01-11T18:00:14.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:13 smithi007 ceph-mon[33361]: pgmap v372: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 725 op/s 2024-01-11T18:00:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 105.0 s, 986.2 tps, lat 102.160 ms stddev 129.405 2024-01-11T18:00:14.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:13 smithi016 ceph-mon[43195]: pgmap v372: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 725 op/s 2024-01-11T18:00:14.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:13 smithi038 ceph-mon[43582]: pgmap v372: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 725 op/s 2024-01-11T18:00:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:15 smithi016 ceph-mon[43195]: pgmap v373: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 768 op/s 2024-01-11T18:00:15.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:15 smithi038 ceph-mon[43582]: pgmap v373: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 768 op/s 2024-01-11T18:00:15.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:15 smithi007 ceph-mon[33361]: pgmap v373: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 768 op/s 2024-01-11T18:00:17.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:17 smithi016 ceph-mon[43195]: pgmap v374: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 802 op/s 2024-01-11T18:00:18.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:17 smithi038 ceph-mon[43582]: pgmap v374: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 802 op/s 2024-01-11T18:00:18.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:17 smithi007 ceph-mon[33361]: pgmap v374: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 802 op/s 2024-01-11T18:00:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 110.0 s, 1160.7 tps, lat 86.424 ms stddev 102.283 2024-01-11T18:00:20.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:19 smithi038 ceph-mon[43582]: pgmap v375: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-01-11T18:00:20.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:19 smithi007 ceph-mon[33361]: pgmap v375: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-01-11T18:00:20.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:19 smithi016 ceph-mon[43195]: pgmap v375: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 669 op/s 2024-01-11T18:00:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.512+0000 7f52b49ff700 1 -- 172.21.15.7:0/1081639126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b0108610 msgr2=0x7f52b010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.512+0000 7f52b49ff700 1 --2- 172.21.15.7:0/1081639126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b0108610 0x7f52b010aa40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009990 tx=0x7f52a0009ca0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.515+0000 7f52b49ff700 1 -- 172.21.15.7:0/1081639126 shutdown_connections 2024-01-11T18:00:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.515+0000 7f52b49ff700 1 --2- 172.21.15.7:0/1081639126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b0108610 0x7f52b010aa40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.515+0000 7f52b49ff700 1 --2- 172.21.15.7:0/1081639126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b010d950 0x7f52b010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.515+0000 7f52b49ff700 1 --2- 172.21.15.7:0/1081639126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b010af80 0x7f52b010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.515+0000 7f52b49ff700 1 -- 172.21.15.7:0/1081639126 >> 172.21.15.7:0/1081639126 conn(0x7f52b006e500 msgr2=0x7f52b0070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.518+0000 7f52b49ff700 1 -- 172.21.15.7:0/1081639126 shutdown_connections 2024-01-11T18:00:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.518+0000 7f52b49ff700 1 -- 172.21.15.7:0/1081639126 wait complete. 2024-01-11T18:00:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.520+0000 7f52b49ff700 1 Processor -- start 2024-01-11T18:00:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.520+0000 7f52b49ff700 1 -- start start 2024-01-11T18:00:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52b49ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 0x7f52b01a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52b49ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b010af80 0x7f52b01a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52b49ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b010d950 0x7f52b01ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52b49ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01ade50 con 0x7f52b0108610 2024-01-11T18:00:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52b49ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01adf90 con 0x7f52b010d950 2024-01-11T18:00:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52b49ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b01ae120 con 0x7f52b010af80 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 0x7f52b01a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 0x7f52b01a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55382/0 (socket says 172.21.15.7:55382) 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.521+0000 7f52ae59c700 1 -- 172.21.15.7:0/3841361841 learned_addr learned my addr 172.21.15.7:0/3841361841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52add9b700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b010af80 0x7f52b01a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52ae59c700 1 -- 172.21.15.7:0/3841361841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b010af80 msgr2=0x7f52b01a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52ae59c700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b010af80 0x7f52b01a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52ae59c700 1 -- 172.21.15.7:0/3841361841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b010d950 msgr2=0x7f52b01ad880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52ae59c700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b010d950 0x7f52b01ad880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52ae59c700 1 -- 172.21.15.7:0/3841361841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a00095f0 con 0x7f52b0108610 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.522+0000 7f52add9b700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b010af80 0x7f52b01a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.523+0000 7f52ae59c700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 0x7f52b01a62d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f52a000f670 tx=0x7f52a000f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.523+0000 7f529f7fe700 1 -- 172.21.15.7:0/3841361841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a001d070 con 0x7f52b0108610 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.523+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b01ae3a0 con 0x7f52b0108610 2024-01-11T18:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.523+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b01ae820 con 0x7f52b0108610 2024-01-11T18:00:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.524+0000 7f529f7fe700 1 -- 172.21.15.7:0/3841361841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a000fc30 con 0x7f52b0108610 2024-01-11T18:00:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.524+0000 7f529f7fe700 1 -- 172.21.15.7:0/3841361841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000e3e0 con 0x7f52b0108610 2024-01-11T18:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.531+0000 7f52c86a0b80 1 Processor -- start 2024-01-11T18:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.532+0000 7f52c86a0b80 1 -- start start 2024-01-11T18:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.532+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268287340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.532+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e268287c10 0x55e268288090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.532+0000 7f52aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268287340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.533+0000 7f52aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268287340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38014/0 (socket says 172.21.15.7:38014) 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.533+0000 7f52aed9d700 1 -- 172.21.15.7:0/3277227528 learned_addr learned my addr 172.21.15.7:0/3277227528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.533+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2682885d0 0x55e26828ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.533+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26817e030 con 0x55e2682885d0 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.533+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2681e1ac0 con 0x55e268287c10 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.533+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2681df3e0 con 0x55e268286f20 2024-01-11T18:00:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.534+0000 7f52aed9d700 1 -- 172.21.15.7:0/3277227528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e268287c10 msgr2=0x55e268288090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:00:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.534+0000 7f52aed9d700 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e268287c10 0x55e268288090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.534+0000 7f52aed9d700 1 -- 172.21.15.7:0/3277227528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2682885d0 msgr2=0x55e26828ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.534+0000 7f52aed9d700 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2682885d0 0x55e26828ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.534+0000 7f52aed9d700 1 -- 172.21.15.7:0/3277227528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e26804f350 con 0x55e268286f20 2024-01-11T18:00:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.536+0000 7f52aed9d700 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268287340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52a8008f70 tx=0x7f52a800d3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5aa98242f38661 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.536+0000 7f529f7fe700 1 -- 172.21.15.7:0/3841361841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f52a002b3b0 con 0x7f52b0108610 2024-01-11T18:00:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.537+0000 7f529cff9700 1 -- 172.21.15.7:0/3277227528 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8013420 con 0x55e268286f20 2024-01-11T18:00:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.537+0000 7f529cff9700 1 -- 172.21.15.7:0/3277227528 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a8013a40 con 0x55e268286f20 2024-01-11T18:00:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.537+0000 7f529cff9700 1 -- 172.21.15.7:0/3277227528 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80129d0 con 0x55e268286f20 2024-01-11T18:00:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.538+0000 7f529f7fe700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f52980a1f50 0x7f52980a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.538+0000 7f529f7fe700 1 -- 172.21.15.7:0/3841361841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f52a00cf090 con 0x7f52b0108610 2024-01-11T18:00:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.539+0000 7f52ae59c700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f52980a1f50 0x7f52980a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.539+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 msgr2=0x55e268287340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.539+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268287340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52a8008f70 tx=0x7f52a800d3c0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.540+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 shutdown_connections 2024-01-11T18:00:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.540+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268287340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.540+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2682885d0 0x55e26828ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.540+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/3277227528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e268287c10 0x55e268288090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.540+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 >> 172.21.15.7:0/3277227528 conn(0x55e2681fc740 msgr2=0x55e268281950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.542+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 shutdown_connections 2024-01-11T18:00:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.542+0000 7f52c86a0b80 1 -- 172.21.15.7:0/3277227528 wait complete. 2024-01-11T18:00:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.543+0000 7f52c86a0b80 1 -- start start 2024-01-11T18:00:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.543+0000 7f52ae59c700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f52980a1f50 0x7f52980a4410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f52a00985c0 tx=0x7f52a00092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.543+0000 7f52c86a0b80 1 Processor -- start 2024-01-11T18:00:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.544+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e2682909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268290ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52c86a0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2681dacd0 con 0x55e268286f20 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52c86a0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268149340 con 0x55e2682885d0 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52c86a0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268148fe0 con 0x55e268287c10 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52c86a0b80 10 client.0 _ll_register_callbacks cb 0x55e26817b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52add9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268290ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52aed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e2682909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52add9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268290ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38022/0 (socket says 172.21.15.7:38022) 2024-01-11T18:00:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.546+0000 7f52add9b700 1 -- 172.21.15.7:0/2005870662 learned_addr learned my addr 172.21.15.7:0/2005870662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52add9b700 1 -- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 msgr2=0x55e268297f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52add9b700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52add9b700 1 -- 172.21.15.7:0/2005870662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 msgr2=0x55e2682909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52add9b700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e2682909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52add9b700 1 -- 172.21.15.7:0/2005870662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a8008ba0 con 0x55e268287c10 2024-01-11T18:00:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.547+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:00:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.548+0000 7f52add9b700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268290ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c250 tx=0x7f52a400c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5855167513497de server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.549+0000 7f5295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.549+0000 7f5295ffb700 1 -- 172.21.15.7:0/2005870662 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40115f0 con 0x55e268287c10 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.549+0000 7f5295ffb700 1 -- 172.21.15.7:0/2005870662 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f52a4011750 con 0x55e268287c10 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.549+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e268279d60 con 0x55e268287c10 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.549+0000 7f5295ffb700 1 -- 172.21.15.7:0/2005870662 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a4011910 con 0x55e268287c10 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.550+0000 7f52c86a0b80 10 client.25873 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.550+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e267ffbb00 con 0x55e268287c10 2024-01-11T18:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.551+0000 7f5295ffb700 1 -- 172.21.15.7:0/2005870662 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e268279d60 con 0x55e268287c10 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f5295ffb700 1 -- 172.21.15.7:0/2005870662 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f52a40193b0 con 0x55e268287c10 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52c86a0b80 10 client.25873 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52c86a0b80 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52c86a0b80 20 client.25873 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e26829c920 0x55e26829ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52c86a0b80 4 client.25873 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2681d3b60 con 0x55e26829c920 2024-01-11T18:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.552+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e26829c920 0x55e26829ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.554+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e26829c920 0x55e26829ed40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d645fefdc5183c server_cookie=1ce5889ccf4233ad in_seq=0 out_seq=0 2024-01-11T18:00:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.554+0000 7f5295ffb700 10 client.25873 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.557+0000 7f5295ffb700 1 -- 172.21.15.7:0/2005870662 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2681d3b60 con 0x55e26829c920 2024-01-11T18:00:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.557+0000 7f5295ffb700 10 client.25873 handle_command_reply: tid=0 2024-01-11T18:00:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.557+0000 7f52c86a0b80 1 client.25873 shutdown 2024-01-11T18:00:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 msgr2=0x55e268290ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268290ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c250 tx=0x7f52a400c610 comp rx=0 tx=0).stop 2024-01-11T18:00:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 shutdown_connections 2024-01-11T18:00:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e26829c920 0x55e26829ed40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e2682909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2005870662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268290ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f5295ffb700 0 client.25873 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.558+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 >> 172.21.15.7:0/2005870662 conn(0x55e2681fc740 msgr2=0x55e268280f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.559+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 shutdown_connections 2024-01-11T18:00:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.560+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2005870662 wait complete. 2024-01-11T18:00:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.560+0000 7f52c86a0b80 20 client.25873 trim_cache size 0 max 16384 2024-01-11T18:00:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.596+0000 7f52c86a0b80 1 Processor -- start 2024-01-11T18:00:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 -- start start 2024-01-11T18:00:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e26828ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268297e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e26828b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268051640 con 0x55e268286f20 2024-01-11T18:00:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2681d3b60 con 0x55e2682885d0 2024-01-11T18:00:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.597+0000 7f52c86a0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2682990a0 con 0x55e268287c10 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.598+0000 7f52add9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268297e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.598+0000 7f52add9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268297e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38036/0 (socket says 172.21.15.7:38036) 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.598+0000 7f52add9b700 1 -- 172.21.15.7:0/2761709877 learned_addr learned my addr 172.21.15.7:0/2761709877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.598+0000 7f52add9b700 1 -- 172.21.15.7:0/2761709877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 msgr2=0x55e26828b630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e26828ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52add9b700 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e26828b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52add9b700 1 -- 172.21.15.7:0/2761709877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 msgr2=0x55e26828ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52add9b700 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e26828ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52add9b700 1 -- 172.21.15.7:0/2761709877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e26819c1d0 con 0x55e268287c10 2024-01-11T18:00:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e26828ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:00:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f52add9b700 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268297e40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f52a4000c00 tx=0x7f52a40047f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4552099b0594fd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f5297fff700 1 -- 172.21.15.7:0/2761709877 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4019d60 con 0x55e268287c10 2024-01-11T18:00:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f5297fff700 1 -- 172.21.15.7:0/2761709877 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a40044f0 con 0x55e268287c10 2024-01-11T18:00:21.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.599+0000 7f5297fff700 1 -- 172.21.15.7:0/2761709877 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40039a0 con 0x55e268287c10 2024-01-11T18:00:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2761709877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 msgr2=0x55e268297e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268297e40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f52a4000c00 tx=0x7f52a40047f0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2761709877 shutdown_connections 2024-01-11T18:00:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e26828b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268286f20 0x55e26828ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2761709877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268287c10 0x55e268297e40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2761709877 >> 172.21.15.7:0/2761709877 conn(0x55e2681fc740 msgr2=0x55e26828ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2761709877 shutdown_connections 2024-01-11T18:00:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.602+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2761709877 wait complete. 2024-01-11T18:00:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.603+0000 7f52c86a0b80 1 -- start start 2024-01-11T18:00:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.603+0000 7f52c86a0b80 1 Processor -- start 2024-01-11T18:00:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.611+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268295ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52c86a0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268287c10 0x55e268296020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268295ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268295ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38042/0 (socket says 172.21.15.7:38042) 2024-01-11T18:00:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52aed9d700 1 -- 172.21.15.7:0/2672436746 learned_addr learned my addr 172.21.15.7:0/2672436746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268297570 con 0x55e268287c10 2024-01-11T18:00:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2682976e0 con 0x55e2682885d0 2024-01-11T18:00:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2682978a0 con 0x55e268286f20 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.612+0000 7f52c86a0b80 10 client.0 _ll_register_callbacks cb 0x55e26817b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52add9b700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268287c10 0x55e268296020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 -- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 msgr2=0x55e268297030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 -- 172.21.15.7:0/2672436746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268287c10 msgr2=0x55e268296020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268287c10 0x55e268296020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e267ffbb00 con 0x55e268286f20 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.613+0000 7f52aed9d700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268295ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52a8008f70 tx=0x7f52a8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674fc975b1290eae server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f5295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f5295ffb700 1 -- 172.21.15.7:0/2672436746 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8023020 con 0x55e268286f20 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f5295ffb700 1 -- 172.21.15.7:0/2672436746 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f52a8013420 con 0x55e268286f20 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52a8013a40 con 0x55e268286f20 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f5295ffb700 1 -- 172.21.15.7:0/2672436746 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52a8026e50 con 0x55e268286f20 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f52c86a0b80 10 client.25879 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.614+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e268053c70 con 0x55e268286f20 2024-01-11T18:00:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.615+0000 7f5295ffb700 1 -- 172.21.15.7:0/2672436746 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a8032020 con 0x55e268286f20 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.615+0000 7f5295ffb700 1 -- 172.21.15.7:0/2672436746 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f52a8005070 con 0x55e268286f20 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52c86a0b80 10 client.25879 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52c86a0b80 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52c86a0b80 20 client.25879 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e2682f3b40 0x55e2682f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52c86a0b80 4 client.25879 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:00:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2681d3880 con 0x55e2682f3b40 2024-01-11T18:00:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.616+0000 7f52add9b700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e2682f3b40 0x55e2682f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.617+0000 7f52add9b700 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e2682f3b40 0x55e2682f5f60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4387f9fe1a3c894a server_cookie=4c6bd26143d6589 in_seq=0 out_seq=0 2024-01-11T18:00:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.617+0000 7f5295ffb700 10 client.25879 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f5295ffb700 1 -- 172.21.15.7:0/2672436746 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2681d3880 con 0x55e2682f3b40 2024-01-11T18:00:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f5295ffb700 10 client.25879 handle_command_reply: tid=0 2024-01-11T18:00:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 client.25879 shutdown 2024-01-11T18:00:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 msgr2=0x55e268295ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268295ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52a8008f70 tx=0x7f52a8000f40 comp rx=0 tx=0).stop 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 shutdown_connections 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e2682f3b40 0x55e2682f5f60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e268286f20 0x55e268295ae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2682885d0 0x55e268297030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 --2- 172.21.15.7:0/2672436746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e268287c10 0x55e268296020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.635+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 >> 172.21.15.7:0/2672436746 conn(0x55e2681fc740 msgr2=0x55e26828e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.636+0000 7f5295ffb700 0 client.25879 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.636+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 shutdown_connections 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.636+0000 7f52c86a0b80 1 -- 172.21.15.7:0/2672436746 wait complete. 2024-01-11T18:00:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.636+0000 7f52c86a0b80 20 client.25879 trim_cache size 0 max 16384 2024-01-11T18:00:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:00:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:00:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:00:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f52980a1f50 msgr2=0x7f52980a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f52980a1f50 0x7f52980a4410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f52a00985c0 tx=0x7f52a00092c0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 msgr2=0x7f52b01a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 0x7f52b01a62d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f52a000f670 tx=0x7f52a000f750 comp rx=0 tx=0).stop 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 shutdown_connections 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f52980a1f50 0x7f52980a4410 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52b010d950 0x7f52b01ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.637+0000 7f52b49ff700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b0108610 0x7f52b01a62d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.638+0000 7f52b49ff700 1 --2- 172.21.15.7:0/3841361841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52b010af80 0x7f52b01a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.638+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 >> 172.21.15.7:0/3841361841 conn(0x7f52b006e500 msgr2=0x7f52b010d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.638+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 shutdown_connections 2024-01-11T18:00:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:21.638+0000 7f52b49ff700 1 -- 172.21.15.7:0/3841361841 wait complete. 2024-01-11T18:00:22.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:21 smithi038 ceph-mon[43582]: pgmap v376: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 14 MiB/s wr, 795 op/s 2024-01-11T18:00:22.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:21 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:00:22.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:21 smithi007 ceph-mon[33361]: pgmap v376: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 14 MiB/s wr, 795 op/s 2024-01-11T18:00:22.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:21 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:00:22.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:21 smithi016 ceph-mon[43195]: pgmap v376: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 14 MiB/s wr, 795 op/s 2024-01-11T18:00:22.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:21 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:00:23.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:23.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:23 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:23.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:23 smithi016 ceph-mon[43195]: pgmap v377: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 785 op/s 2024-01-11T18:00:23.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:23.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:23 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:23.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:23 smithi038 ceph-mon[43582]: pgmap v377: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 785 op/s 2024-01-11T18:00:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:23 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:23.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:23 smithi007 ceph-mon[33361]: pgmap v377: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 785 op/s 2024-01-11T18:00:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 115.0 s, 1002.3 tps, lat 98.017 ms stddev 128.944 2024-01-11T18:00:24.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:00:24.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:00:24.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:24.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:00:24.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:00:24.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:24.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:00:24.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:00:24.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:00:25.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:25 smithi016 ceph-mon[43195]: pgmap v378: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 14 MiB/s wr, 719 op/s 2024-01-11T18:00:25.775 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:25 smithi038 ceph-mon[43582]: pgmap v378: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 14 MiB/s wr, 719 op/s 2024-01-11T18:00:25.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:25 smithi007 ceph-mon[33361]: pgmap v378: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 14 MiB/s wr, 719 op/s 2024-01-11T18:00:27.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:27 smithi016 ceph-mon[43195]: pgmap v379: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 739 op/s 2024-01-11T18:00:28.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:27 smithi038 ceph-mon[43582]: pgmap v379: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 739 op/s 2024-01-11T18:00:28.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:27 smithi007 ceph-mon[33361]: pgmap v379: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 739 op/s 2024-01-11T18:00:29.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 120.0 s, 1348.0 tps, lat 75.449 ms stddev 95.363 2024-01-11T18:00:30.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:29 smithi038 ceph-mon[43582]: pgmap v380: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-01-11T18:00:30.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:29 smithi007 ceph-mon[33361]: pgmap v380: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-01-11T18:00:30.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:29 smithi016 ceph-mon[43195]: pgmap v380: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-01-11T18:00:31.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:31 smithi016 ceph-mon[43195]: pgmap v381: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 718 op/s 2024-01-11T18:00:32.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:31 smithi038 ceph-mon[43582]: pgmap v381: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 718 op/s 2024-01-11T18:00:32.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:31 smithi007 ceph-mon[33361]: pgmap v381: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 14 MiB/s wr, 718 op/s 2024-01-11T18:00:34.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:33 smithi038 ceph-mon[43582]: pgmap v382: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 712 op/s 2024-01-11T18:00:34.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:33 smithi007 ceph-mon[33361]: pgmap v382: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 712 op/s 2024-01-11T18:00:34.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 125.0 s, 1040.3 tps, lat 95.839 ms stddev 109.017 2024-01-11T18:00:34.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:33 smithi016 ceph-mon[43195]: pgmap v382: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 712 op/s 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 -- 172.21.15.7:0/3057551899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810d950 msgr2=0x7f40b810ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3057551899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810d950 0x7f40b810ddd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009f90 tx=0x7f40b000b330 comp rx=0 tx=0).stop 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 -- 172.21.15.7:0/3057551899 shutdown_connections 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3057551899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810d950 0x7f40b810ddd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3057551899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b8108610 0x7f40b810aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3057551899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b810af80 0x7f40b810d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.930+0000 7f40bfdff700 1 -- 172.21.15.7:0/3057551899 >> 172.21.15.7:0/3057551899 conn(0x7f40b806e7a0 msgr2=0x7f40b8070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.931+0000 7f40bfdff700 1 -- 172.21.15.7:0/3057551899 shutdown_connections 2024-01-11T18:00:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.931+0000 7f40bfdff700 1 -- 172.21.15.7:0/3057551899 wait complete. 2024-01-11T18:00:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.932+0000 7f40bfdff700 1 Processor -- start 2024-01-11T18:00:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.933+0000 7f40bfdff700 1 -- start start 2024-01-11T18:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.933+0000 7f40bfdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b8108610 0x7f40b81a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.933+0000 7f40bfdff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810af80 0x7f40b81a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.933+0000 7f40bfdff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b810d950 0x7f40b81ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810af80 0x7f40b81a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810af80 0x7f40b81a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37208/0 (socket says 172.21.15.7:37208) 2024-01-11T18:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bd39a700 1 -- 172.21.15.7:0/3087761209 learned_addr learned my addr 172.21.15.7:0/3087761209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40be39c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b810d950 0x7f40b81ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b81adfb0 con 0x7f40b8108610 2024-01-11T18:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b81ae0f0 con 0x7f40b810d950 2024-01-11T18:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b81ae280 con 0x7f40b810af80 2024-01-11T18:00:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b8108610 0x7f40b81a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40be39c700 1 -- 172.21.15.7:0/3087761209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810af80 msgr2=0x7f40b81a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40be39c700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810af80 0x7f40b81a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40be39c700 1 -- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b8108610 msgr2=0x7f40b81a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40be39c700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b8108610 0x7f40b81a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.934+0000 7f40be39c700 1 -- 172.21.15.7:0/3087761209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b0009c40 con 0x7f40b810d950 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.935+0000 7f40be39c700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b810d950 0x7f40b81ad9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b6c0 tx=0x7f40b0004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.936+0000 7f40aeffd700 1 -- 172.21.15.7:0/3087761209 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000e040 con 0x7f40b810d950 2024-01-11T18:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.936+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b81ae500 con 0x7f40b810d950 2024-01-11T18:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.936+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b81ae980 con 0x7f40b810d950 2024-01-11T18:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.936+0000 7f40aeffd700 1 -- 172.21.15.7:0/3087761209 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b0009410 con 0x7f40b810d950 2024-01-11T18:00:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.936+0000 7f40aeffd700 1 -- 172.21.15.7:0/3087761209 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00052a0 con 0x7f40b810d950 2024-01-11T18:00:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.946+0000 7f40aeffd700 1 -- 172.21.15.7:0/3087761209 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f40b001d470 con 0x7f40b810d950 2024-01-11T18:00:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.947+0000 7f40d3b5db80 1 Processor -- start 2024-01-11T18:00:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.948+0000 7f40d3b5db80 1 -- start start 2024-01-11T18:00:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 0x563d504c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504ca5d0 0x563d504d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d503c0030 con 0x563d504c9c10 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d50423ac0 con 0x563d504c8f20 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d504213e0 con 0x563d504ca5d0 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 0x563d504c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40bdb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40bdb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45626/0 (socket says 172.21.15.7:45626) 2024-01-11T18:00:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40bdb9b700 1 -- 172.21.15.7:0/3548147799 learned_addr learned my addr 172.21.15.7:0/3548147799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.950+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 0x563d504c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34288/0 (socket says 172.21.15.7:34288) 2024-01-11T18:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.951+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504ca5d0 0x563d504d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.951+0000 7f40aeffd700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40a40a2020 0x7f40a40a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.951+0000 7f40aeffd700 1 -- 172.21.15.7:0/3087761209 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40b00cf130 con 0x7f40b810d950 2024-01-11T18:00:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40bd39a700 1 -- 172.21.15.7:0/3548147799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504ca5d0 msgr2=0x563d504d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504ca5d0 0x563d504d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40be39c700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40a40a2020 0x7f40a40a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40bd39a700 1 -- 172.21.15.7:0/3548147799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 msgr2=0x563d504ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40bd39a700 1 -- 172.21.15.7:0/3548147799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d50291350 con 0x563d504c8f20 2024-01-11T18:00:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.952+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.954+0000 7f40be39c700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40a40a2020 0x7f40a40a44e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f40b002dff0 tx=0x7f40b000bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.955+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 0x563d504c9340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009f00 tx=0x7f40a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01ac78c7bdd03e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.955+0000 7f40a3fff700 1 -- 172.21.15.7:0/3548147799 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8018070 con 0x563d504c8f20 2024-01-11T18:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.955+0000 7f40a3fff700 1 -- 172.21.15.7:0/3548147799 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a8013e30 con 0x563d504c8f20 2024-01-11T18:00:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.955+0000 7f40a3fff700 1 -- 172.21.15.7:0/3548147799 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8010430 con 0x563d504c8f20 2024-01-11T18:00:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3548147799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 msgr2=0x563d504c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 0x563d504c9340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009f00 tx=0x7f40a8009260 comp rx=0 tx=0).stop 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3548147799 shutdown_connections 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504ca5d0 0x563d504d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504c8f20 0x563d504c9340 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3548147799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3548147799 >> 172.21.15.7:0/3548147799 conn(0x563d5043e740 msgr2=0x563d504c3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.956+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3548147799 shutdown_connections 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.957+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3548147799 wait complete. 2024-01-11T18:00:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.957+0000 7f40d3b5db80 1 -- start start 2024-01-11T18:00:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 Processor -- start 2024-01-11T18:00:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5041ccd0 con 0x563d504c9c10 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5038b340 con 0x563d504ca5d0 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5038afe0 con 0x563d504c8f20 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.958+0000 7f40d3b5db80 10 client.0 _ll_register_callbacks cb 0x563d503bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.959+0000 7f40bdb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.959+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.959+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37220/0 (socket says 172.21.15.7:37220) 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.959+0000 7f40bd39a700 1 -- 172.21.15.7:0/3847709008 learned_addr learned my addr 172.21.15.7:0/3847709008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.959+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bdb9b700 1 -- 172.21.15.7:0/3847709008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 msgr2=0x563d504d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bdb9b700 1 -- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 msgr2=0x563d504d9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bdb9b700 1 -- 172.21.15.7:0/3847709008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b400b960 con 0x563d504c9c10 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504d2ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f40b40126d0 tx=0x7f40b40129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57024f2a552f01a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.960+0000 7f40a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.961+0000 7f40a17fa700 1 -- 172.21.15.7:0/3847709008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004460 con 0x563d504c9c10 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.961+0000 7f40a17fa700 1 -- 172.21.15.7:0/3847709008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40b4003680 con 0x563d504c9c10 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.961+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d504bbd60 con 0x563d504c9c10 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.961+0000 7f40a17fa700 1 -- 172.21.15.7:0/3847709008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b4003840 con 0x563d504c9c10 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.961+0000 7f40d3b5db80 10 client.15792 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.961+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f40a8013460 con 0x563d504c9c10 2024-01-11T18:00:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40a17fa700 1 -- 172.21.15.7:0/3847709008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d504bbd60 con 0x563d504c9c10 2024-01-11T18:00:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40a17fa700 1 -- 172.21.15.7:0/3847709008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f40b4025a80 con 0x563d504c9c10 2024-01-11T18:00:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40d3b5db80 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40d3b5db80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40d3b5db80 20 client.15792 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d504ef8e0 0x563d504f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40d3b5db80 4 client.15792 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:00:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.963+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d50415b60 con 0x563d504ef8e0 2024-01-11T18:00:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.965+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d504ef8e0 0x563d504f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.966+0000 7f40bd39a700 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d504ef8e0 0x563d504f1d00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1507daab992b4f4c server_cookie=3dad9c132b0ddab4 in_seq=0 out_seq=0 2024-01-11T18:00:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.966+0000 7f40a17fa700 10 client.15792 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.967+0000 7f40a17fa700 1 -- 172.21.15.7:0/3847709008 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d50415b60 con 0x563d504ef8e0 2024-01-11T18:00:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.967+0000 7f40a17fa700 10 client.15792 handle_command_reply: tid=0 2024-01-11T18:00:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.968+0000 7f40d3b5db80 1 client.15792 shutdown 2024-01-11T18:00:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 msgr2=0x563d504d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504d2ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f40b40126d0 tx=0x7f40b40129e0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 shutdown_connections 2024-01-11T18:00:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d504ef8e0 0x563d504f1d00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/3847709008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9c10 0x563d504d2ef0 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f40b40126d0 tx=0x7f40b40129e0 comp rx=0 tx=0).stop 2024-01-11T18:00:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 >> 172.21.15.7:0/3847709008 conn(0x563d5043e740 msgr2=0x563d504c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.969+0000 7f40a17fa700 0 client.15792 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.973+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 shutdown_connections 2024-01-11T18:00:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.973+0000 7f40d3b5db80 1 -- 172.21.15.7:0/3847709008 wait complete. 2024-01-11T18:00:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:34.973+0000 7f40d3b5db80 20 client.15792 trim_cache size 0 max 16384 2024-01-11T18:00:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.012+0000 7f40d3b5db80 1 Processor -- start 2024-01-11T18:00:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 -- start start 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9ff0 0x563d504f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d50293640 con 0x563d504c9ff0 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d504d2520 con 0x563d504ca5d0 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d504c4440 con 0x563d504c8f20 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.014+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.015+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504bb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37222/0 (socket says 172.21.15.7:37222) 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.015+0000 7f40bd39a700 1 -- 172.21.15.7:0/887628922 learned_addr learned my addr 172.21.15.7:0/887628922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.015+0000 7f40bd39a700 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9ff0 0x563d504f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.015+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.016+0000 7f40bd39a700 1 -- 172.21.15.7:0/887628922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 msgr2=0x563d504bb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.016+0000 7f40bd39a700 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504bb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.016+0000 7f40bd39a700 1 -- 172.21.15.7:0/887628922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 msgr2=0x563d504d9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.016+0000 7f40bd39a700 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.016+0000 7f40bd39a700 1 -- 172.21.15.7:0/887628922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d5023db00 con 0x563d504c9ff0 2024-01-11T18:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.016+0000 7f40bd39a700 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9ff0 0x563d504f6660 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x563d504c8c40 tx=0x7f40a8003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fc26818499c83f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.017+0000 7f40a37fe700 1 -- 172.21.15.7:0/887628922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8018070 con 0x563d504c9ff0 2024-01-11T18:00:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.017+0000 7f40a37fe700 1 -- 172.21.15.7:0/887628922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d50415b60 con 0x563d504c9ff0 2024-01-11T18:00:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.018+0000 7f40a37fe700 1 -- 172.21.15.7:0/887628922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8018070 con 0x563d504c9ff0 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.018+0000 7f40d3b5db80 1 -- 172.21.15.7:0/887628922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9ff0 msgr2=0x563d504f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.018+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9ff0 0x563d504f6660 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x563d504c8c40 tx=0x7f40a8003800 comp rx=0 tx=0).stop 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 -- 172.21.15.7:0/887628922 shutdown_connections 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c8f20 0x563d504bb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c9ff0 0x563d504f6660 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/887628922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 -- 172.21.15.7:0/887628922 >> 172.21.15.7:0/887628922 conn(0x563d5043e740 msgr2=0x563d504d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 -- 172.21.15.7:0/887628922 shutdown_connections 2024-01-11T18:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.019+0000 7f40d3b5db80 1 -- 172.21.15.7:0/887628922 wait complete. 2024-01-11T18:00:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.020+0000 7f40d3b5db80 1 -- start start 2024-01-11T18:00:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.021+0000 7f40d3b5db80 1 Processor -- start 2024-01-11T18:00:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 0x563d504d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c9ff0 0x563d504d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d504d8c90 con 0x563d504c8f20 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 0x563d504d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40bd39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 0x563d504d72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45660/0 (socket says 172.21.15.7:45660) 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40bd39a700 1 -- 172.21.15.7:0/2332659826 learned_addr learned my addr 172.21.15.7:0/2332659826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d504d8e00 con 0x563d504ca5d0 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d504d8fc0 con 0x563d504c9ff0 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.022+0000 7f40d3b5db80 10 client.0 _ll_register_callbacks cb 0x563d503bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.023+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c9ff0 0x563d504d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.023+0000 7f40bd39a700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.024+0000 7f40bd39a700 1 -- 172.21.15.7:0/2332659826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c9ff0 msgr2=0x563d504d7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.024+0000 7f40bd39a700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c9ff0 0x563d504d7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.024+0000 7f40bd39a700 1 -- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 msgr2=0x563d504d8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.024+0000 7f40bd39a700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.024+0000 7f40bd39a700 1 -- 172.21.15.7:0/2332659826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a8029040 con 0x563d504c8f20 2024-01-11T18:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.025+0000 7f40bd39a700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 0x563d504d72f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f40a80055a0 tx=0x7f40a8005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a118615d0ae16e51 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.025+0000 7f40a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:00:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.025+0000 7f40a17fa700 1 -- 172.21.15.7:0/2332659826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8036030 con 0x563d504c8f20 2024-01-11T18:00:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.025+0000 7f40a17fa700 1 -- 172.21.15.7:0/2332659826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40a8000e40 con 0x563d504c8f20 2024-01-11T18:00:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.025+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40a8013e30 con 0x563d504c8f20 2024-01-11T18:00:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.026+0000 7f40a17fa700 1 -- 172.21.15.7:0/2332659826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a8039040 con 0x563d504c8f20 2024-01-11T18:00:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.026+0000 7f40d3b5db80 10 client.15798 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.026+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563d503de1d0 con 0x563d504c8f20 2024-01-11T18:00:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40a17fa700 1 -- 172.21.15.7:0/2332659826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a8013e30 con 0x563d504c8f20 2024-01-11T18:00:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40a17fa700 1 -- 172.21.15.7:0/2332659826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f40a80108a0 con 0x563d504c8f20 2024-01-11T18:00:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40d3b5db80 10 client.15798 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40d3b5db80 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40d3b5db80 20 client.15798 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d50545f20 0x563d50548310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40d3b5db80 4 client.15798 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.027+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d50415880 con 0x563d50545f20 2024-01-11T18:00:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.028+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d50545f20 0x563d50548310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.029+0000 7f40a17fa700 10 client.15798 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.029+0000 7f40bdb9b700 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d50545f20 0x563d50548310 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b518df5016fde2 server_cookie=de4949e84204cb35 in_seq=0 out_seq=0 2024-01-11T18:00:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.120+0000 7f40a17fa700 1 -- 172.21.15.7:0/2332659826 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d50415880 con 0x563d50545f20 2024-01-11T18:00:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40a17fa700 10 client.15798 handle_command_reply: tid=0 2024-01-11T18:00:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 client.15798 shutdown 2024-01-11T18:00:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 msgr2=0x563d504d72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 0x563d504d72f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f40a80055a0 tx=0x7f40a8005c80 comp rx=0 tx=0).stop 2024-01-11T18:00:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 shutdown_connections 2024-01-11T18:00:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563d50545f20 0x563d50548310 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d504ca5d0 0x563d504d8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563d504c8f20 0x563d504d72f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40a17fa700 0 client.15798 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 --2- 172.21.15.7:0/2332659826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d504c9ff0 0x563d504d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.121+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 >> 172.21.15.7:0/2332659826 conn(0x563d5043e740 msgr2=0x563d504cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.122+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 shutdown_connections 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.122+0000 7f40d3b5db80 1 -- 172.21.15.7:0/2332659826 wait complete. 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.122+0000 7f40d3b5db80 20 client.15798 trim_cache size 0 max 16384 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:00:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:00:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:00:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:00:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.125+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40a40a2020 msgr2=0x7f40a40a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.125+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40a40a2020 0x7f40a40a44e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f40b002dff0 tx=0x7f40b000bf90 comp rx=0 tx=0).stop 2024-01-11T18:00:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.125+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b810d950 msgr2=0x7f40b81ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.125+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b810d950 0x7f40b81ad9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b6c0 tx=0x7f40b0004c40 comp rx=0 tx=0).stop 2024-01-11T18:00:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.125+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 shutdown_connections 2024-01-11T18:00:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.126+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40a40a2020 0x7f40a40a44e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.126+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b810d950 0x7f40b81ad9e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.126+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b8108610 0x7f40b81a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.126+0000 7f40bfdff700 1 --2- 172.21.15.7:0/3087761209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40b810af80 0x7f40b81a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.126+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 >> 172.21.15.7:0/3087761209 conn(0x7f40b806e7a0 msgr2=0x7f40b810b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.128+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 shutdown_connections 2024-01-11T18:00:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:35.128+0000 7f40bfdff700 1 -- 172.21.15.7:0/3087761209 wait complete. 2024-01-11T18:00:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:35 smithi016 ceph-mon[43195]: pgmap v383: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 649 op/s 2024-01-11T18:00:35.776 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:35 smithi038 ceph-mon[43582]: pgmap v383: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 649 op/s 2024-01-11T18:00:35.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:35 smithi007 ceph-mon[33361]: pgmap v383: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 649 op/s 2024-01-11T18:00:37.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:37 smithi016 ceph-mon[43195]: pgmap v384: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-01-11T18:00:38.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:37 smithi038 ceph-mon[43582]: pgmap v384: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-01-11T18:00:38.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:37 smithi007 ceph-mon[33361]: pgmap v384: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-01-11T18:00:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 130.0 s, 534.1 tps, lat 189.514 ms stddev 448.797 2024-01-11T18:00:40.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:39 smithi038 ceph-mon[43582]: pgmap v385: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-01-11T18:00:40.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:39 smithi007 ceph-mon[33361]: pgmap v385: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-01-11T18:00:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:39 smithi016 ceph-mon[43195]: pgmap v385: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-01-11T18:00:42.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:41 smithi038 ceph-mon[43582]: pgmap v386: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 24 MiB/s wr, 1.61k op/s 2024-01-11T18:00:42.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:41 smithi007 ceph-mon[33361]: pgmap v386: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 24 MiB/s wr, 1.61k op/s 2024-01-11T18:00:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:41 smithi016 ceph-mon[43195]: pgmap v386: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 24 MiB/s wr, 1.61k op/s 2024-01-11T18:00:44.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:43 smithi038 ceph-mon[43582]: pgmap v387: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-01-11T18:00:44.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:43 smithi007 ceph-mon[33361]: pgmap v387: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-01-11T18:00:44.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 135.0 s, 1420.0 tps, lat 68.893 ms stddev 80.277 2024-01-11T18:00:44.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:43 smithi016 ceph-mon[43195]: pgmap v387: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-01-11T18:00:45.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:45 smithi016 ceph-mon[43195]: pgmap v388: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 1.69k op/s 2024-01-11T18:00:45.774 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:45 smithi038 ceph-mon[43582]: pgmap v388: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 1.69k op/s 2024-01-11T18:00:45.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:45 smithi007 ceph-mon[33361]: pgmap v388: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 1.69k op/s 2024-01-11T18:00:47.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:47 smithi016 ceph-mon[43195]: pgmap v389: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-01-11T18:00:48.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:47 smithi038 ceph-mon[43582]: pgmap v389: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-01-11T18:00:48.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:47 smithi007 ceph-mon[33361]: pgmap v389: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-01-11T18:00:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/2264086912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0104800 msgr2=0x7fa9c0106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/2264086912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0104800 0x7fa9c0106bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40098f0 tx=0x7fa9b4009c00 comp rx=0 tx=0).stop 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/2264086912 shutdown_connections 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/2264086912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c0107130 0x7fa9c0109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/2264086912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c01042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/2264086912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0104800 0x7fa9c0106bf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/2264086912 >> 172.21.15.7:0/2264086912 conn(0x7fa9c00fb820 msgr2=0x7fa9c00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/2264086912 shutdown_connections 2024-01-11T18:00:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.389+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/2264086912 wait complete. 2024-01-11T18:00:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.390+0000 7fa9c4b9d700 1 Processor -- start 2024-01-11T18:00:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 -- start start 2024-01-11T18:00:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c0104800 0x7fa9c0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0107130 0x7fa9c01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c01a1080 con 0x7fa9c0101ed0 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c01a11c0 con 0x7fa9c0104800 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9c4b9d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c01a1350 con 0x7fa9c0107130 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c0199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36824/0 (socket says 172.21.15.7:36824) 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/708842665 learned_addr learned my addr 172.21.15.7:0/708842665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bffff700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0107130 0x7fa9c01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9beffd700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c0104800 0x7fa9c0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/708842665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0107130 msgr2=0x7fa9c01a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0107130 0x7fa9c01a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/708842665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c0104800 msgr2=0x7fa9c0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.391+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c0104800 0x7fa9c0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/708842665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4009520 con 0x7fa9c0101ed0 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c0199500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000e9c0 tx=0x7fa9b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9bcff9700 1 -- 172.21.15.7:0/708842665 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0004a30 con 0x7fa9c0101ed0 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9bcff9700 1 -- 172.21.15.7:0/708842665 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b000cc80 con 0x7fa9c0101ed0 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c01a1630 con 0x7fa9c0101ed0 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9bcff9700 1 -- 172.21.15.7:0/708842665 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00054e0 con 0x7fa9c0101ed0 2024-01-11T18:00:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.392+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c01a1b40 con 0x7fa9c0101ed0 2024-01-11T18:00:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.395+0000 7fa9bcff9700 1 -- 172.21.15.7:0/708842665 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa9b0004b90 con 0x7fa9c0101ed0 2024-01-11T18:00:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.396+0000 7fa9d9aafb80 1 Processor -- start 2024-01-11T18:00:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 -- start start 2024-01-11T18:00:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca130c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca130d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130d870 0x55bca1314100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca1203030 con 0x55bca130c120 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca1266ac0 con 0x55bca130ceb0 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca12643e0 con 0x55bca130d870 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca130c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca130d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca130c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36832/0 (socket says 172.21.15.7:36832) 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9bffff700 1 -- 172.21.15.7:0/494517358 learned_addr learned my addr 172.21.15.7:0/494517358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.397+0000 7fa9beffd700 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130d870 0x55bca1314100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9bcff9700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9a80a1ec0 0x7fa9a80a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9beffd700 1 -- 172.21.15.7:0/494517358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 msgr2=0x55bca130d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9beffd700 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca130d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9beffd700 1 -- 172.21.15.7:0/494517358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 msgr2=0x55bca130c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9beffd700 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca130c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9beffd700 1 -- 172.21.15.7:0/494517358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca10d4350 con 0x55bca130d870 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9bcff9700 1 -- 172.21.15.7:0/708842665 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa9b0014070 con 0x7fa9c0101ed0 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9bffff700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9a80a1ec0 0x7fa9a80a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9beffd700 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130d870 0x55bca1314100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009810 tx=0x7fa9b4005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa54ba1897d473b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.398+0000 7fa9a5ffb700 1 -- 172.21.15.7:0/494517358 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400f6e0 con 0x55bca130d870 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9a5ffb700 1 -- 172.21.15.7:0/494517358 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b401d040 con 0x55bca130d870 2024-01-11T18:00:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9a5ffb700 1 -- 172.21.15.7:0/494517358 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40055a0 con 0x55bca130d870 2024-01-11T18:00:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/494517358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130d870 msgr2=0x55bca1314100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9bffff700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9a80a1ec0 0x7fa9a80a4380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55bca130bec0 tx=0x7fa9b800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130d870 0x55bca1314100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009810 tx=0x7fa9b4005dc0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/494517358 shutdown_connections 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130d870 0x55bca1314100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca130c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/494517358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca130d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/494517358 >> 172.21.15.7:0/494517358 conn(0x55bca1281740 msgr2=0x55bca1306b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/494517358 shutdown_connections 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.399+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/494517358 wait complete. 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.400+0000 7fa9d9aafb80 1 -- start start 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.400+0000 7fa9d9aafb80 1 Processor -- start 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.400+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca1315ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.400+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca1316010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.400+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130d870 0x55bca131d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca125fcd0 con 0x55bca130d870 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca11ce340 con 0x55bca130ceb0 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca11cdfe0 con 0x55bca130c120 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9beffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca1316010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9d9aafb80 10 client.0 _ll_register_callbacks cb 0x55bca12006c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca1315ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130d870 0x55bca131d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9beffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca1316010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33904/0 (socket says 172.21.15.7:33904) 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9beffd700 1 -- 172.21.15.7:0/2609801144 learned_addr learned my addr 172.21.15.7:0/2609801144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 msgr2=0x55bca1316010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca1316010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/2609801144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130d870 msgr2=0x55bca131d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130d870 0x55bca131d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/2609801144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c010a130 con 0x55bca130c120 2024-01-11T18:00:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9beffd700 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca1316010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca1315ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0097210 tx=0x7fa9b0095210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d3bbf5fe64e6c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.401+0000 7fa99effd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa99effd700 1 -- 172.21.15.7:0/2609801144 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b009a2e0 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bca12fefb0 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa99effd700 1 -- 172.21.15.7:0/2609801144 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa9b000cdf0 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa99effd700 1 -- 172.21.15.7:0/2609801144 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b009a100 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 10 client.25909 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bca1080b00 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa99effd700 1 -- 172.21.15.7:0/2609801144 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bca12fefb0 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa99effd700 1 -- 172.21.15.7:0/2609801144 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa9b0016330 con 0x55bca130c120 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 10 client.25909 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 20 client.25909 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13218f0 0x55bca1323d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 4 client.25909 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.402+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bca1258b60 con 0x55bca13218f0 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.403+0000 7fa9beffd700 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13218f0 0x55bca1323d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.403+0000 7fa9beffd700 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13218f0 0x55bca1323d10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc99248821f9dbba server_cookie=ba87acb2589a0f9f in_seq=0 out_seq=0 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.403+0000 7fa99effd700 10 client.25909 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa99effd700 1 -- 172.21.15.7:0/2609801144 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bca1258b60 con 0x55bca13218f0 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa99effd700 10 client.25909 handle_command_reply: tid=0 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 client.25909 shutdown 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 msgr2=0x55bca1315ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca1315ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0097210 tx=0x7fa9b0095210 comp rx=0 tx=0).stop 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 shutdown_connections 2024-01-11T18:00:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13218f0 0x55bca1323d10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca1315ad0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa99effd700 0 client.25909 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130d870 0x55bca131d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/2609801144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130ceb0 0x55bca1316010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 >> 172.21.15.7:0/2609801144 conn(0x55bca1281740 msgr2=0x55bca13061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 shutdown_connections 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.404+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/2609801144 wait complete. 2024-01-11T18:00:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.405+0000 7fa9d9aafb80 20 client.25909 trim_cache size 0 max 16384 2024-01-11T18:00:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.442+0000 7fa9d9aafb80 1 Processor -- start 2024-01-11T18:00:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 -- start start 2024-01-11T18:00:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca12feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130ceb0 0x55bca131d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca1302c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca10d6640 con 0x55bca130ceb0 2024-01-11T18:00:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca1310330 con 0x55bca130d870 2024-01-11T18:00:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.443+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca1313b20 con 0x55bca130c120 2024-01-11T18:00:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.444+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca1302c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.444+0000 7fa9beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130ceb0 0x55bca131d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.444+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca1302c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33916/0 (socket says 172.21.15.7:33916) 2024-01-11T18:00:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.444+0000 7fa9bffff700 1 -- 172.21.15.7:0/1548406775 learned_addr learned my addr 172.21.15.7:0/1548406775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.445+0000 7fa9beffd700 1 -- 172.21.15.7:0/1548406775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 msgr2=0x55bca12feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.445+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca12feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.445+0000 7fa9beffd700 1 -- 172.21.15.7:0/1548406775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 msgr2=0x55bca1302c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.445+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca1302c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.445+0000 7fa9beffd700 1 -- 172.21.15.7:0/1548406775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca12211d0 con 0x55bca130ceb0 2024-01-11T18:00:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.445+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130ceb0 0x55bca131d010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400fb70 tx=0x7fa9b4009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6635dc631b5b570 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.446+0000 7fa9a57fa700 1 -- 172.21.15.7:0/1548406775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4005440 con 0x55bca130ceb0 2024-01-11T18:00:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.446+0000 7fa9a57fa700 1 -- 172.21.15.7:0/1548406775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b400ba20 con 0x55bca130ceb0 2024-01-11T18:00:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.446+0000 7fa9a57fa700 1 -- 172.21.15.7:0/1548406775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4003c10 con 0x55bca130ceb0 2024-01-11T18:00:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.447+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1548406775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130ceb0 msgr2=0x55bca131d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.447+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130ceb0 0x55bca131d010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400fb70 tx=0x7fa9b4009f50 comp rx=0 tx=0).stop 2024-01-11T18:00:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1548406775 shutdown_connections 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130c120 0x55bca12feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca1302c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1548406775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130ceb0 0x55bca131d010 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1548406775 >> 172.21.15.7:0/1548406775 conn(0x55bca1281740 msgr2=0x55bca13061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1548406775 shutdown_connections 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.448+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1548406775 wait complete. 2024-01-11T18:00:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.449+0000 7fa9d9aafb80 1 -- start start 2024-01-11T18:00:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.451+0000 7fa9d9aafb80 1 Processor -- start 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca1319e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130ceb0 0x55bca131a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca137e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca131aa90 con 0x55bca130c120 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca137eb40 con 0x55bca130d870 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca1371dd0 con 0x55bca130ceb0 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.452+0000 7fa9d9aafb80 10 client.0 _ll_register_callbacks cb 0x55bca12006c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:00:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.453+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca1319e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.454+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca1319e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36868/0 (socket says 172.21.15.7:36868) 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.454+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1672390449 learned_addr learned my addr 172.21.15.7:0/1672390449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.455+0000 7fa9bffff700 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca137e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.455+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1672390449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130ceb0 msgr2=0x55bca131a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.455+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130ceb0 0x55bca131a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.455+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 msgr2=0x55bca137e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.455+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca137e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.455+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1672390449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bca10d4350 con 0x55bca130c120 2024-01-11T18:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.456+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca1319e70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0096ca0 tx=0x7fa9b00cb840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f632e839379c27a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.456+0000 7fa99effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.456+0000 7fa99effd700 1 -- 172.21.15.7:0/1672390449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0096ae0 con 0x55bca130c120 2024-01-11T18:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.456+0000 7fa99effd700 1 -- 172.21.15.7:0/1672390449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa9b000cdf0 con 0x55bca130c120 2024-01-11T18:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.457+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9b401d040 con 0x55bca130c120 2024-01-11T18:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.457+0000 7fa99effd700 1 -- 172.21.15.7:0/1672390449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b009a2e0 con 0x55bca130c120 2024-01-11T18:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.457+0000 7fa9d9aafb80 10 client.15816 fetch_fsmap learned FSMap version 29 2024-01-11T18:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.457+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bca10d8c70 con 0x55bca130c120 2024-01-11T18:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.458+0000 7fa99effd700 1 -- 172.21.15.7:0/1672390449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b401d040 con 0x55bca130c120 2024-01-11T18:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.458+0000 7fa99effd700 1 -- 172.21.15.7:0/1672390449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa9b00b5030 con 0x55bca130c120 2024-01-11T18:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.459+0000 7fa9d9aafb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.459+0000 7fa9d9aafb80 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.459+0000 7fa9d9aafb80 20 client.15816 populate_metadata read hostname 'smithi007' 2024-01-11T18:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.459+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13733d0 0x55bca13757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.459+0000 7fa9d9aafb80 4 client.15816 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.459+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bca1258880 con 0x55bca13733d0 2024-01-11T18:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.462+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13733d0 0x55bca13757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.463+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13733d0 0x55bca13757f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af50c44d01e4f41 server_cookie=21ab314955b53fef in_seq=0 out_seq=0 2024-01-11T18:00:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.463+0000 7fa99effd700 10 client.15816 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.786+0000 7fa99effd700 1 -- 172.21.15.7:0/1672390449 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bca1258880 con 0x55bca13733d0 2024-01-11T18:00:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.786+0000 7fa99effd700 10 client.15816 handle_command_reply: tid=0 2024-01-11T18:00:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.786+0000 7fa9d9aafb80 1 client.15816 shutdown 2024-01-11T18:00:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 msgr2=0x55bca1319e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca1319e70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0096ca0 tx=0x7fa9b00cb840 comp rx=0 tx=0).stop 2024-01-11T18:00:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 shutdown_connections 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bca13733d0 0x55bca13757f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa99effd700 0 client.15816 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca130d870 0x55bca137e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca130c120 0x55bca1319e70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 --2- 172.21.15.7:0/1672390449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bca130ceb0 0x55bca131a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 >> 172.21.15.7:0/1672390449 conn(0x55bca1281740 msgr2=0x55bca1302c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 shutdown_connections 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 1 -- 172.21.15.7:0/1672390449 wait complete. 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.787+0000 7fa9d9aafb80 20 client.15816 trim_cache size 0 max 16384 2024-01-11T18:00:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:00:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:00:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:00:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:00:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.788+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9a80a1ec0 msgr2=0x7fa9a80a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.788+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9a80a1ec0 0x7fa9a80a4380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55bca130bec0 tx=0x7fa9b800b040 comp rx=0 tx=0).stop 2024-01-11T18:00:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.788+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 msgr2=0x7fa9c0199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.788+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c0199500 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000e9c0 tx=0x7fa9b000ecd0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 shutdown_connections 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa9a80a1ec0 0x7fa9a80a4380 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c0107130 0x7fa9c01a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c0101ed0 0x7fa9c0199500 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 --2- 172.21.15.7:0/708842665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c0104800 0x7fa9c0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 >> 172.21.15.7:0/708842665 conn(0x7fa9c00fb820 msgr2=0x7fa9c01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 shutdown_connections 2024-01-11T18:00:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:00:48.789+0000 7fa9c4b9d700 1 -- 172.21.15.7:0/708842665 wait complete. 2024-01-11T18:00:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 140.0 s, 1336.2 tps, lat 76.052 ms stddev 92.766 2024-01-11T18:00:49.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:49 smithi016 ceph-mon[43195]: pgmap v390: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.34k op/s 2024-01-11T18:00:50.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:49 smithi038 ceph-mon[43582]: pgmap v390: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.34k op/s 2024-01-11T18:00:50.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:49 smithi007 ceph-mon[33361]: pgmap v390: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.34k op/s 2024-01-11T18:00:52.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:51 smithi038 ceph-mon[43582]: pgmap v391: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-01-11T18:00:52.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:51 smithi007 ceph-mon[33361]: pgmap v391: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-01-11T18:00:52.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:51 smithi016 ceph-mon[43195]: pgmap v391: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-01-11T18:00:54.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:53 smithi038 ceph-mon[43582]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 813 op/s 2024-01-11T18:00:54.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:53 smithi007 ceph-mon[33361]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 813 op/s 2024-01-11T18:00:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 145.0 s, 1070.0 tps, lat 92.897 ms stddev 119.124 2024-01-11T18:00:54.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:53 smithi016 ceph-mon[43195]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 813 op/s 2024-01-11T18:00:55.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:55 smithi016 ceph-mon[43195]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 569 op/s 2024-01-11T18:00:55.774 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:55 smithi038 ceph-mon[43582]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 569 op/s 2024-01-11T18:00:55.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:55 smithi007 ceph-mon[33361]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 569 op/s 2024-01-11T18:00:58.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:57 smithi038 ceph-mon[43582]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 669 op/s 2024-01-11T18:00:58.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:57 smithi007 ceph-mon[33361]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 669 op/s 2024-01-11T18:00:58.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:57 smithi016 ceph-mon[43195]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 669 op/s 2024-01-11T18:00:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 150.0 s, 1227.6 tps, lat 81.707 ms stddev 100.438 2024-01-11T18:01:00.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:00:59 smithi007 ceph-mon[33361]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 601 op/s 2024-01-11T18:01:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:00:59 smithi016 ceph-mon[43195]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 601 op/s 2024-01-11T18:01:00.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:00:59 smithi038 ceph-mon[43582]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 601 op/s 2024-01-11T18:01:02.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:01 smithi038 ceph-mon[43582]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-01-11T18:01:02.041 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:01 smithi007 ceph-mon[33361]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.062+0000 7f345e7ff700 1 -- 172.21.15.7:0/1761773002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ff280 msgr2=0x7f34580ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.062+0000 7f345e7ff700 1 --2- 172.21.15.7:0/1761773002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ff280 0x7f34580ff700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f344c009990 tx=0x7f344c009ca0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.063+0000 7f345e7ff700 1 -- 172.21.15.7:0/1761773002 shutdown_connections 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.063+0000 7f345e7ff700 1 --2- 172.21.15.7:0/1761773002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34580ffc40 0x7f3458105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.063+0000 7f345e7ff700 1 --2- 172.21.15.7:0/1761773002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581007b0 0x7f3458100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.063+0000 7f345e7ff700 1 --2- 172.21.15.7:0/1761773002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ff280 0x7f34580ff700 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f344c009990 tx=0x7f344c009ca0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.063+0000 7f345e7ff700 1 -- 172.21.15.7:0/1761773002 >> 172.21.15.7:0/1761773002 conn(0x7f34580fad60 msgr2=0x7f34580fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.065+0000 7f345e7ff700 1 -- 172.21.15.7:0/1761773002 shutdown_connections 2024-01-11T18:01:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.065+0000 7f345e7ff700 1 -- 172.21.15.7:0/1761773002 wait complete. 2024-01-11T18:01:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.067+0000 7f345e7ff700 1 Processor -- start 2024-01-11T18:01:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.068+0000 7f345e7ff700 1 -- start start 2024-01-11T18:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.068+0000 7f345e7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ffc40 0x7f34581b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345e7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34581007b0 0x7f34581b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 0x7f34581beec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581bf490 con 0x7f34581bcad0 2024-01-11T18:01:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345e7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581bf5d0 con 0x7f34580ffc40 2024-01-11T18:01:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345e7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581bf760 con 0x7f34581007b0 2024-01-11T18:01:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 0x7f34581beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 0x7f34581beec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46516/0 (socket says 172.21.15.7:46516) 2024-01-11T18:01:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.069+0000 7f345cd9c700 1 -- 172.21.15.7:0/498745368 learned_addr learned my addr 172.21.15.7:0/498745368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.071+0000 7f345cd9c700 1 -- 172.21.15.7:0/498745368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34581007b0 msgr2=0x7f34581b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.071+0000 7f345cd9c700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34581007b0 0x7f34581b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.071+0000 7f345cd9c700 1 -- 172.21.15.7:0/498745368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ffc40 msgr2=0x7f34581b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.071+0000 7f345cd9c700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ffc40 0x7f34581b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.071+0000 7f345cd9c700 1 -- 172.21.15.7:0/498745368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c0095f0 con 0x7f34581bcad0 2024-01-11T18:01:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.073+0000 7f345cd9c700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 0x7f34581beec0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3450007d70 tx=0x7f345000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.073+0000 7f34557fa700 1 -- 172.21.15.7:0/498745368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345000ad40 con 0x7f34581bcad0 2024-01-11T18:01:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.073+0000 7f34557fa700 1 -- 172.21.15.7:0/498745368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34500128e0 con 0x7f34581bcad0 2024-01-11T18:01:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.073+0000 7f34557fa700 1 -- 172.21.15.7:0/498745368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450011780 con 0x7f34581bcad0 2024-01-11T18:01:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.073+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34581bfa40 con 0x7f34581bcad0 2024-01-11T18:01:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.074+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34581bff80 con 0x7f34581bcad0 2024-01-11T18:01:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.080+0000 7f34724bab80 1 Processor -- start 2024-01-11T18:01:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.090+0000 7f34557fa700 1 -- 172.21.15.7:0/498745368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f3450012420 con 0x7f34581bcad0 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.092+0000 7f34724bab80 1 -- start start 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.092+0000 7f34724bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 0x5623ddffc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.092+0000 7f34724bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623ddffd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.092+0000 7f34724bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffd5d0 0x5623de003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.093+0000 7f34724bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddef3030 con 0x5623ddffbf20 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.093+0000 7f34724bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddf56ac0 con 0x5623ddffd5d0 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.093+0000 7f34724bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddf543e0 con 0x5623ddffcc10 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.093+0000 7f3457fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623ddffd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.093+0000 7f3457fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623ddffd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:57542/0 (socket says 172.21.15.7:57542) 2024-01-11T18:01:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.093+0000 7f3457fff700 1 -- 172.21.15.7:0/2297668964 learned_addr learned my addr 172.21.15.7:0/2297668964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f34557fa700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34400a1f50 0x7f34400a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f3457fff700 1 -- 172.21.15.7:0/2297668964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffd5d0 msgr2=0x5623de003e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f3457fff700 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffd5d0 0x5623de003e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f34557fa700 1 -- 172.21.15.7:0/498745368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f34500cdf80 con 0x7f34581bcad0 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f3457fff700 1 -- 172.21.15.7:0/2297668964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 msgr2=0x5623ddffc340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f3457fff700 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 0x5623ddffc340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.094+0000 7f3457fff700 1 -- 172.21.15.7:0/2297668964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623dddc4350 con 0x5623ddffcc10 2024-01-11T18:01:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.095+0000 7f345cd9c700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34400a1f50 0x7f34400a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.096+0000 7f3457fff700 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623ddffd090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f34480098a0 tx=0x7f3448009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a314102c6815e52f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.096+0000 7f345cd9c700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34400a1f50 0x7f34400a4410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3450007d40 tx=0x7f3450012f20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.097+0000 7f343e7fc700 1 -- 172.21.15.7:0/2297668964 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800b870 con 0x5623ddffcc10 2024-01-11T18:01:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.097+0000 7f343e7fc700 1 -- 172.21.15.7:0/2297668964 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344800be90 con 0x5623ddffcc10 2024-01-11T18:01:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.097+0000 7f343e7fc700 1 -- 172.21.15.7:0/2297668964 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448010380 con 0x5623ddffcc10 2024-01-11T18:01:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 -- 172.21.15.7:0/2297668964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 msgr2=0x5623ddffd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623ddffd090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f34480098a0 tx=0x7f3448009bb0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 -- 172.21.15.7:0/2297668964 shutdown_connections 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffd5d0 0x5623de003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 0x5623ddffc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 --2- 172.21.15.7:0/2297668964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623ddffd090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 -- 172.21.15.7:0/2297668964 >> 172.21.15.7:0/2297668964 conn(0x5623ddf71740 msgr2=0x5623ddff6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 -- 172.21.15.7:0/2297668964 shutdown_connections 2024-01-11T18:01:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.099+0000 7f34724bab80 1 -- 172.21.15.7:0/2297668964 wait complete. 2024-01-11T18:01:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.100+0000 7f34724bab80 1 -- start start 2024-01-11T18:01:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.100+0000 7f34724bab80 1 Processor -- start 2024-01-11T18:01:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de0059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffcc10 0x5623de005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de00cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddf4fcd0 con 0x5623ddffcc10 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddebe340 con 0x5623ddffbf20 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddebdfe0 con 0x5623ddffd5d0 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34724bab80 10 client.0 _ll_register_callbacks cb 0x5623ddef06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34577fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de00cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.103+0000 7f34577fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de0059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.104+0000 7f34577fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de0059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42840/0 (socket says 172.21.15.7:42840) 2024-01-11T18:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.104+0000 7f34577fe700 1 -- 172.21.15.7:0/2758861461 learned_addr learned my addr 172.21.15.7:0/2758861461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.104+0000 7f34577fe700 1 -- 172.21.15.7:0/2758861461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 msgr2=0x5623de00cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.104+0000 7f34577fe700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de00cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.104+0000 7f3457fff700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffcc10 0x5623de005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.104+0000 7f34577fe700 1 -- 172.21.15.7:0/2758861461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffcc10 msgr2=0x5623de005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.105+0000 7f34577fe700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffcc10 0x5623de005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.105+0000 7f34577fe700 1 -- 172.21.15.7:0/2758861461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34480094d0 con 0x5623ddffbf20 2024-01-11T18:01:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.105+0000 7f3457fff700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffcc10 0x5623de005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.106+0000 7f34577fe700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de0059f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f344c0036f0 tx=0x7f344c0037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33690f999f8493a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.106+0000 7f34377fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.106+0000 7f34377fe700 1 -- 172.21.15.7:0/2758861461 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c014270 con 0x5623ddffbf20 2024-01-11T18:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.106+0000 7f34377fe700 1 -- 172.21.15.7:0/2758861461 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f344c003ab0 con 0x5623ddffbf20 2024-01-11T18:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.106+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623ddfeed60 con 0x5623ddffbf20 2024-01-11T18:01:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.107+0000 7f34377fe700 1 -- 172.21.15.7:0/2758861461 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f344c003c70 con 0x5623ddffbf20 2024-01-11T18:01:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.107+0000 7f34724bab80 10 client.25883 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.107+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5623ddd70b00 con 0x5623ddffbf20 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34377fe700 1 -- 172.21.15.7:0/2758861461 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344c02c080 con 0x5623ddffbf20 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34377fe700 1 -- 172.21.15.7:0/2758861461 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f344c014610 con 0x5623ddffbf20 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34724bab80 10 client.25883 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34724bab80 10 client.25883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34724bab80 20 client.25883 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34724bab80 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de011830 0x5623de013c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34724bab80 4 client.25883 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:01:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.109+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623ddf48b60 con 0x5623de011830 2024-01-11T18:01:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.111+0000 7f3457fff700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de011830 0x5623de013c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.112+0000 7f3457fff700 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de011830 0x5623de013c50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46b07b50be53349 server_cookie=649134e6600cabe2 in_seq=0 out_seq=0 2024-01-11T18:01:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.113+0000 7f34377fe700 10 client.25883 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.113+0000 7f34377fe700 1 -- 172.21.15.7:0/2758861461 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623ddf48b60 con 0x5623de011830 2024-01-11T18:01:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.113+0000 7f34377fe700 10 client.25883 handle_command_reply: tid=0 2024-01-11T18:01:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.113+0000 7f34724bab80 1 client.25883 shutdown 2024-01-11T18:01:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.115+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 msgr2=0x5623de0059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.115+0000 7f34724bab80 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de0059f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f344c0036f0 tx=0x7f344c0037c0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.115+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 shutdown_connections 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.115+0000 7f34724bab80 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de011830 0x5623de013c50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.115+0000 7f34724bab80 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de00cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.115+0000 7f34724bab80 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de0059f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.116+0000 7f34724bab80 1 --2- 172.21.15.7:0/2758861461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffcc10 0x5623de005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.116+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 >> 172.21.15.7:0/2758861461 conn(0x5623ddf71740 msgr2=0x5623ddff5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.116+0000 7f34377fe700 0 client.25883 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.118+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 shutdown_connections 2024-01-11T18:01:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.118+0000 7f34724bab80 1 -- 172.21.15.7:0/2758861461 wait complete. 2024-01-11T18:01:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.119+0000 7f34724bab80 20 client.25883 trim_cache size 0 max 16384 2024-01-11T18:01:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 Processor -- start 2024-01-11T18:01:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 -- start start 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 0x5623de004f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 0x5623de00cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de0185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623dddc6640 con 0x5623ddffbf20 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623de00ec80 con 0x5623ddffcc10 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.151+0000 7f34724bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddff7cd0 con 0x5623ddffd5d0 2024-01-11T18:01:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 0x5623de00cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 0x5623de00cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42844/0 (socket says 172.21.15.7:42844) 2024-01-11T18:01:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 -- 172.21.15.7:0/3477107714 learned_addr learned my addr 172.21.15.7:0/3477107714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f34577fe700 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de0185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 -- 172.21.15.7:0/3477107714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 msgr2=0x5623de0185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de0185d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 -- 172.21.15.7:0/3477107714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 msgr2=0x5623de004f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 0x5623de004f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.152+0000 7f3457fff700 1 -- 172.21.15.7:0/3477107714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623ddf111d0 con 0x5623ddffcc10 2024-01-11T18:01:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.153+0000 7f3457fff700 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 0x5623de00cf30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3448000c00 tx=0x7f3448009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9905bbe22ad0a62 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.153+0000 7f343dffb700 1 -- 172.21.15.7:0/3477107714 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480114a0 con 0x5623ddffcc10 2024-01-11T18:01:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.154+0000 7f343dffb700 1 -- 172.21.15.7:0/3477107714 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3448023860 con 0x5623ddffcc10 2024-01-11T18:01:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.154+0000 7f343dffb700 1 -- 172.21.15.7:0/3477107714 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800b8b0 con 0x5623ddffcc10 2024-01-11T18:01:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 -- 172.21.15.7:0/3477107714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 msgr2=0x5623de00cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 0x5623de00cf30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3448000c00 tx=0x7f3448009240 comp rx=0 tx=0).stop 2024-01-11T18:01:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 -- 172.21.15.7:0/3477107714 shutdown_connections 2024-01-11T18:01:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffd5d0 0x5623de0185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffbf20 0x5623de004f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 --2- 172.21.15.7:0/3477107714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffcc10 0x5623de00cf30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.155+0000 7f34724bab80 1 -- 172.21.15.7:0/3477107714 >> 172.21.15.7:0/3477107714 conn(0x5623ddf71740 msgr2=0x5623ddff29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.156+0000 7f34724bab80 1 -- 172.21.15.7:0/3477107714 shutdown_connections 2024-01-11T18:01:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.156+0000 7f34724bab80 1 -- 172.21.15.7:0/3477107714 wait complete. 2024-01-11T18:01:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.156+0000 7f34724bab80 1 -- start start 2024-01-11T18:01:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.157+0000 7f34724bab80 1 Processor -- start 2024-01-11T18:01:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de00a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623de00a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffd5d0 0x5623de00b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ddffcad0 con 0x5623ddffd5d0 2024-01-11T18:01:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623de00bb50 con 0x5623ddffbf20 2024-01-11T18:01:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623de00bce0 con 0x5623ddffcc10 2024-01-11T18:01:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.159+0000 7f34724bab80 10 client.0 _ll_register_callbacks cb 0x5623de00db40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f34577fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de00a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f34577fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de00a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42846/0 (socket says 172.21.15.7:42846) 2024-01-11T18:01:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f34577fe700 1 -- 172.21.15.7:0/521736166 learned_addr learned my addr 172.21.15.7:0/521736166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f3457fff700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623de00a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f3457fff700 1 -- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 msgr2=0x5623de00a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f3457fff700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de00a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f3457fff700 1 -- 172.21.15.7:0/521736166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffd5d0 msgr2=0x5623de00b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f3457fff700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffd5d0 0x5623de00b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.160+0000 7f3457fff700 1 -- 172.21.15.7:0/521736166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623dddc4350 con 0x5623ddffcc10 2024-01-11T18:01:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.161+0000 7f34577fe700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de00a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:01:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.162+0000 7f3457fff700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623de00a600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5623ddf4baf0 tx=0x7f3448023c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75b54231c77fa91 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.163+0000 7f34377fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:01:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.163+0000 7f34377fe700 1 -- 172.21.15.7:0/521736166 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448035070 con 0x5623ddffcc10 2024-01-11T18:01:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.163+0000 7f34377fe700 1 -- 172.21.15.7:0/521736166 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3448020040 con 0x5623ddffcc10 2024-01-11T18:01:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.164+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f344800be90 con 0x5623ddffcc10 2024-01-11T18:01:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.164+0000 7f34377fe700 1 -- 172.21.15.7:0/521736166 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34480114a0 con 0x5623ddffcc10 2024-01-11T18:01:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.164+0000 7f34724bab80 10 client.25921 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.164+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5623dddc8c70 con 0x5623ddffcc10 2024-01-11T18:01:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.165+0000 7f34377fe700 1 -- 172.21.15.7:0/521736166 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344800be90 con 0x5623ddffcc10 2024-01-11T18:01:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.165+0000 7f34377fe700 1 -- 172.21.15.7:0/521736166 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3448038030 con 0x5623ddffcc10 2024-01-11T18:01:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34724bab80 10 client.25921 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34724bab80 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34724bab80 20 client.25921 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34724bab80 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de067d40 0x5623de06a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34724bab80 4 client.25921 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:01:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623ddf48880 con 0x5623de067d40 2024-01-11T18:01:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.166+0000 7f34577fe700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de067d40 0x5623de06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.167+0000 7f34377fe700 10 client.25921 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.167+0000 7f34577fe700 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de067d40 0x5623de06a130 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc6d2469914e6c9 server_cookie=9cd2c625b0552564 in_seq=0 out_seq=0 2024-01-11T18:01:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.181+0000 7f34377fe700 1 -- 172.21.15.7:0/521736166 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623ddf48880 con 0x5623de067d40 2024-01-11T18:01:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.181+0000 7f34377fe700 10 client.25921 handle_command_reply: tid=0 2024-01-11T18:01:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.182+0000 7f34724bab80 1 client.25921 shutdown 2024-01-11T18:01:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 msgr2=0x5623de00a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623de00a600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5623ddf4baf0 tx=0x7f3448023c70 comp rx=0 tx=0).stop 2024-01-11T18:01:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 shutdown_connections 2024-01-11T18:01:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5623de067d40 0x5623de06a130 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5623ddffd5d0 0x5623de00b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5623ddffbf20 0x5623de00a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 --2- 172.21.15.7:0/521736166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623ddffcc10 0x5623de00a600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34377fe700 0 client.25921 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.183+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 >> 172.21.15.7:0/521736166 conn(0x5623ddf71740 msgr2=0x5623de004f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.184+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 shutdown_connections 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.184+0000 7f34724bab80 1 -- 172.21.15.7:0/521736166 wait complete. 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.184+0000 7f34724bab80 20 client.25921 trim_cache size 0 max 16384 2024-01-11T18:01:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:01:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:01:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:01:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:01:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34400a1f50 msgr2=0x7f34400a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34400a1f50 0x7f34400a4410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3450007d40 tx=0x7f3450012f20 comp rx=0 tx=0).stop 2024-01-11T18:01:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 msgr2=0x7f34581beec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 0x7f34581beec0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3450007d70 tx=0x7f345000a340 comp rx=0 tx=0).stop 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 shutdown_connections 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f34400a1f50 0x7f34400a4410 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34581bcad0 0x7f34581beec0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34580ffc40 0x7f34581b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 --2- 172.21.15.7:0/498745368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34581007b0 0x7f34581b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 >> 172.21.15.7:0/498745368 conn(0x7f34580fad60 msgr2=0x7f3458101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 shutdown_connections 2024-01-11T18:01:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:02.185+0000 7f345e7ff700 1 -- 172.21.15.7:0/498745368 wait complete. 2024-01-11T18:01:02.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:01 smithi016 ceph-mon[43195]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-01-11T18:01:04.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:03 smithi007 ceph-mon[33361]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-01-11T18:01:04.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 155.0 s, 1071.4 tps, lat 93.458 ms stddev 110.450 2024-01-11T18:01:04.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:03 smithi016 ceph-mon[43195]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-01-11T18:01:04.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:03 smithi038 ceph-mon[43582]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-01-11T18:01:05.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:05 smithi016 ceph-mon[43195]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.6 MiB/s wr, 607 op/s 2024-01-11T18:01:05.774 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:05 smithi038 ceph-mon[43582]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.6 MiB/s wr, 607 op/s 2024-01-11T18:01:05.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:05 smithi007 ceph-mon[33361]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.6 MiB/s wr, 607 op/s 2024-01-11T18:01:08.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:07 smithi007 ceph-mon[33361]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 703 op/s 2024-01-11T18:01:08.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:07 smithi016 ceph-mon[43195]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 703 op/s 2024-01-11T18:01:08.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:07 smithi038 ceph-mon[43582]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 703 op/s 2024-01-11T18:01:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 160.0 s, 1149.0 tps, lat 86.242 ms stddev 97.311 2024-01-11T18:01:10.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:09 smithi007 ceph-mon[33361]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 MiB/s wr, 606 op/s 2024-01-11T18:01:10.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:09 smithi016 ceph-mon[43195]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 MiB/s wr, 606 op/s 2024-01-11T18:01:10.280 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:09 smithi038 ceph-mon[43582]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 MiB/s wr, 606 op/s 2024-01-11T18:01:11.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:10 smithi016 ceph-mon[43195]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.5 MiB/s wr, 638 op/s 2024-01-11T18:01:11.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:10 smithi038 ceph-mon[43582]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.5 MiB/s wr, 638 op/s 2024-01-11T18:01:11.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:10 smithi007 ceph-mon[33361]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.5 MiB/s wr, 638 op/s 2024-01-11T18:01:14.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:13 smithi007 ceph-mon[33361]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.0 MiB/s wr, 669 op/s 2024-01-11T18:01:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 165.0 s, 1059.4 tps, lat 95.341 ms stddev 119.552 2024-01-11T18:01:14.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:13 smithi016 ceph-mon[43195]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.0 MiB/s wr, 669 op/s 2024-01-11T18:01:14.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:13 smithi038 ceph-mon[43582]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.0 MiB/s wr, 669 op/s 2024-01-11T18:01:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.440+0000 7f6ac99ff700 1 -- 172.21.15.7:0/2364634288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4102030 msgr2=0x7f6ac41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.440+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/2364634288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4102030 0x7f6ac41088e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00f560 tx=0x7f6abc00f870 comp rx=0 tx=0).stop 2024-01-11T18:01:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.442+0000 7f6ac99ff700 1 -- 172.21.15.7:0/2364634288 shutdown_connections 2024-01-11T18:01:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.442+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/2364634288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ac41004a0 0x7f6ac41008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.442+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/2364634288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4102030 0x7f6ac41088e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.442+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/2364634288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4101690 0x7f6ac4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.442+0000 7f6ac99ff700 1 -- 172.21.15.7:0/2364634288 >> 172.21.15.7:0/2364634288 conn(0x7f6ac40fba30 msgr2=0x7f6ac40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.443+0000 7f6ac99ff700 1 -- 172.21.15.7:0/2364634288 shutdown_connections 2024-01-11T18:01:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.443+0000 7f6ac99ff700 1 -- 172.21.15.7:0/2364634288 wait complete. 2024-01-11T18:01:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.445+0000 7f6ac99ff700 1 Processor -- start 2024-01-11T18:01:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.445+0000 7f6ac99ff700 1 -- start start 2024-01-11T18:01:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.445+0000 7f6ac99ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ac41004a0 0x7f6ac41954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.445+0000 7f6ac99ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4101690 0x7f6ac41959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.445+0000 7f6ac99ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4102030 0x7f6ac419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac99ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac419d020 con 0x7f6ac4101690 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac99ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac419d160 con 0x7f6ac4102030 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac99ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac419d2f0 con 0x7f6ac41004a0 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4101690 0x7f6ac41959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4101690 0x7f6ac41959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58662/0 (socket says 172.21.15.7:58662) 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac27fc700 1 -- 172.21.15.7:0/3822906066 learned_addr learned my addr 172.21.15.7:0/3822906066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.446+0000 7f6ac37fe700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4102030 0x7f6ac419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.447+0000 7f6ac37fe700 1 -- 172.21.15.7:0/3822906066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ac41004a0 msgr2=0x7f6ac41954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.447+0000 7f6ac37fe700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ac41004a0 0x7f6ac41954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.447+0000 7f6ac37fe700 1 -- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4101690 msgr2=0x7f6ac41959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.447+0000 7f6ac37fe700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4101690 0x7f6ac41959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.447+0000 7f6ac37fe700 1 -- 172.21.15.7:0/3822906066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab800b5a0 con 0x7f6ac4102030 2024-01-11T18:01:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.447+0000 7f6ac37fe700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4102030 0x7f6ac419ca50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00fc00 tx=0x7f6abc017860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.449+0000 7f6ac89fd700 1 -- 172.21.15.7:0/3822906066 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00f2b0 con 0x7f6ac4102030 2024-01-11T18:01:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.449+0000 7f6ac89fd700 1 -- 172.21.15.7:0/3822906066 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6abc00d070 con 0x7f6ac4102030 2024-01-11T18:01:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.449+0000 7f6ac89fd700 1 -- 172.21.15.7:0/3822906066 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0075c0 con 0x7f6ac4102030 2024-01-11T18:01:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.450+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc009d00 con 0x7f6ac4102030 2024-01-11T18:01:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.450+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac419d810 con 0x7f6ac4102030 2024-01-11T18:01:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.462+0000 7f6ac89fd700 1 -- 172.21.15.7:0/3822906066 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6abc005110 con 0x7f6ac4102030 2024-01-11T18:01:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.462+0000 7f6add702b80 1 Processor -- start 2024-01-11T18:01:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.462+0000 7f6add702b80 1 -- start start 2024-01-11T18:01:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.463+0000 7f6add702b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6add702b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6add702b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c25d0 0x55b0483c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6add702b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0482b8030 con 0x55b0483c25d0 2024-01-11T18:01:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52146/0 (socket says 172.21.15.7:52146) 2024-01-11T18:01:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/1307233793 learned_addr learned my addr 172.21.15.7:0/1307233793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.465+0000 7f6ac89fd700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6aac0a2010 0x7f6aac0a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.465+0000 7f6ac89fd700 1 -- 172.21.15.7:0/3822906066 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6abc00f030 con 0x7f6ac4102030 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.464+0000 7f6add702b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04831bac0 con 0x55b0483c1c10 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.465+0000 7f6add702b80 1 -- 172.21.15.7:0/1307233793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0483193e0 con 0x55b0483c0f20 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.465+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.466+0000 7f6ac27fc700 1 -- 172.21.15.7:0/1307233793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 msgr2=0x55b0483c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.466+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.466+0000 7f6ac27fc700 1 -- 172.21.15.7:0/1307233793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c25d0 msgr2=0x55b0483c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.466+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c25d0 0x55b0483c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.466+0000 7f6ac27fc700 1 -- 172.21.15.7:0/1307233793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b048189350 con 0x55b0483c0f20 2024-01-11T18:01:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.466+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483c1340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8015ed0 tx=0x7f6ab8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f639deb749b6163f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.467+0000 7f6ac37fe700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6aac0a2010 0x7f6aac0a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.467+0000 7f6aa9ffb700 1 -- 172.21.15.7:0/1307233793 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8004030 con 0x55b0483c0f20 2024-01-11T18:01:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.467+0000 7f6aa9ffb700 1 -- 172.21.15.7:0/1307233793 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ab8003680 con 0x55b0483c0f20 2024-01-11T18:01:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.467+0000 7f6aa9ffb700 1 -- 172.21.15.7:0/1307233793 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8009840 con 0x55b0483c0f20 2024-01-11T18:01:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.468+0000 7f6add702b80 1 -- 172.21.15.7:0/1307233793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 msgr2=0x55b0483c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.468+0000 7f6add702b80 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483c1340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8015ed0 tx=0x7f6ab8009260 comp rx=0 tx=0).stop 2024-01-11T18:01:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.468+0000 7f6add702b80 1 -- 172.21.15.7:0/1307233793 shutdown_connections 2024-01-11T18:01:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.469+0000 7f6add702b80 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483c1340 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.469+0000 7f6add702b80 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c25d0 0x55b0483c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.469+0000 7f6add702b80 1 --2- 172.21.15.7:0/1307233793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.469+0000 7f6add702b80 1 -- 172.21.15.7:0/1307233793 >> 172.21.15.7:0/1307233793 conn(0x55b048336740 msgr2=0x55b0483bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.470+0000 7f6add702b80 1 -- 172.21.15.7:0/1307233793 shutdown_connections 2024-01-11T18:01:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.470+0000 7f6add702b80 1 -- 172.21.15.7:0/1307233793 wait complete. 2024-01-11T18:01:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.471+0000 7f6add702b80 1 -- start start 2024-01-11T18:01:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.471+0000 7f6add702b80 1 Processor -- start 2024-01-11T18:01:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.472+0000 7f6add702b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6add702b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6add702b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c25d0 0x55b0483d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6add702b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b048314cd0 con 0x55b0483c1c10 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6add702b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b048283340 con 0x55b0483c25d0 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6add702b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b048282fe0 con 0x55b0483c0f20 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6add702b80 10 client.0 _ll_register_callbacks cb 0x55b0482b56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6ac27fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6ac27fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50034/0 (socket says 172.21.15.7:50034) 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6ac27fc700 1 -- 172.21.15.7:0/2216937194 learned_addr learned my addr 172.21.15.7:0/2216937194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.473+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c25d0 0x55b0483d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.474+0000 7f6ac27fc700 1 -- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c25d0 msgr2=0x55b0483d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.474+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c25d0 0x55b0483d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.474+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.474+0000 7f6ac27fc700 1 -- 172.21.15.7:0/2216937194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 msgr2=0x55b0483caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.474+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.474+0000 7f6ac27fc700 1 -- 172.21.15.7:0/2216937194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab801d040 con 0x55b0483c0f20 2024-01-11T18:01:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.475+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483ca9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005810 tx=0x7f6ab8003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4816f0e497cba1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.475+0000 7f6aa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:01:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.475+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/2216937194 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8009a20 con 0x55b0483c0f20 2024-01-11T18:01:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.475+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/2216937194 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6ab8009b80 con 0x55b0483c0f20 2024-01-11T18:01:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.476+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0483b3d60 con 0x55b0483c0f20 2024-01-11T18:01:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.476+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/2216937194 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ab8009d40 con 0x55b0483c0f20 2024-01-11T18:01:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.477+0000 7f6ac37fe700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6aac0a2010 0x7f6aac0a44d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0976e0 tx=0x7f6abc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.477+0000 7f6add702b80 10 client.25927 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.477+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b048135b00 con 0x55b0483c0f20 2024-01-11T18:01:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.477+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/2216937194 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0483b3d60 con 0x55b0483c0f20 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/2216937194 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6ab8021450 con 0x55b0483c0f20 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6add702b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6add702b80 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6add702b80 20 client.25927 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6add702b80 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b0483d6700 0x55b0483d8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6add702b80 4 client.25927 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b04830db60 con 0x55b0483d6700 2024-01-11T18:01:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.478+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b0483d6700 0x55b0483d8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.479+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b0483d6700 0x55b0483d8b20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8d32edcbe2eff3 server_cookie=962eae594cb897f7 in_seq=0 out_seq=0 2024-01-11T18:01:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.479+0000 7f6aa2ffd700 10 client.25927 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.480+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/2216937194 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b04830db60 con 0x55b0483d6700 2024-01-11T18:01:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.480+0000 7f6aa2ffd700 10 client.25927 handle_command_reply: tid=0 2024-01-11T18:01:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.480+0000 7f6add702b80 1 client.25927 shutdown 2024-01-11T18:01:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 msgr2=0x55b0483ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483ca9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005810 tx=0x7f6ab8003dd0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 shutdown_connections 2024-01-11T18:01:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b0483d6700 0x55b0483d8b20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c0f20 0x55b0483ca9c0 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005810 tx=0x7f6ab8003dd0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c25d0 0x55b0483d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 --2- 172.21.15.7:0/2216937194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 >> 172.21.15.7:0/2216937194 conn(0x55b048336740 msgr2=0x55b0483baf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.481+0000 7f6aa2ffd700 0 client.25927 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.482+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 shutdown_connections 2024-01-11T18:01:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.482+0000 7f6add702b80 1 -- 172.21.15.7:0/2216937194 wait complete. 2024-01-11T18:01:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.483+0000 7f6add702b80 20 client.25927 trim_cache size 0 max 16384 2024-01-11T18:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.532+0000 7f6add702b80 1 Processor -- start 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.532+0000 7f6add702b80 1 -- start start 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6add702b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6add702b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 0x55b0483d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6add702b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483dd480 0x55b0483dd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6add702b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04818b640 con 0x55b0483dd480 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6add702b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04830db90 con 0x55b0483c1c10 2024-01-11T18:01:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6add702b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0483d30c0 con 0x55b0483c25d0 2024-01-11T18:01:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.533+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 0x55b0483d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483dd480 0x55b0483dd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 0x55b0483d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50050/0 (socket says 172.21.15.7:50050) 2024-01-11T18:01:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/640233129 learned_addr learned my addr 172.21.15.7:0/640233129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/640233129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 msgr2=0x55b0483c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/640233129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483dd480 msgr2=0x55b0483dd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483dd480 0x55b0483dd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.534+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/640233129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0482d61d0 con 0x55b0483c25d0 2024-01-11T18:01:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.535+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 0x55b0483d1f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40048a0 tx=0x7f6ab4009210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d9c6b399a4b221 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.536+0000 7f6aa97fa700 1 -- 172.21.15.7:0/640233129 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40178b0 con 0x55b0483c25d0 2024-01-11T18:01:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.536+0000 7f6aa97fa700 1 -- 172.21.15.7:0/640233129 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ab40190b0 con 0x55b0483c25d0 2024-01-11T18:01:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.536+0000 7f6aa97fa700 1 -- 172.21.15.7:0/640233129 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40226d0 con 0x55b0483c25d0 2024-01-11T18:01:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 -- 172.21.15.7:0/640233129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 msgr2=0x55b0483d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 0x55b0483d1f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40048a0 tx=0x7f6ab4009210 comp rx=0 tx=0).stop 2024-01-11T18:01:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 -- 172.21.15.7:0/640233129 shutdown_connections 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483dd480 0x55b0483dd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483c1c10 0x55b0483c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 --2- 172.21.15.7:0/640233129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483c25d0 0x55b0483d1f00 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40048a0 tx=0x7f6ab4009210 comp rx=0 tx=0).stop 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 -- 172.21.15.7:0/640233129 >> 172.21.15.7:0/640233129 conn(0x55b048336740 msgr2=0x55b0483c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 -- 172.21.15.7:0/640233129 shutdown_connections 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.537+0000 7f6add702b80 1 -- 172.21.15.7:0/640233129 wait complete. 2024-01-11T18:01:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.538+0000 7f6add702b80 1 -- start start 2024-01-11T18:01:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.538+0000 7f6add702b80 1 Processor -- start 2024-01-11T18:01:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 0x55b0483cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483d0c30 0x55b0483d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0483d15f0 con 0x55b0483c1c10 2024-01-11T18:01:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0483d1760 con 0x55b0483dd480 2024-01-11T18:01:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0483d18a0 con 0x55b0483d0c30 2024-01-11T18:01:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6add702b80 10 client.0 _ll_register_callbacks cb 0x55b0482b56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.539+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 0x55b0483cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 0x55b0483cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52174/0 (socket says 172.21.15.7:52174) 2024-01-11T18:01:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/347733557 learned_addr learned my addr 172.21.15.7:0/347733557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483d0c30 0x55b0483d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/347733557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483d0c30 msgr2=0x55b0483d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483d0c30 0x55b0483d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/347733557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 msgr2=0x55b0483cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.540+0000 7f6ac2ffd700 1 -- 172.21.15.7:0/347733557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab4019e70 con 0x55b0483dd480 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.541+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483d0c30 0x55b0483d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.541+0000 7f6ac2ffd700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 0x55b0483cfd30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4019a60 tx=0x7f6ab4009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8ff90dfa12e431 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.541+0000 7f6aa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:01:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.542+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/347733557 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4005ac0 con 0x55b0483dd480 2024-01-11T18:01:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.542+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/347733557 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6ab4005c20 con 0x55b0483dd480 2024-01-11T18:01:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.542+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ab8003680 con 0x55b0483dd480 2024-01-11T18:01:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.543+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/347733557 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ab4005de0 con 0x55b0483dd480 2024-01-11T18:01:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.543+0000 7f6add702b80 10 client.25907 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.543+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b04818dc70 con 0x55b0483dd480 2024-01-11T18:01:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.544+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/347733557 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ab8003680 con 0x55b0483dd480 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.544+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/347733557 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6ab402b030 con 0x55b0483dd480 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.544+0000 7f6add702b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.544+0000 7f6add702b80 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.544+0000 7f6add702b80 20 client.25907 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.545+0000 7f6add702b80 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b04843e950 0x55b048440d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.545+0000 7f6add702b80 4 client.25907 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:01:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.545+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b04830d880 con 0x55b04843e950 2024-01-11T18:01:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.546+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b04843e950 0x55b048440d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.547+0000 7f6ac27fc700 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b04843e950 0x55b048440d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e18896f21a2b80 server_cookie=a0adcd216237e08b in_seq=0 out_seq=0 2024-01-11T18:01:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.547+0000 7f6aa2ffd700 10 client.25907 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.561+0000 7f6aa2ffd700 1 -- 172.21.15.7:0/347733557 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b04830d880 con 0x55b04843e950 2024-01-11T18:01:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.562+0000 7f6aa2ffd700 10 client.25907 handle_command_reply: tid=0 2024-01-11T18:01:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.562+0000 7f6add702b80 1 client.25907 shutdown 2024-01-11T18:01:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 msgr2=0x55b0483cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 0x55b0483cfd30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4019a60 tx=0x7f6ab4009dd0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 shutdown_connections 2024-01-11T18:01:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b04843e950 0x55b048440d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0483d0c30 0x55b0483d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0483c1c10 0x55b0483cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 --2- 172.21.15.7:0/347733557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0483dd480 0x55b0483cfd30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 >> 172.21.15.7:0/347733557 conn(0x55b048336740 msgr2=0x55b0483c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6aa2ffd700 0 client.25907 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.563+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 shutdown_connections 2024-01-11T18:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.564+0000 7f6add702b80 1 -- 172.21.15.7:0/347733557 wait complete. 2024-01-11T18:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.564+0000 7f6add702b80 20 client.25907 trim_cache size 0 max 16384 2024-01-11T18:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.565+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6aac0a2010 msgr2=0x7f6aac0a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6aac0a2010 0x7f6aac0a44d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0976e0 tx=0x7f6abc009040 comp rx=0 tx=0).stop 2024-01-11T18:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4102030 msgr2=0x7f6ac419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4102030 0x7f6ac419ca50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00fc00 tx=0x7f6abc017860 comp rx=0 tx=0).stop 2024-01-11T18:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 shutdown_connections 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6aac0a2010 0x7f6aac0a44d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ac41004a0 0x7f6ac41954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ac4102030 0x7f6ac419ca50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 --2- 172.21.15.7:0/3822906066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac4101690 0x7f6ac41959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 >> 172.21.15.7:0/3822906066 conn(0x7f6ac40fba30 msgr2=0x7f6ac4104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 shutdown_connections 2024-01-11T18:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:15.566+0000 7f6ac99ff700 1 -- 172.21.15.7:0/3822906066 wait complete. 2024-01-11T18:01:15.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:15 smithi016 ceph-mon[43195]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2024-01-11T18:01:15.774 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:15 smithi038 ceph-mon[43582]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2024-01-11T18:01:15.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:15 smithi007 ceph-mon[33361]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2024-01-11T18:01:18.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:17 smithi038 ceph-mon[43582]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-01-11T18:01:18.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:17 smithi007 ceph-mon[33361]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-01-11T18:01:18.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:17 smithi016 ceph-mon[43195]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-01-11T18:01:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 170.0 s, 1237.6 tps, lat 79.854 ms stddev 96.491 2024-01-11T18:01:20.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:19 smithi007 ceph-mon[33361]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.1 MiB/s wr, 549 op/s 2024-01-11T18:01:20.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:19 smithi016 ceph-mon[43195]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.1 MiB/s wr, 549 op/s 2024-01-11T18:01:20.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:19 smithi038 ceph-mon[43582]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.1 MiB/s wr, 549 op/s 2024-01-11T18:01:22.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:21 smithi007 ceph-mon[33361]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 599 op/s 2024-01-11T18:01:22.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:21 smithi016 ceph-mon[43195]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 599 op/s 2024-01-11T18:01:22.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:21 smithi038 ceph-mon[43582]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 599 op/s 2024-01-11T18:01:23.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:22 smithi016 ceph-mon[43195]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 626 op/s 2024-01-11T18:01:23.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:22 smithi038 ceph-mon[43582]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 626 op/s 2024-01-11T18:01:23.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:22 smithi007 ceph-mon[33361]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 626 op/s 2024-01-11T18:01:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 175.0 s, 1220.8 tps, lat 82.587 ms stddev 95.339 2024-01-11T18:01:24.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:24 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:01:24.773 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:24 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:01:24.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:24 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:01:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:25 smithi007 ceph-mon[33361]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 527 op/s 2024-01-11T18:01:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:25 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:25 smithi016 ceph-mon[43195]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 527 op/s 2024-01-11T18:01:25.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:25.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:25 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:25 smithi038 ceph-mon[43582]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 527 op/s 2024-01-11T18:01:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:26.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:25 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:26.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:01:26.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:01:26.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:26.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:01:26.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:01:26.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:27.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:01:27.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:01:27.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:01:27.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:27 smithi016 ceph-mon[43195]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 642 op/s 2024-01-11T18:01:28.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:27 smithi038 ceph-mon[43582]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 642 op/s 2024-01-11T18:01:28.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:27 smithi007 ceph-mon[33361]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 642 op/s 2024-01-11T18:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.825+0000 7facd99ff700 1 -- 172.21.15.7:0/2901418902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd41004a0 msgr2=0x7facd41008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.825+0000 7facd99ff700 1 --2- 172.21.15.7:0/2901418902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd41004a0 0x7facd41008c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7facc400b0a0 tx=0x7facc400b3b0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 -- 172.21.15.7:0/2901418902 shutdown_connections 2024-01-11T18:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 --2- 172.21.15.7:0/2901418902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd41004a0 0x7facd41008c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 --2- 172.21.15.7:0/2901418902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd4102030 0x7facd41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 --2- 172.21.15.7:0/2901418902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 0x7facd4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 -- 172.21.15.7:0/2901418902 >> 172.21.15.7:0/2901418902 conn(0x7facd40fba30 msgr2=0x7facd40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 -- 172.21.15.7:0/2901418902 shutdown_connections 2024-01-11T18:01:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.826+0000 7facd99ff700 1 -- 172.21.15.7:0/2901418902 wait complete. 2024-01-11T18:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.828+0000 7facd99ff700 1 Processor -- start 2024-01-11T18:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.829+0000 7facd99ff700 1 -- start start 2024-01-11T18:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.830+0000 7facd99ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd41004a0 0x7facd41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.830+0000 7facd99ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 0x7facd41a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.830+0000 7facd99ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 0x7facd41a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.830+0000 7facd99ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd41a5780 con 0x7facd41004a0 2024-01-11T18:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.830+0000 7facd99ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd4195a40 con 0x7facd4101690 2024-01-11T18:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.830+0000 7facd99ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd4195c00 con 0x7facd4102030 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.831+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd41004a0 0x7facd41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.831+0000 7facd37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 0x7facd41a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.831+0000 7facd37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 0x7facd41a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33760/0 (socket says 172.21.15.7:33760) 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.831+0000 7facd37fe700 1 -- 172.21.15.7:0/3725535397 learned_addr learned my addr 172.21.15.7:0/3725535397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.831+0000 7facca7fc700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 0x7facd41a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facd37fe700 1 -- 172.21.15.7:0/3725535397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 msgr2=0x7facd41a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facd37fe700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 0x7facd41a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facd37fe700 1 -- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd41004a0 msgr2=0x7facd41a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facd37fe700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd41004a0 0x7facd41a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facd37fe700 1 -- 172.21.15.7:0/3725535397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009d00 con 0x7facd4102030 2024-01-11T18:01:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facca7fc700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 0x7facd41a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:01:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.832+0000 7facd37fe700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 0x7facd41a4fb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d980 tx=0x7faccc00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.833+0000 7facd89fd700 1 -- 172.21.15.7:0/3725535397 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc009940 con 0x7facd4102030 2024-01-11T18:01:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.833+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd4195ee0 con 0x7facd4102030 2024-01-11T18:01:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.833+0000 7facd89fd700 1 -- 172.21.15.7:0/3725535397 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faccc004830 con 0x7facd4102030 2024-01-11T18:01:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.833+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd41964c0 con 0x7facd4102030 2024-01-11T18:01:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.833+0000 7facd89fd700 1 -- 172.21.15.7:0/3725535397 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc009940 con 0x7facd4102030 2024-01-11T18:01:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.839+0000 7facd89fd700 1 -- 172.21.15.7:0/3725535397 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7faccc0052c0 con 0x7facd4102030 2024-01-11T18:01:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.841+0000 7faced659b80 1 Processor -- start 2024-01-11T18:01:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.841+0000 7faced659b80 1 -- start start 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.842+0000 7faced659b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f8f20 0x5653668f9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.842+0000 7faced659b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 0x5653668fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.842+0000 7faced659b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366900e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.842+0000 7faced659b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653667f0030 con 0x5653668f8f20 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.842+0000 7faced659b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565366853ac0 con 0x5653668fa5d0 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.842+0000 7faced659b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653668513e0 con 0x5653668f9c10 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facca7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366900e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 0x5653668fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 0x5653668fa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33774/0 (socket says 172.21.15.7:33774) 2024-01-11T18:01:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 -- 172.21.15.7:0/1646628896 learned_addr learned my addr 172.21.15.7:0/1646628896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facca7fc700 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f8f20 0x5653668f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd89fd700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7facc00a1f30 0x7facc00a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 -- 172.21.15.7:0/1646628896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 msgr2=0x565366900e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366900e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 -- 172.21.15.7:0/1646628896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f8f20 msgr2=0x5653668f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f8f20 0x5653668f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.843+0000 7facd2ffd700 1 -- 172.21.15.7:0/1646628896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653666c1350 con 0x5653668f9c10 2024-01-11T18:01:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.844+0000 7facca7fc700 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366900e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:01:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.844+0000 7facd2ffd700 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 0x5653668fa090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7facd41014f0 tx=0x7facc4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ed7456c72d2db3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.844+0000 7facd89fd700 1 -- 172.21.15.7:0/3725535397 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faccc0cd8f0 con 0x7facd4102030 2024-01-11T18:01:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.844+0000 7facc9ffb700 1 -- 172.21.15.7:0/1646628896 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400e050 con 0x5653668f9c10 2024-01-11T18:01:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.844+0000 7facc9ffb700 1 -- 172.21.15.7:0/1646628896 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc40092d0 con 0x5653668f9c10 2024-01-11T18:01:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.844+0000 7facc9ffb700 1 -- 172.21.15.7:0/1646628896 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40052a0 con 0x5653668f9c10 2024-01-11T18:01:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.845+0000 7facd37fe700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7facc00a1f30 0x7facc00a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.845+0000 7faced659b80 1 -- 172.21.15.7:0/1646628896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 msgr2=0x5653668fa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.845+0000 7faced659b80 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 0x5653668fa090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7facd41014f0 tx=0x7facc4009780 comp rx=0 tx=0).stop 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 -- 172.21.15.7:0/1646628896 shutdown_connections 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366900e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f8f20 0x5653668f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 --2- 172.21.15.7:0/1646628896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f9c10 0x5653668fa090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 -- 172.21.15.7:0/1646628896 >> 172.21.15.7:0/1646628896 conn(0x56536686e740 msgr2=0x5653668f3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 -- 172.21.15.7:0/1646628896 shutdown_connections 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7facd37fe700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7facc00a1f30 0x7facc00a43f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faccc09a020 tx=0x7faccc096040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.846+0000 7faced659b80 1 -- 172.21.15.7:0/1646628896 wait complete. 2024-01-11T18:01:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.847+0000 7faced659b80 1 -- start start 2024-01-11T18:01:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.848+0000 7faced659b80 1 Processor -- start 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366902900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366902e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366909e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536684ccd0 con 0x5653668f9c10 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653667bb340 con 0x5653668fa5d0 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653667bafe0 con 0x5653668f8f20 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7faced659b80 10 client.0 _ll_register_callbacks cb 0x5653667ed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7facca7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366902900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7facca7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366902900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33782/0 (socket says 172.21.15.7:33782) 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7facca7fc700 1 -- 172.21.15.7:0/399589408 learned_addr learned my addr 172.21.15.7:0/399589408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.849+0000 7facd2ffd700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366902e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facca7fc700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366909e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facd2ffd700 1 -- 172.21.15.7:0/399589408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 msgr2=0x565366902900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facd2ffd700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366902900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facd2ffd700 1 -- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 msgr2=0x565366909e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facd2ffd700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366909e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facd2ffd700 1 -- 172.21.15.7:0/399589408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd4109230 con 0x5653668f9c10 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facd2ffd700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366902e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7facc400d010 tx=0x7facc40060a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61f552d8da8a5d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.850+0000 7facb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.851+0000 7facb2ffd700 1 -- 172.21.15.7:0/399589408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4005250 con 0x5653668f9c10 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.851+0000 7facb2ffd700 1 -- 172.21.15.7:0/399589408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7facc40053b0 con 0x5653668f9c10 2024-01-11T18:01:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.851+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653668ebd60 con 0x5653668f9c10 2024-01-11T18:01:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.852+0000 7facb2ffd700 1 -- 172.21.15.7:0/399589408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7facc4005590 con 0x5653668f9c10 2024-01-11T18:01:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.853+0000 7faced659b80 10 client.15840 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.853+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56536666db00 con 0x5653668f9c10 2024-01-11T18:01:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7facb2ffd700 1 -- 172.21.15.7:0/399589408 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653668ebd60 con 0x5653668f9c10 2024-01-11T18:01:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7facb2ffd700 1 -- 172.21.15.7:0/399589408 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7facc402f050 con 0x5653668f9c10 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7faced659b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7faced659b80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7faced659b80 20 client.15840 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7faced659b80 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56536690e310 0x565366910730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.855+0000 7faced659b80 4 client.15840 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.856+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565366845b60 con 0x56536690e310 2024-01-11T18:01:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.856+0000 7facca7fc700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56536690e310 0x565366910730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.858+0000 7facca7fc700 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56536690e310 0x565366910730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6290b28d0933711b server_cookie=3803abad91ea2568 in_seq=0 out_seq=0 2024-01-11T18:01:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.858+0000 7facb2ffd700 10 client.15840 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.858+0000 7facb2ffd700 1 -- 172.21.15.7:0/399589408 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565366845b60 con 0x56536690e310 2024-01-11T18:01:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.858+0000 7facb2ffd700 10 client.15840 handle_command_reply: tid=0 2024-01-11T18:01:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 client.15840 shutdown 2024-01-11T18:01:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 msgr2=0x565366902e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366902e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7facc400d010 tx=0x7facc40060a0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 shutdown_connections 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56536690e310 0x565366910730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366902900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x565366909e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 --2- 172.21.15.7:0/399589408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366902e40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 >> 172.21.15.7:0/399589408 conn(0x56536686e740 msgr2=0x5653668f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.859+0000 7facb2ffd700 0 client.15840 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.860+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 shutdown_connections 2024-01-11T18:01:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.860+0000 7faced659b80 1 -- 172.21.15.7:0/399589408 wait complete. 2024-01-11T18:01:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.860+0000 7faced659b80 20 client.15840 trim_cache size 0 max 16384 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.891+0000 7faced659b80 1 Processor -- start 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.891+0000 7faced659b80 1 -- start start 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.892+0000 7faced659b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668f8f20 0x5653668f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.892+0000 7faced659b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366909e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.892+0000 7faced659b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 0x565366901e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.892+0000 7faced659b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653666c3640 con 0x5653668f9c10 2024-01-11T18:01:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.892+0000 7faced659b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653668f0560 con 0x5653668f8f20 2024-01-11T18:01:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.892+0000 7faced659b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653668f4be0 con 0x5653668fa5d0 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.893+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366909e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.893+0000 7facca7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 0x565366901e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.893+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366909e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53992/0 (socket says 172.21.15.7:53992) 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.893+0000 7facca7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 0x565366901e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33792/0 (socket says 172.21.15.7:33792) 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.893+0000 7facd2ffd700 1 -- 172.21.15.7:0/1812327282 learned_addr learned my addr 172.21.15.7:0/1812327282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.894+0000 7facd2ffd700 1 -- 172.21.15.7:0/1812327282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 msgr2=0x565366901e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.894+0000 7facd2ffd700 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 0x565366901e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.894+0000 7facd2ffd700 1 -- 172.21.15.7:0/1812327282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668f8f20 msgr2=0x5653668f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.894+0000 7facd2ffd700 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668f8f20 0x5653668f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.894+0000 7facd2ffd700 1 -- 172.21.15.7:0/1812327282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56536680e1d0 con 0x5653668f9c10 2024-01-11T18:01:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.894+0000 7facd2ffd700 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366909e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7facc4000f80 tx=0x7facc4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9d2b44f2949903 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.896+0000 7facca7fc700 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 0x565366901e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-01-11T18:01:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.896+0000 7facc97fa700 1 -- 172.21.15.7:0/1812327282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400e030 con 0x5653668f9c10 2024-01-11T18:01:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.896+0000 7facc97fa700 1 -- 172.21.15.7:0/1812327282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc4008f60 con 0x5653668f9c10 2024-01-11T18:01:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.897+0000 7facc97fa700 1 -- 172.21.15.7:0/1812327282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40055b0 con 0x5653668f9c10 2024-01-11T18:01:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 -- 172.21.15.7:0/1812327282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 msgr2=0x565366909e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366909e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7facc4000f80 tx=0x7facc4009780 comp rx=0 tx=0).stop 2024-01-11T18:01:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 -- 172.21.15.7:0/1812327282 shutdown_connections 2024-01-11T18:01:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668fa5d0 0x565366901e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668f8f20 0x5653668f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 --2- 172.21.15.7:0/1812327282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653668f9c10 0x565366909e40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 -- 172.21.15.7:0/1812327282 >> 172.21.15.7:0/1812327282 conn(0x56536686e740 msgr2=0x565366900c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 -- 172.21.15.7:0/1812327282 shutdown_connections 2024-01-11T18:01:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.899+0000 7faced659b80 1 -- 172.21.15.7:0/1812327282 wait complete. 2024-01-11T18:01:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.900+0000 7faced659b80 1 -- start start 2024-01-11T18:01:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.900+0000 7faced659b80 1 Processor -- start 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366907090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x5653669075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653669085e0 0x565366908a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x565366908fa0 con 0x5653669085e0 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565366909110 con 0x5653668fa5d0 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653669092d0 con 0x5653668f8f20 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7faced659b80 10 client.0 _ll_register_callbacks cb 0x5653667ed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x5653669075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7facd2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x5653669075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50564/0 (socket says 172.21.15.7:50564) 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7facd2ffd700 1 -- 172.21.15.7:0/2850243803 learned_addr learned my addr 172.21.15.7:0/2850243803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.901+0000 7facca7fc700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653669085e0 0x565366908a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.902+0000 7facca7fc700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366907090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facca7fc700 1 -- 172.21.15.7:0/2850243803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 msgr2=0x565366907090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facca7fc700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366907090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facca7fc700 1 -- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 msgr2=0x5653669075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facca7fc700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x5653669075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facca7fc700 1 -- 172.21.15.7:0/2850243803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56536666db00 con 0x5653669085e0 2024-01-11T18:01:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facca7fc700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653669085e0 0x565366908a60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7facbc00cce0 tx=0x7facbc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141f4c6a5526ec94 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.904+0000 7facb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:01:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.905+0000 7facb2ffd700 1 -- 172.21.15.7:0/2850243803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0175e0 con 0x5653669085e0 2024-01-11T18:01:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.905+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facc40092d0 con 0x5653669085e0 2024-01-11T18:01:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.905+0000 7facb2ffd700 1 -- 172.21.15.7:0/2850243803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7facbc017740 con 0x5653669085e0 2024-01-11T18:01:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.906+0000 7facb2ffd700 1 -- 172.21.15.7:0/2850243803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7facbc017900 con 0x5653669085e0 2024-01-11T18:01:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.906+0000 7faced659b80 10 client.15846 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.906+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5653666c5c70 con 0x5653669085e0 2024-01-11T18:01:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7facb2ffd700 1 -- 172.21.15.7:0/2850243803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc40092d0 con 0x5653669085e0 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7facb2ffd700 1 -- 172.21.15.7:0/2850243803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7facbc0159e0 con 0x5653669085e0 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7faced659b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7faced659b80 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7faced659b80 20 client.15846 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7faced659b80 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565366965120 0x565366967540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7faced659b80 4 client.15846 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.907+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565366845880 con 0x565366965120 2024-01-11T18:01:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.908+0000 7facd2ffd700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565366965120 0x565366967540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.912+0000 7facb2ffd700 10 client.15846 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:28.912+0000 7facd2ffd700 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565366965120 0x565366967540 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29ac0a93155e66d server_cookie=6866690a955df9d in_seq=0 out_seq=0 2024-01-11T18:01:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.037+0000 7facb2ffd700 1 -- 172.21.15.7:0/2850243803 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565366845880 con 0x565366965120 2024-01-11T18:01:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.037+0000 7facb2ffd700 10 client.15846 handle_command_reply: tid=0 2024-01-11T18:01:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.038+0000 7faced659b80 1 client.15846 shutdown 2024-01-11T18:01:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.038+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653669085e0 msgr2=0x565366908a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.038+0000 7faced659b80 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653669085e0 0x565366908a60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7facbc00cce0 tx=0x7facbc004770 comp rx=0 tx=0).stop 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 shutdown_connections 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565366965120 0x565366967540 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653668f8f20 0x565366907090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5653669085e0 0x565366908a60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 --2- 172.21.15.7:0/2850243803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653668fa5d0 0x5653669075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7facb2ffd700 0 client.15846 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 >> 172.21.15.7:0/2850243803 conn(0x56536686e740 msgr2=0x565366901e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.039+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 shutdown_connections 2024-01-11T18:01:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.040+0000 7faced659b80 1 -- 172.21.15.7:0/2850243803 wait complete. 2024-01-11T18:01:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.040+0000 7faced659b80 20 client.15846 trim_cache size 0 max 16384 2024-01-11T18:01:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7facc00a1f30 msgr2=0x7facc00a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7facc00a1f30 0x7facc00a43f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faccc09a020 tx=0x7faccc096040 comp rx=0 tx=0).stop 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 msgr2=0x7facd41a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 0x7facd41a4fb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d980 tx=0x7faccc00dd40 comp rx=0 tx=0).stop 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 shutdown_connections 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7facc00a1f30 0x7facc00a43f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd4102030 0x7facd41a4fb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd41004a0 0x7facd41a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 --2- 172.21.15.7:0/3725535397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd4101690 0x7facd41a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 >> 172.21.15.7:0/3725535397 conn(0x7facd40fba30 msgr2=0x7facd40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 shutdown_connections 2024-01-11T18:01:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:29.041+0000 7facd99ff700 1 -- 172.21.15.7:0/3725535397 wait complete. 2024-01-11T18:01:29.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 180.0 s, 799.6 tps, lat 123.079 ms stddev 162.824 2024-01-11T18:01:30.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:29 smithi007 ceph-mon[33361]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-01-11T18:01:30.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:29 smithi016 ceph-mon[43195]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-01-11T18:01:30.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:29 smithi038 ceph-mon[43582]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-01-11T18:01:32.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:31 smithi007 ceph-mon[33361]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.2 MiB/s wr, 619 op/s 2024-01-11T18:01:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:31 smithi016 ceph-mon[43195]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.2 MiB/s wr, 619 op/s 2024-01-11T18:01:32.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:31 smithi038 ceph-mon[43582]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.2 MiB/s wr, 619 op/s 2024-01-11T18:01:34.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:33 smithi007 ceph-mon[33361]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.9 MiB/s wr, 648 op/s 2024-01-11T18:01:34.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 185.0 s, 725.2 tps, lat 135.074 ms stddev 177.098 2024-01-11T18:01:34.247 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:33 smithi016 ceph-mon[43195]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.9 MiB/s wr, 648 op/s 2024-01-11T18:01:34.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:33 smithi038 ceph-mon[43582]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.9 MiB/s wr, 648 op/s 2024-01-11T18:01:35.773 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:35 smithi038 ceph-mon[43582]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 596 op/s 2024-01-11T18:01:35.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:35 smithi007 ceph-mon[33361]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 596 op/s 2024-01-11T18:01:35.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:35 smithi016 ceph-mon[43195]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 596 op/s 2024-01-11T18:01:38.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:37 smithi007 ceph-mon[33361]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 785 op/s 2024-01-11T18:01:38.253 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:37 smithi016 ceph-mon[43195]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 785 op/s 2024-01-11T18:01:38.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:37 smithi038 ceph-mon[43582]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 785 op/s 2024-01-11T18:01:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 190.0 s, 670.8 tps, lat 152.257 ms stddev 174.037 2024-01-11T18:01:40.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:39 smithi007 ceph-mon[33361]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 9.9 MiB/s wr, 699 op/s 2024-01-11T18:01:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:39 smithi016 ceph-mon[43195]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 9.9 MiB/s wr, 699 op/s 2024-01-11T18:01:40.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:39 smithi038 ceph-mon[43582]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 9.9 MiB/s wr, 699 op/s 2024-01-11T18:01:42.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:41 smithi007 ceph-mon[33361]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 740 op/s 2024-01-11T18:01:42.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:41 smithi016 ceph-mon[43195]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 740 op/s 2024-01-11T18:01:42.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:41 smithi038 ceph-mon[43582]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 740 op/s 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 -- 172.21.15.7:0/599951624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 msgr2=0x7f9538103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 --2- 172.21.15.7:0/599951624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 0x7f9538103c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f952800b0d0 tx=0x7f952800b3e0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 -- 172.21.15.7:0/599951624 shutdown_connections 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 --2- 172.21.15.7:0/599951624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f953810bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 --2- 172.21.15.7:0/599951624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 0x7f9538103c10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 --2- 172.21.15.7:0/599951624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95381049e0 0x7f9538104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 -- 172.21.15.7:0/599951624 >> 172.21.15.7:0/599951624 conn(0x7f95380fed60 msgr2=0x7f95381011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.310+0000 7f953dfff700 1 -- 172.21.15.7:0/599951624 shutdown_connections 2024-01-11T18:01:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.311+0000 7f953dfff700 1 -- 172.21.15.7:0/599951624 wait complete. 2024-01-11T18:01:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.312+0000 7f953dfff700 1 Processor -- start 2024-01-11T18:01:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 -- start start 2024-01-11T18:01:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 0x7f9538199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95381049e0 0x7f9538199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f95381a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381a1410 con 0x7f95381049e0 2024-01-11T18:01:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381a1550 con 0x7f95381037f0 2024-01-11T18:01:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f953dfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381a16e0 con 0x7f9538105380 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f9537fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f95381a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.313+0000 7f9537fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f95381a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35840/0 (socket says 172.21.15.7:35840) 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f9537fff700 1 -- 172.21.15.7:0/1808814506 learned_addr learned my addr 172.21.15.7:0/1808814506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f952effd700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95381049e0 0x7f9538199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f95377fe700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 0x7f9538199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f9537fff700 1 -- 172.21.15.7:0/1808814506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 msgr2=0x7f9538199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f9537fff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 0x7f9538199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f9537fff700 1 -- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95381049e0 msgr2=0x7f9538199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f9537fff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95381049e0 0x7f9538199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.314+0000 7f9537fff700 1 -- 172.21.15.7:0/1808814506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9528009d00 con 0x7f9538105380 2024-01-11T18:01:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.315+0000 7f9537fff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f95381a0e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f953000e940 tx=0x7f953000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.315+0000 7f95357fa700 1 -- 172.21.15.7:0/1808814506 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300097b0 con 0x7f9538105380 2024-01-11T18:01:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.315+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95381a19c0 con 0x7f9538105380 2024-01-11T18:01:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.315+0000 7f95357fa700 1 -- 172.21.15.7:0/1808814506 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953000ee90 con 0x7f9538105380 2024-01-11T18:01:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.315+0000 7f95357fa700 1 -- 172.21.15.7:0/1808814506 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530005520 con 0x7f9538105380 2024-01-11T18:01:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.316+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95381a1f50 con 0x7f9538105380 2024-01-11T18:01:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.327+0000 7f95357fa700 1 -- 172.21.15.7:0/1808814506 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f9530004750 con 0x7f9538105380 2024-01-11T18:01:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.328+0000 7f9551cc9b80 1 Processor -- start 2024-01-11T18:01:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 -- start start 2024-01-11T18:01:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754ceab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceb1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cda1030 con 0x55754ceaac10 2024-01-11T18:01:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754ce04ac0 con 0x55754ceab5d0 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.329+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754ce023e0 con 0x55754cea9f20 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754ceab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754ceab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39776/0 (socket says 172.21.15.7:39776) 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 -- 172.21.15.7:0/2437460897 learned_addr learned my addr 172.21.15.7:0/2437460897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f952effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95357fa700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95240a1ed0 0x7f95240a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f952effd700 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceb1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95357fa700 1 -- 172.21.15.7:0/1808814506 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9530014070 con 0x7f9538105380 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 -- 172.21.15.7:0/2437460897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 msgr2=0x55754ceaa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 -- 172.21.15.7:0/2437460897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 msgr2=0x55754ceb1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceb1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f95377fe700 1 -- 172.21.15.7:0/2437460897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55754cc72350 con 0x55754ceaac10 2024-01-11T18:01:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f952effd700 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:01:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.330+0000 7f9537fff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95240a1ed0 0x7f95240a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.331+0000 7f95377fe700 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754ceab090 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9538104840 tx=0x7f9528005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7799efeed8486b03 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.331+0000 7f952dffb700 1 -- 172.21.15.7:0/2437460897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800e040 con 0x55754ceaac10 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.331+0000 7f952dffb700 1 -- 172.21.15.7:0/2437460897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95280076e0 con 0x55754ceaac10 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.331+0000 7f952dffb700 1 -- 172.21.15.7:0/2437460897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95280052d0 con 0x55754ceaac10 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.332+0000 7f9551cc9b80 1 -- 172.21.15.7:0/2437460897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 msgr2=0x55754ceab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.332+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754ceab090 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9538104840 tx=0x7f9528005b50 comp rx=0 tx=0).stop 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.332+0000 7f9537fff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95240a1ed0 0x7f95240a4390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9530096e00 tx=0x7f953000cea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.333+0000 7f9551cc9b80 1 -- 172.21.15.7:0/2437460897 shutdown_connections 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.333+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.333+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceb1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.333+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/2437460897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754ceab090 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.333+0000 7f9551cc9b80 1 -- 172.21.15.7:0/2437460897 >> 172.21.15.7:0/2437460897 conn(0x55754ce1f740 msgr2=0x55754cea4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.333+0000 7f9551cc9b80 1 -- 172.21.15.7:0/2437460897 shutdown_connections 2024-01-11T18:01:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.334+0000 7f9551cc9b80 1 -- 172.21.15.7:0/2437460897 wait complete. 2024-01-11T18:01:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.335+0000 7f9551cc9b80 1 -- start start 2024-01-11T18:01:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 Processor -- start 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754cebc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cebc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceab5d0 0x55754cec36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cdfdcd0 con 0x55754ceab5d0 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cd6c340 con 0x55754ceaac10 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cd6bfe0 con 0x55754cea9f20 2024-01-11T18:01:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.336+0000 7f9551cc9b80 10 client.0 _ll_register_callbacks cb 0x55754cd9e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.337+0000 7f952effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceab5d0 0x55754cec36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.337+0000 7f952effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754cebc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.337+0000 7f952effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754cebc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35846/0 (socket says 172.21.15.7:35846) 2024-01-11T18:01:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.337+0000 7f952effd700 1 -- 172.21.15.7:0/747787821 learned_addr learned my addr 172.21.15.7:0/747787821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f952effd700 1 -- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 msgr2=0x55754cebc670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f952effd700 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cebc670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f952effd700 1 -- 172.21.15.7:0/747787821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceab5d0 msgr2=0x55754cec36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f952effd700 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceab5d0 0x55754cec36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f952effd700 1 -- 172.21.15.7:0/747787821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953810c580 con 0x55754cea9f20 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f9516ffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:01:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.338+0000 7f952effd700 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754cebc130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9520006d30 tx=0x7f9520004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff968b91eaf3e682 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.339+0000 7f9516ffd700 1 -- 172.21.15.7:0/747787821 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520016bb0 con 0x55754cea9f20 2024-01-11T18:01:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.339+0000 7f9516ffd700 1 -- 172.21.15.7:0/747787821 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f952000c650 con 0x55754cea9f20 2024-01-11T18:01:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.339+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55754ce9cd60 con 0x55754cea9f20 2024-01-11T18:01:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.339+0000 7f9516ffd700 1 -- 172.21.15.7:0/747787821 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9520016d10 con 0x55754cea9f20 2024-01-11T18:01:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.339+0000 7f9551cc9b80 10 client.25960 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.339+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55754cc1eb00 con 0x55754cea9f20 2024-01-11T18:01:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9516ffd700 1 -- 172.21.15.7:0/747787821 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55754ce9cd60 con 0x55754cea9f20 2024-01-11T18:01:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9516ffd700 1 -- 172.21.15.7:0/747787821 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9520014390 con 0x55754cea9f20 2024-01-11T18:01:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9551cc9b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9551cc9b80 10 client.25960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9551cc9b80 20 client.25960 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cec8040 0x55754ceca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9551cc9b80 4 client.25960 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.340+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55754cdf6b60 con 0x55754cec8040 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.341+0000 7f95377fe700 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cec8040 0x55754ceca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.341+0000 7f95377fe700 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cec8040 0x55754ceca460 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a336fa6e5383a4 server_cookie=8b6b86c7530f66b9 in_seq=0 out_seq=0 2024-01-11T18:01:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.341+0000 7f9516ffd700 10 client.25960 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.342+0000 7f9516ffd700 1 -- 172.21.15.7:0/747787821 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55754cdf6b60 con 0x55754cec8040 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.342+0000 7f9516ffd700 10 client.25960 handle_command_reply: tid=0 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.342+0000 7f9551cc9b80 1 client.25960 shutdown 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.342+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 msgr2=0x55754cebc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.342+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754cebc130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9520006d30 tx=0x7f9520004750 comp rx=0 tx=0).stop 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 shutdown_connections 2024-01-11T18:01:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cec8040 0x55754ceca460 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754cebc130 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceab5d0 0x55754cec36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/747787821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cebc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 >> 172.21.15.7:0/747787821 conn(0x55754ce1f740 msgr2=0x55754cea3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9516ffd700 0 client.25960 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 shutdown_connections 2024-01-11T18:01:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 1 -- 172.21.15.7:0/747787821 wait complete. 2024-01-11T18:01:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.343+0000 7f9551cc9b80 20 client.25960 trim_cache size 0 max 16384 2024-01-11T18:01:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.370+0000 7f9551cc9b80 1 Processor -- start 2024-01-11T18:01:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.370+0000 7f9551cc9b80 1 -- start start 2024-01-11T18:01:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754cec3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cc74640 con 0x55754ceaac10 2024-01-11T18:01:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754ceaba70 con 0x55754ceab5d0 2024-01-11T18:01:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cea1560 con 0x55754cea9f20 2024-01-11T18:01:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f952effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f952effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35850/0 (socket says 172.21.15.7:35850) 2024-01-11T18:01:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f952effd700 1 -- 172.21.15.7:0/1230781165 learned_addr learned my addr 172.21.15.7:0/1230781165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.371+0000 7f952effd700 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.372+0000 7f952effd700 1 -- 172.21.15.7:0/1230781165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 msgr2=0x55754ceca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.372+0000 7f952effd700 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.372+0000 7f952effd700 1 -- 172.21.15.7:0/1230781165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 msgr2=0x55754cec3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.372+0000 7f952effd700 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754cec3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.372+0000 7f952effd700 1 -- 172.21.15.7:0/1230781165 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55754cdbf1d0 con 0x55754cea9f20 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.372+0000 7f952effd700 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaaff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f952000c360 tx=0x7f9520004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b74951a98014f37 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.373+0000 7f952d7fa700 1 -- 172.21.15.7:0/1230781165 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000c6a0 con 0x55754cea9f20 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.373+0000 7f952d7fa700 1 -- 172.21.15.7:0/1230781165 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9520016e40 con 0x55754cea9f20 2024-01-11T18:01:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.373+0000 7f952d7fa700 1 -- 172.21.15.7:0/1230781165 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95200238f0 con 0x55754cea9f20 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.373+0000 7f9551cc9b80 1 -- 172.21.15.7:0/1230781165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 msgr2=0x55754ceaaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.373+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaaff0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f952000c360 tx=0x7f9520004860 comp rx=0 tx=0).stop 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 -- 172.21.15.7:0/1230781165 shutdown_connections 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754cea9f20 0x55754ceaaff0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceab5d0 0x55754ceca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/1230781165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754ceaac10 0x55754cec3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 -- 172.21.15.7:0/1230781165 >> 172.21.15.7:0/1230781165 conn(0x55754ce1f740 msgr2=0x55754cebb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 -- 172.21.15.7:0/1230781165 shutdown_connections 2024-01-11T18:01:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 -- 172.21.15.7:0/1230781165 wait complete. 2024-01-11T18:01:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.374+0000 7f9551cc9b80 1 -- start start 2024-01-11T18:01:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.375+0000 7f9551cc9b80 1 Processor -- start 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754cea9f20 0x55754cec09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cec0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754ceab5d0 0x55754cec1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cec2420 con 0x55754cea9f20 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cec2590 con 0x55754ceaac10 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55754cec2750 con 0x55754ceab5d0 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.376+0000 7f9551cc9b80 10 client.0 _ll_register_callbacks cb 0x55754cec4280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.377+0000 7f95377fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cec0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.377+0000 7f95377fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cec0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37008/0 (socket says 172.21.15.7:37008) 2024-01-11T18:01:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.377+0000 7f95377fe700 1 -- 172.21.15.7:0/424710891 learned_addr learned my addr 172.21.15.7:0/424710891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.377+0000 7f952effd700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754ceab5d0 0x55754cec1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.377+0000 7f952effd700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754cea9f20 0x55754cec09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f95377fe700 1 -- 172.21.15.7:0/424710891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754ceab5d0 msgr2=0x55754cec1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f95377fe700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754ceab5d0 0x55754cec1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f95377fe700 1 -- 172.21.15.7:0/424710891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754cea9f20 msgr2=0x55754cec09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f95377fe700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754cea9f20 0x55754cec09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f95377fe700 1 -- 172.21.15.7:0/424710891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55754cc1eb00 con 0x55754ceaac10 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f95377fe700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cec0f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55754cea67f0 tx=0x7f95280099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffce64748f1dbb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f9516ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f9516ffd700 1 -- 172.21.15.7:0/424710891 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800e040 con 0x55754ceaac10 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.378+0000 7f9516ffd700 1 -- 172.21.15.7:0/424710891 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9528007520 con 0x55754ceaac10 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.379+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95280076e0 con 0x55754ceaac10 2024-01-11T18:01:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.379+0000 7f9516ffd700 1 -- 172.21.15.7:0/424710891 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9528006000 con 0x55754ceaac10 2024-01-11T18:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.379+0000 7f9551cc9b80 10 client.25931 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.379+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55754cc76c70 con 0x55754ceaac10 2024-01-11T18:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.379+0000 7f9516ffd700 1 -- 172.21.15.7:0/424710891 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95280076e0 con 0x55754ceaac10 2024-01-11T18:01:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9516ffd700 1 -- 172.21.15.7:0/424710891 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f952800b100 con 0x55754ceaac10 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9551cc9b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9551cc9b80 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9551cc9b80 20 client.25931 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cf1e680 0x55754cf20a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9551cc9b80 4 client.25931 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55754cdf6880 con 0x55754cf1e680 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f952effd700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cf1e680 0x55754cf20a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f9516ffd700 10 client.25931 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.380+0000 7f952effd700 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cf1e680 0x55754cf20a70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65287578f2041b6 server_cookie=112b5b2b262faeb1 in_seq=0 out_seq=0 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9516ffd700 1 -- 172.21.15.7:0/424710891 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55754cdf6880 con 0x55754cf1e680 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9516ffd700 10 client.25931 handle_command_reply: tid=0 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 client.25931 shutdown 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 msgr2=0x55754cec0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cec0f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55754cea67f0 tx=0x7f95280099e0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 shutdown_connections 2024-01-11T18:01:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55754cf1e680 0x55754cf20a70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55754ceab5d0 0x55754cec1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55754cea9f20 0x55754cec09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 --2- 172.21.15.7:0/424710891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55754ceaac10 0x55754cec0f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.396+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 >> 172.21.15.7:0/424710891 conn(0x55754ce1f740 msgr2=0x55754ceca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.397+0000 7f9516ffd700 0 client.25931 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.397+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 shutdown_connections 2024-01-11T18:01:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.398+0000 7f9551cc9b80 1 -- 172.21.15.7:0/424710891 wait complete. 2024-01-11T18:01:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.398+0000 7f9551cc9b80 20 client.25931 trim_cache size 0 max 16384 2024-01-11T18:01:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:01:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:01:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:01:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:01:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.399+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95240a1ed0 msgr2=0x7f95240a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.399+0000 7f953dfff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95240a1ed0 0x7f95240a4390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9530096e00 tx=0x7f953000cea0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.399+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 msgr2=0x7f95381a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f95381a0e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f953000e940 tx=0x7f953000ec50 comp rx=0 tx=0).stop 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 shutdown_connections 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f95240a1ed0 0x7f95240a4390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9538105380 0x7f95381a0e40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381037f0 0x7f9538199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 --2- 172.21.15.7:0/1808814506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95381049e0 0x7f9538199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 >> 172.21.15.7:0/1808814506 conn(0x7f95380fed60 msgr2=0x7f9538107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 shutdown_connections 2024-01-11T18:01:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:42.400+0000 7f953dfff700 1 -- 172.21.15.7:0/1808814506 wait complete. 2024-01-11T18:01:44.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:43 smithi007 ceph-mon[33361]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-01-11T18:01:44.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 195.0 s, 815.2 tps, lat 122.529 ms stddev 149.557 2024-01-11T18:01:44.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:43 smithi016 ceph-mon[43195]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-01-11T18:01:44.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:43 smithi038 ceph-mon[43582]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-01-11T18:01:45.774 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:45 smithi038 ceph-mon[43582]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 MiB/s wr, 678 op/s 2024-01-11T18:01:45.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:45 smithi007 ceph-mon[33361]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 MiB/s wr, 678 op/s 2024-01-11T18:01:45.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:45 smithi016 ceph-mon[43195]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 MiB/s wr, 678 op/s 2024-01-11T18:01:48.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:47 smithi007 ceph-mon[33361]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 753 op/s 2024-01-11T18:01:48.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:47 smithi016 ceph-mon[43195]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 753 op/s 2024-01-11T18:01:48.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:47 smithi038 ceph-mon[43582]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 753 op/s 2024-01-11T18:01:49.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 200.0 s, 855.7 tps, lat 113.143 ms stddev 147.788 2024-01-11T18:01:50.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:49 smithi007 ceph-mon[33361]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 MiB/s wr, 575 op/s 2024-01-11T18:01:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:49 smithi016 ceph-mon[43195]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 MiB/s wr, 575 op/s 2024-01-11T18:01:50.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:49 smithi038 ceph-mon[43582]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 MiB/s wr, 575 op/s 2024-01-11T18:01:52.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:51 smithi007 ceph-mon[33361]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 599 op/s 2024-01-11T18:01:52.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:51 smithi016 ceph-mon[43195]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 599 op/s 2024-01-11T18:01:52.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:51 smithi038 ceph-mon[43582]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 599 op/s 2024-01-11T18:01:53.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:52 smithi007 ceph-mon[33361]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 MiB/s wr, 596 op/s 2024-01-11T18:01:53.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:52 smithi016 ceph-mon[43195]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 MiB/s wr, 596 op/s 2024-01-11T18:01:53.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:52 smithi038 ceph-mon[43582]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 MiB/s wr, 596 op/s 2024-01-11T18:01:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 205.0 s, 873.9 tps, lat 112.861 ms stddev 151.796 2024-01-11T18:01:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.667+0000 7fb6051ff700 1 -- 172.21.15.7:0/1330197110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb600103140 msgr2=0x7fb6001099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.667+0000 7fb6051ff700 1 --2- 172.21.15.7:0/1330197110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb600103140 0x7fb6001099f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800b0d0 tx=0x7fb5f800b3e0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.668+0000 7fb6051ff700 1 -- 172.21.15.7:0/1330197110 shutdown_connections 2024-01-11T18:01:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.668+0000 7fb6051ff700 1 --2- 172.21.15.7:0/1330197110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001015b0 0x7fb6001019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.668+0000 7fb6051ff700 1 --2- 172.21.15.7:0/1330197110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb600103140 0x7fb6001099f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.668+0000 7fb6051ff700 1 --2- 172.21.15.7:0/1330197110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6001027a0 0x7fb600102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.668+0000 7fb6051ff700 1 -- 172.21.15.7:0/1330197110 >> 172.21.15.7:0/1330197110 conn(0x7fb6000fcb60 msgr2=0x7fb6000fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.668+0000 7fb6051ff700 1 -- 172.21.15.7:0/1330197110 shutdown_connections 2024-01-11T18:01:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.669+0000 7fb6051ff700 1 -- 172.21.15.7:0/1330197110 wait complete. 2024-01-11T18:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.673+0000 7fb6051ff700 1 Processor -- start 2024-01-11T18:01:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.674+0000 7fb6051ff700 1 -- start start 2024-01-11T18:01:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.675+0000 7fb6051ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6001015b0 0x7fb6001974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.675+0000 7fb6051ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001027a0 0x7fb6001979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.675+0000 7fb6051ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600103140 0x7fb60019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.675+0000 7fb6051ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019f020 con 0x7fb6001015b0 2024-01-11T18:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.675+0000 7fb6051ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019f160 con 0x7fb600103140 2024-01-11T18:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.675+0000 7fb6051ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019f2f0 con 0x7fb6001027a0 2024-01-11T18:01:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.676+0000 7fb5fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6001015b0 0x7fb6001974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.676+0000 7fb5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001027a0 0x7fb6001979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.676+0000 7fb5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001027a0 0x7fb6001979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52890/0 (socket says 172.21.15.7:52890) 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.676+0000 7fb5fe59c700 1 -- 172.21.15.7:0/2852714782 learned_addr learned my addr 172.21.15.7:0/2852714782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.676+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600103140 0x7fb60019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.677+0000 7fb5fed9d700 1 -- 172.21.15.7:0/2852714782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001027a0 msgr2=0x7fb6001979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.677+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001027a0 0x7fb6001979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.677+0000 7fb5fed9d700 1 -- 172.21.15.7:0/2852714782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600103140 msgr2=0x7fb60019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.677+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600103140 0x7fb60019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.677+0000 7fb5fed9d700 1 -- 172.21.15.7:0/2852714782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009d00 con 0x7fb6001015b0 2024-01-11T18:01:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.677+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6001015b0 0x7fb6001974a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c940 tx=0x7fb5f400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.678+0000 7fb5e7fff700 1 -- 172.21.15.7:0/2852714782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4007aa0 con 0x7fb6001015b0 2024-01-11T18:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.678+0000 7fb5e7fff700 1 -- 172.21.15.7:0/2852714782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f400ce90 con 0x7fb6001015b0 2024-01-11T18:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.678+0000 7fb5e7fff700 1 -- 172.21.15.7:0/2852714782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400f6b0 con 0x7fb6001015b0 2024-01-11T18:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.678+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60019f5d0 con 0x7fb6001015b0 2024-01-11T18:01:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.679+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60019fa90 con 0x7fb6001015b0 2024-01-11T18:01:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.694+0000 7fb618e62b80 1 Processor -- start 2024-01-11T18:01:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.694+0000 7fb618e62b80 1 -- start start 2024-01-11T18:01:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.697+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2952340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.697+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2953090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.697+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2959e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.697+0000 7fb618e62b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2849030 con 0x55e0f29535d0 2024-01-11T18:01:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.697+0000 7fb618e62b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28acac0 con 0x55e0f2951f20 2024-01-11T18:01:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.697+0000 7fb618e62b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28aa3e0 con 0x55e0f2952c10 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.698+0000 7fb5e7fff700 1 -- 172.21.15.7:0/2852714782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fb5f4018480 con 0x7fb6001015b0 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.698+0000 7fb5fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2959e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.698+0000 7fb5fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2959e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39214/0 (socket says 172.21.15.7:39214) 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.698+0000 7fb5fe59c700 1 -- 172.21.15.7:0/2857531601 learned_addr learned my addr 172.21.15.7:0/2857531601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.699+0000 7fb5fe59c700 1 -- 172.21.15.7:0/2857531601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 msgr2=0x55e0f2953090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.699+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2953090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.699+0000 7fb5fe59c700 1 -- 172.21.15.7:0/2857531601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 msgr2=0x55e0f2952340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.699+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2952340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.699+0000 7fb5fe59c700 1 -- 172.21.15.7:0/2857531601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f271a350 con 0x55e0f29535d0 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.699+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2959e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000bea0 tx=0x7fb5f00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcab26889e3513ea server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.700+0000 7fb5e57fa700 1 -- 172.21.15.7:0/2857531601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0011440 con 0x55e0f29535d0 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.700+0000 7fb5e57fa700 1 -- 172.21.15.7:0/2857531601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f0015070 con 0x55e0f29535d0 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.700+0000 7fb5e57fa700 1 -- 172.21.15.7:0/2857531601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0019900 con 0x55e0f29535d0 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.701+0000 7fb618e62b80 1 -- 172.21.15.7:0/2857531601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 msgr2=0x55e0f2959e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.701+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2959e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000bea0 tx=0x7fb5f00106b0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.703+0000 7fb618e62b80 1 -- 172.21.15.7:0/2857531601 shutdown_connections 2024-01-11T18:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.703+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2959e60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.703+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2952340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.703+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2857531601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2953090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.703+0000 7fb618e62b80 1 -- 172.21.15.7:0/2857531601 >> 172.21.15.7:0/2857531601 conn(0x55e0f28c7740 msgr2=0x55e0f294c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.703+0000 7fb618e62b80 1 -- 172.21.15.7:0/2857531601 shutdown_connections 2024-01-11T18:01:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.704+0000 7fb618e62b80 1 -- 172.21.15.7:0/2857531601 wait complete. 2024-01-11T18:01:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.704+0000 7fb5e7fff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1ef0 0x7fb5e80a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.704+0000 7fb5e7fff700 1 -- 172.21.15.7:0/2852714782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5f4013070 con 0x7fb6001015b0 2024-01-11T18:01:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.705+0000 7fb618e62b80 1 -- start start 2024-01-11T18:01:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.705+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1ef0 0x7fb5e80a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.705+0000 7fb618e62b80 1 Processor -- start 2024-01-11T18:01:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.707+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2957530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.708+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2957a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.708+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2958aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb5fed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2957530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb5fed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2957530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39796/0 (socket says 172.21.15.7:39796) 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb5fed9d700 1 -- 172.21.15.7:0/3269462689 learned_addr learned my addr 172.21.15.7:0/3269462689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1ef0 0x7fb5e80a43b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0010ab0 tx=0x7fb5f00193d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28a5cd0 con 0x55e0f29535d0 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2814340 con 0x55e0f2951f20 2024-01-11T18:01:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.709+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2813fe0 con 0x55e0f2952c10 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2957a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb618e62b80 10 client.0 _ll_register_callbacks cb 0x55e0f28466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb5fed9d700 1 -- 172.21.15.7:0/3269462689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 msgr2=0x55e0f2957a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2957a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb5fed9d700 1 -- 172.21.15.7:0/3269462689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 msgr2=0x55e0f2958aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2958aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.710+0000 7fb5fed9d700 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f000bad0 con 0x55e0f2951f20 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.713+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2957530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4096d00 tx=0x7fb5f409a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5354621b4499ca86 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.713+0000 7fb5e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.713+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/3269462689 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40850f0 con 0x55e0f2951f20 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.713+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/3269462689 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5f409af00 con 0x55e0f2951f20 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.713+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0f2944d60 con 0x55e0f2951f20 2024-01-11T18:01:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.713+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/3269462689 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5f409a800 con 0x55e0f2951f20 2024-01-11T18:01:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.714+0000 7fb618e62b80 10 client.25937 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.714+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e0f26c6b00 con 0x55e0f2951f20 2024-01-11T18:01:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.715+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/3269462689 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0f2944d60 con 0x55e0f2951f20 2024-01-11T18:01:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.715+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/3269462689 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb5f40a2670 con 0x55e0f2951f20 2024-01-11T18:01:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.715+0000 7fb618e62b80 10 client.25937 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.715+0000 7fb618e62b80 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.715+0000 7fb618e62b80 20 client.25937 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.716+0000 7fb618e62b80 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f2970040 0x55e0f2972460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.716+0000 7fb618e62b80 4 client.25937 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.716+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0f289eb60 con 0x55e0f2970040 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.716+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f2970040 0x55e0f2972460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.717+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f2970040 0x55e0f2972460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c3db934bfd7369 server_cookie=f813296483bc3526 in_seq=0 out_seq=0 2024-01-11T18:01:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.717+0000 7fb5e2ff5700 10 client.25937 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.718+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/3269462689 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0f289eb60 con 0x55e0f2970040 2024-01-11T18:01:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.718+0000 7fb5e2ff5700 10 client.25937 handle_command_reply: tid=0 2024-01-11T18:01:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.718+0000 7fb618e62b80 1 client.25937 shutdown 2024-01-11T18:01:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.718+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 msgr2=0x55e0f2957530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.718+0000 7fb618e62b80 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2957530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4096d00 tx=0x7fb5f409a260 comp rx=0 tx=0).stop 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.719+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 shutdown_connections 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.719+0000 7fb618e62b80 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f2970040 0x55e0f2972460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.719+0000 7fb618e62b80 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f2958aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.719+0000 7fb618e62b80 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f2957530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.719+0000 7fb618e62b80 1 --2- 172.21.15.7:0/3269462689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f2957a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.719+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 >> 172.21.15.7:0/3269462689 conn(0x55e0f28c7740 msgr2=0x55e0f294bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.720+0000 7fb5e2ff5700 0 client.25937 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.722+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 shutdown_connections 2024-01-11T18:01:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.722+0000 7fb618e62b80 1 -- 172.21.15.7:0/3269462689 wait complete. 2024-01-11T18:01:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.722+0000 7fb618e62b80 20 client.25937 trim_cache size 0 max 16384 2024-01-11T18:01:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.754+0000 7fb618e62b80 1 Processor -- start 2024-01-11T18:01:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.754+0000 7fb618e62b80 1 -- start start 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2951f20 0x55e0f295c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f2952c10 0x55e0f2976dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f29535d0 0x55e0f295ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb618e62b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f271c640 con 0x55e0f2952c10 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb618e62b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f295a290 con 0x55e0f29535d0 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb618e62b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f29582a0 con 0x55e0f2951f20 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb5fed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2951f20 0x55e0f295c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb5fed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2951f20 0x55e0f295c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52910/0 (socket says 172.21.15.7:52910) 2024-01-11T18:01:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.755+0000 7fb5fed9d700 1 -- 172.21.15.7:0/4183320075 learned_addr learned my addr 172.21.15.7:0/4183320075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.756+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f29535d0 0x55e0f295ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.756+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f2952c10 0x55e0f2976dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.757+0000 7fb5ff59e700 1 -- 172.21.15.7:0/4183320075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2951f20 msgr2=0x55e0f295c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.757+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2951f20 0x55e0f295c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.757+0000 7fb5ff59e700 1 -- 172.21.15.7:0/4183320075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f2952c10 msgr2=0x55e0f2976dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.757+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f2952c10 0x55e0f2976dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.757+0000 7fb5ff59e700 1 -- 172.21.15.7:0/4183320075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f28671d0 con 0x55e0f29535d0 2024-01-11T18:01:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.758+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f29535d0 0x55e0f295ceb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55e0f294d930 tx=0x7fb5f80060c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282b805adf421629 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.761+0000 7fb5e4ff9700 1 -- 172.21.15.7:0/4183320075 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800e030 con 0x55e0f29535d0 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.761+0000 7fb5e4ff9700 1 -- 172.21.15.7:0/4183320075 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f80076e0 con 0x55e0f29535d0 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.761+0000 7fb5e4ff9700 1 -- 172.21.15.7:0/4183320075 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80052c0 con 0x55e0f29535d0 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 -- 172.21.15.7:0/4183320075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f29535d0 msgr2=0x55e0f295ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f29535d0 0x55e0f295ceb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55e0f294d930 tx=0x7fb5f80060c0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 -- 172.21.15.7:0/4183320075 shutdown_connections 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2951f20 0x55e0f295c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f29535d0 0x55e0f295ceb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 --2- 172.21.15.7:0/4183320075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f2952c10 0x55e0f2976dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 -- 172.21.15.7:0/4183320075 >> 172.21.15.7:0/4183320075 conn(0x55e0f28c7740 msgr2=0x55e0f2956ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 -- 172.21.15.7:0/4183320075 shutdown_connections 2024-01-11T18:01:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.762+0000 7fb618e62b80 1 -- 172.21.15.7:0/4183320075 wait complete. 2024-01-11T18:01:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.763+0000 7fb618e62b80 1 -- start start 2024-01-11T18:01:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.763+0000 7fb618e62b80 1 Processor -- start 2024-01-11T18:01:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.765+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f296a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.765+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f296a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.765+0000 7fb618e62b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f296b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.766+0000 7fb5ff59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f296b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.766+0000 7fb618e62b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f296bd90 con 0x55e0f29535d0 2024-01-11T18:01:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb5fed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f296a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f296a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb618e62b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f296bf00 con 0x55e0f2951f20 2024-01-11T18:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f296a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:52922/0 (socket says 172.21.15.7:52922) 2024-01-11T18:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb5fe59c700 1 -- 172.21.15.7:0/2250695368 learned_addr learned my addr 172.21.15.7:0/2250695368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f296c0c0 con 0x55e0f2952c10 2024-01-11T18:01:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.767+0000 7fb618e62b80 10 client.0 _ll_register_callbacks cb 0x55e0f28466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.768+0000 7fb5ff59e700 1 -- 172.21.15.7:0/2250695368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 msgr2=0x55e0f296a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.772+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f296a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.772+0000 7fb5ff59e700 1 -- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 msgr2=0x55e0f296a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.772+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f296a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.772+0000 7fb5ff59e700 1 -- 172.21.15.7:0/2250695368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60010a340 con 0x55e0f29535d0 2024-01-11T18:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.772+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f296a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:01:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.772+0000 7fb5fe59c700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f296a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.773+0000 7fb5e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.773+0000 7fb5ff59e700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f296b850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8007aa0 tx=0x7fb5f80050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccb2d76ab022a4f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:01:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.774+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/2250695368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800e040 con 0x55e0f29535d0 2024-01-11T18:01:55.773 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:55 smithi038 ceph-mon[43582]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 558 op/s 2024-01-11T18:01:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.774+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/2250695368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5f802a030 con 0x55e0f29535d0 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.775+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5f0015070 con 0x55e0f29535d0 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.775+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/2250695368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5f8005260 con 0x55e0f29535d0 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.775+0000 7fb618e62b80 10 client.15870 fetch_fsmap learned FSMap version 29 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.775+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e0f271ec70 con 0x55e0f29535d0 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.776+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/2250695368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f0015070 con 0x55e0f29535d0 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.776+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/2250695368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb5f800b100 con 0x55e0f29535d0 2024-01-11T18:01:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.776+0000 7fb618e62b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.776+0000 7fb618e62b80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.776+0000 7fb618e62b80 20 client.15870 populate_metadata read hostname 'smithi007' 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.777+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f29c6c10 0x55e0f29c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.777+0000 7fb618e62b80 4 client.15870 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.777+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0f289e880 con 0x55e0f29c6c10 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.777+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f29c6c10 0x55e0f29c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:01:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.778+0000 7fb5fed9d700 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f29c6c10 0x55e0f29c9030 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413332ff8be6c5d5 server_cookie=3e49ac4cb477f873 in_seq=0 out_seq=0 2024-01-11T18:01:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.778+0000 7fb5e2ff5700 10 client.15870 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:55.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:55 smithi007 ceph-mon[33361]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 558 op/s 2024-01-11T18:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.822+0000 7fb5e2ff5700 1 -- 172.21.15.7:0/2250695368 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0f289e880 con 0x55e0f29c6c10 2024-01-11T18:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.822+0000 7fb5e2ff5700 10 client.15870 handle_command_reply: tid=0 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.822+0000 7fb618e62b80 1 client.15870 shutdown 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.822+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 msgr2=0x55e0f296b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.822+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f296b850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8007aa0 tx=0x7fb5f80050f0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 shutdown_connections 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e0f29c6c10 0x55e0f29c9030 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0f29535d0 0x55e0f296b850 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e0f2951f20 0x55e0f296a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 --2- 172.21.15.7:0/2250695368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e0f2952c10 0x55e0f296a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb5e2ff5700 0 client.15870 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 >> 172.21.15.7:0/2250695368 conn(0x55e0f28c7740 msgr2=0x55e0f295ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 shutdown_connections 2024-01-11T18:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 1 -- 172.21.15.7:0/2250695368 wait complete. 2024-01-11T18:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.823+0000 7fb618e62b80 20 client.15870 trim_cache size 0 max 16384 2024-01-11T18:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1ef0 msgr2=0x7fb5e80a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1ef0 0x7fb5e80a43b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0010ab0 tx=0x7fb5f00193d0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6001015b0 msgr2=0x7fb6001974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6001015b0 0x7fb6001974a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c940 tx=0x7fb5f400cc50 comp rx=0 tx=0).stop 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 shutdown_connections 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fb5e80a1ef0 0x7fb5e80a43b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600103140 0x7fb60019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6001015b0 0x7fb6001974a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 --2- 172.21.15.7:0/2852714782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6001027a0 0x7fb6001979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 >> 172.21.15.7:0/2852714782 conn(0x7fb6000fcb60 msgr2=0x7fb6001059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 shutdown_connections 2024-01-11T18:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:01:55.824+0000 7fb6051ff700 1 -- 172.21.15.7:0/2852714782 wait complete. 2024-01-11T18:01:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:55 smithi016 ceph-mon[43195]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 558 op/s 2024-01-11T18:01:58.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:57 smithi007 ceph-mon[33361]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 655 op/s 2024-01-11T18:01:58.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:57 smithi016 ceph-mon[43195]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 655 op/s 2024-01-11T18:01:58.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:57 smithi038 ceph-mon[43582]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 655 op/s 2024-01-11T18:01:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 210.0 s, 811.4 tps, lat 129.335 ms stddev 191.171 2024-01-11T18:02:00.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:01:59 smithi007 ceph-mon[33361]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-01-11T18:02:00.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:01:59 smithi016 ceph-mon[43195]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-01-11T18:02:00.279 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:01:59 smithi038 ceph-mon[43582]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-01-11T18:02:02.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:01 smithi007 ceph-mon[33361]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 612 op/s 2024-01-11T18:02:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:01 smithi016 ceph-mon[43195]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 612 op/s 2024-01-11T18:02:02.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:01 smithi038 ceph-mon[43582]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 612 op/s 2024-01-11T18:02:03.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:02 smithi016 ceph-mon[43195]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 635 op/s 2024-01-11T18:02:03.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:02 smithi038 ceph-mon[43582]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 635 op/s 2024-01-11T18:02:03.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:02 smithi007 ceph-mon[33361]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 635 op/s 2024-01-11T18:02:04.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 215.0 s, 743.0 tps, lat 132.427 ms stddev 170.063 2024-01-11T18:02:05.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:05 smithi007 ceph-mon[33361]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.9 MiB/s wr, 609 op/s 2024-01-11T18:02:05.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:05 smithi016 ceph-mon[43195]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.9 MiB/s wr, 609 op/s 2024-01-11T18:02:06.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:05 smithi038 ceph-mon[43582]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.9 MiB/s wr, 609 op/s 2024-01-11T18:02:08.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:07 smithi007 ceph-mon[33361]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 682 op/s 2024-01-11T18:02:08.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:07 smithi016 ceph-mon[43195]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 682 op/s 2024-01-11T18:02:08.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:07 smithi038 ceph-mon[43582]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 682 op/s 2024-01-11T18:02:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.098+0000 7f974fbff700 1 -- 172.21.15.7:0/269197992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97481049e0 msgr2=0x7f9748104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.098+0000 7f974fbff700 1 --2- 172.21.15.7:0/269197992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97481049e0 0x7f9748104e40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9744009fc0 tx=0x7f974400b330 comp rx=0 tx=0).stop 2024-01-11T18:02:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.102+0000 7f974fbff700 1 -- 172.21.15.7:0/269197992 shutdown_connections 2024-01-11T18:02:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.102+0000 7f974fbff700 1 --2- 172.21.15.7:0/269197992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 0x7f9748103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.102+0000 7f974fbff700 1 --2- 172.21.15.7:0/269197992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9748105380 0x7f974810bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.102+0000 7f974fbff700 1 --2- 172.21.15.7:0/269197992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97481049e0 0x7f9748104e40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.102+0000 7f974fbff700 1 -- 172.21.15.7:0/269197992 >> 172.21.15.7:0/269197992 conn(0x7f97480fed60 msgr2=0x7f97481011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.103+0000 7f974fbff700 1 -- 172.21.15.7:0/269197992 shutdown_connections 2024-01-11T18:02:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.104+0000 7f974fbff700 1 -- 172.21.15.7:0/269197992 wait complete. 2024-01-11T18:02:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.105+0000 7f974fbff700 1 Processor -- start 2024-01-11T18:02:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.106+0000 7f974fbff700 1 -- start start 2024-01-11T18:02:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974fbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 0x7f9748110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974fbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97481049e0 0x7f9748110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974fbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9748105380 0x7f9748115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974fbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748116290 con 0x7f97481049e0 2024-01-11T18:02:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974fbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748116400 con 0x7f9748105380 2024-01-11T18:02:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974d99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 0x7f9748110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974d99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 0x7f9748110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48838/0 (socket says 172.21.15.7:48838) 2024-01-11T18:02:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.107+0000 7f974d99b700 1 -- 172.21.15.7:0/1608259410 learned_addr learned my addr 172.21.15.7:0/1608259410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.108+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97481165c0 con 0x7f97481037f0 2024-01-11T18:02:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.108+0000 7f974e19c700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9748105380 0x7f9748115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.108+0000 7f974d19a700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97481049e0 0x7f9748110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.109+0000 7f974e19c700 1 -- 172.21.15.7:0/1608259410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 msgr2=0x7f9748110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.109+0000 7f974e19c700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 0x7f9748110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.109+0000 7f974e19c700 1 -- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97481049e0 msgr2=0x7f9748110c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.109+0000 7f974e19c700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97481049e0 0x7f9748110c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.109+0000 7f974e19c700 1 -- 172.21.15.7:0/1608259410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744009c40 con 0x7f9748105380 2024-01-11T18:02:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.109+0000 7f974e19c700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9748105380 0x7f9748115cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f974000d720 tx=0x7f974000da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.110+0000 7f973effd700 1 -- 172.21.15.7:0/1608259410 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740004220 con 0x7f9748105380 2024-01-11T18:02:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.110+0000 7f973effd700 1 -- 172.21.15.7:0/1608259410 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9740004cf0 con 0x7f9748105380 2024-01-11T18:02:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.110+0000 7f973effd700 1 -- 172.21.15.7:0/1608259410 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000f690 con 0x7f9748105380 2024-01-11T18:02:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.110+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9748116840 con 0x7f9748105380 2024-01-11T18:02:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.110+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97481aea80 con 0x7f9748105380 2024-01-11T18:02:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.117+0000 7f973effd700 1 -- 172.21.15.7:0/1608259410 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f9740004410 con 0x7f9748105380 2024-01-11T18:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.121+0000 7f973effd700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f97340a20a0 0x7f97340a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.121+0000 7f973effd700 1 -- 172.21.15.7:0/1608259410 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f97400cd900 con 0x7f9748105380 2024-01-11T18:02:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 220.0 s, 638.8 tps, lat 138.864 ms stddev 220.669 2024-01-11T18:02:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.121+0000 7f974d99b700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f97340a20a0 0x7f97340a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.123+0000 7f974d99b700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f97340a20a0 0x7f97340a4560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9738009cf0 tx=0x7f9738009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.123+0000 7f97639bfb80 1 Processor -- start 2024-01-11T18:02:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.123+0000 7f97639bfb80 1 -- start start 2024-01-11T18:02:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.124+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d1b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.124+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.124+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.124+0000 7f97639bfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545c12030 con 0x563545d1bc10 2024-01-11T18:02:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.124+0000 7f97639bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545c75ac0 con 0x563545d1c5d0 2024-01-11T18:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.124+0000 7f97639bfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545c733e0 con 0x563545d1af20 2024-01-11T18:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.125+0000 7f974d99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d1b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.125+0000 7f974d99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d1b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48848/0 (socket says 172.21.15.7:48848) 2024-01-11T18:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.125+0000 7f974d99b700 1 -- 172.21.15.7:0/3983846867 learned_addr learned my addr 172.21.15.7:0/3983846867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.125+0000 7f974e19c700 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.125+0000 7f974d19a700 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.125+0000 7f974e19c700 1 -- 172.21.15.7:0/3983846867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 msgr2=0x563545d1b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.126+0000 7f974e19c700 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d1b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.126+0000 7f974e19c700 1 -- 172.21.15.7:0/3983846867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 msgr2=0x563545d1c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.126+0000 7f974e19c700 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d1c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.126+0000 7f974e19c700 1 -- 172.21.15.7:0/3983846867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563545ae3350 con 0x563545d1c5d0 2024-01-11T18:02:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.127+0000 7f974e19c700 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d22e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9740005950 tx=0x7f9740004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddcf05310d840dd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.128+0000 7f9733fff700 1 -- 172.21.15.7:0/3983846867 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974009a2d0 con 0x563545d1c5d0 2024-01-11T18:02:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.128+0000 7f9733fff700 1 -- 172.21.15.7:0/3983846867 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f974009e420 con 0x563545d1c5d0 2024-01-11T18:02:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.128+0000 7f9733fff700 1 -- 172.21.15.7:0/3983846867 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974009a2d0 con 0x563545d1c5d0 2024-01-11T18:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.129+0000 7f97639bfb80 1 -- 172.21.15.7:0/3983846867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 msgr2=0x563545d22e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.129+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d22e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9740005950 tx=0x7f9740004380 comp rx=0 tx=0).stop 2024-01-11T18:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 -- 172.21.15.7:0/3983846867 shutdown_connections 2024-01-11T18:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d1b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d22e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3983846867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 -- 172.21.15.7:0/3983846867 >> 172.21.15.7:0/3983846867 conn(0x563545c90740 msgr2=0x563545d15950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 -- 172.21.15.7:0/3983846867 shutdown_connections 2024-01-11T18:02:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.132+0000 7f97639bfb80 1 -- 172.21.15.7:0/3983846867 wait complete. 2024-01-11T18:02:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.133+0000 7f97639bfb80 1 -- start start 2024-01-11T18:02:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.133+0000 7f97639bfb80 1 Processor -- start 2024-01-11T18:02:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.135+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1af20 0x563545d24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.135+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 0x563545d24c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.135+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1c5d0 0x563545d2bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f974d19a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 0x563545d24c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f974d19a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 0x563545d24c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38326/0 (socket says 172.21.15.7:38326) 2024-01-11T18:02:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f97639bfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545c6ecd0 con 0x563545d1af20 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f97639bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545bdd340 con 0x563545d1bc10 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f97639bfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545bdcfe0 con 0x563545d1c5d0 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f97639bfb80 10 client.0 _ll_register_callbacks cb 0x563545c0f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.136+0000 7f974d19a700 1 -- 172.21.15.7:0/1058252885 learned_addr learned my addr 172.21.15.7:0/1058252885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.137+0000 7f974d19a700 1 -- 172.21.15.7:0/1058252885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1c5d0 msgr2=0x563545d2bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.137+0000 7f974d19a700 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1c5d0 0x563545d2bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.137+0000 7f974d19a700 1 -- 172.21.15.7:0/1058252885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1af20 msgr2=0x563545d24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.137+0000 7f974d19a700 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1af20 0x563545d24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.137+0000 7f974d19a700 1 -- 172.21.15.7:0/1058252885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97481aea80 con 0x563545d1bc10 2024-01-11T18:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.138+0000 7f974d19a700 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 0x563545d24c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9744007a30 tx=0x7f9744009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b26d6d91956462 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.138+0000 7f97317fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.138+0000 7f97317fa700 1 -- 172.21.15.7:0/1058252885 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974400e050 con 0x563545d1bc10 2024-01-11T18:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.138+0000 7f97317fa700 1 -- 172.21.15.7:0/1058252885 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9744004c50 con 0x563545d1bc10 2024-01-11T18:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.139+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563545d0dd60 con 0x563545d1bc10 2024-01-11T18:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.139+0000 7f97317fa700 1 -- 172.21.15.7:0/1058252885 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9744004e10 con 0x563545d1bc10 2024-01-11T18:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.139+0000 7f97639bfb80 10 client.25952 fetch_fsmap learned FSMap version 29 2024-01-11T18:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.139+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563545a8fb00 con 0x563545d1bc10 2024-01-11T18:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.140+0000 7f97317fa700 1 -- 172.21.15.7:0/1058252885 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563545d0dd60 con 0x563545d1bc10 2024-01-11T18:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.140+0000 7f97317fa700 1 -- 172.21.15.7:0/1058252885 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f974400b050 con 0x563545d1bc10 2024-01-11T18:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.140+0000 7f97639bfb80 10 client.25952 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.140+0000 7f97639bfb80 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.140+0000 7f97639bfb80 20 client.25952 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.141+0000 7f97639bfb80 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d30630 0x563545d32a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.141+0000 7f97639bfb80 4 client.25952 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.141+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563545c67b60 con 0x563545d30630 2024-01-11T18:02:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.141+0000 7f974d99b700 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d30630 0x563545d32a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.142+0000 7f97317fa700 10 client.25952 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.145+0000 7f974d99b700 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d30630 0x563545d32a50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae7c93642f71872 server_cookie=270a18c8739a6ecd in_seq=0 out_seq=0 2024-01-11T18:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.146+0000 7f97317fa700 1 -- 172.21.15.7:0/1058252885 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563545c67b60 con 0x563545d30630 2024-01-11T18:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.146+0000 7f97317fa700 10 client.25952 handle_command_reply: tid=0 2024-01-11T18:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.146+0000 7f97639bfb80 1 client.25952 shutdown 2024-01-11T18:02:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 msgr2=0x563545d24c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 0x563545d24c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9744007a30 tx=0x7f9744009510 comp rx=0 tx=0).stop 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 shutdown_connections 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d30630 0x563545d32a50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1c5d0 0x563545d2bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1af20 0x563545d24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97317fa700 0 client.25952 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 --2- 172.21.15.7:0/1058252885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1bc10 0x563545d24c80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.147+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 >> 172.21.15.7:0/1058252885 conn(0x563545c90740 msgr2=0x563545d14f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.148+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 shutdown_connections 2024-01-11T18:02:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.149+0000 7f97639bfb80 1 -- 172.21.15.7:0/1058252885 wait complete. 2024-01-11T18:02:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.149+0000 7f97639bfb80 20 client.25952 trim_cache size 0 max 16384 2024-01-11T18:02:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.183+0000 7f97639bfb80 1 Processor -- start 2024-01-11T18:02:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.183+0000 7f97639bfb80 1 -- start start 2024-01-11T18:02:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.184+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 0x563545d0d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.184+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1bc10 0x563545d2bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.184+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1c5d0 0x563545d1b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.184+0000 7f97639bfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545ae5640 con 0x563545d1c5d0 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.184+0000 7f97639bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545d17690 con 0x563545d1af20 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.184+0000 7f97639bfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545d25930 con 0x563545d1bc10 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 0x563545d0d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 0x563545d0d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38328/0 (socket says 172.21.15.7:38328) 2024-01-11T18:02:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 -- 172.21.15.7:0/3389148440 learned_addr learned my addr 172.21.15.7:0/3389148440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 -- 172.21.15.7:0/3389148440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1bc10 msgr2=0x563545d2bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1bc10 0x563545d2bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 -- 172.21.15.7:0/3389148440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1c5d0 msgr2=0x563545d1b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1c5d0 0x563545d1b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.185+0000 7f974d99b700 1 -- 172.21.15.7:0/3389148440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563545c301d0 con 0x563545d1af20 2024-01-11T18:02:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.186+0000 7f974d99b700 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 0x563545d0d8a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x563545d33350 tx=0x7f9738013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2eae83fecdc4c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.187+0000 7f97337fe700 1 -- 172.21.15.7:0/3389148440 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738022040 con 0x563545d1af20 2024-01-11T18:02:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.187+0000 7f97337fe700 1 -- 172.21.15.7:0/3389148440 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f973802c070 con 0x563545d1af20 2024-01-11T18:02:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.187+0000 7f97337fe700 1 -- 172.21.15.7:0/3389148440 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973801d8d0 con 0x563545d1af20 2024-01-11T18:02:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 -- 172.21.15.7:0/3389148440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 msgr2=0x563545d0d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 0x563545d0d8a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x563545d33350 tx=0x7f9738013040 comp rx=0 tx=0).stop 2024-01-11T18:02:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 -- 172.21.15.7:0/3389148440 shutdown_connections 2024-01-11T18:02:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1c5d0 0x563545d1b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1af20 0x563545d0d8a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 --2- 172.21.15.7:0/3389148440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1bc10 0x563545d2bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.188+0000 7f97639bfb80 1 -- 172.21.15.7:0/3389148440 >> 172.21.15.7:0/3389148440 conn(0x563545c90740 msgr2=0x563545d23cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.193+0000 7f97639bfb80 1 -- 172.21.15.7:0/3389148440 shutdown_connections 2024-01-11T18:02:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.193+0000 7f97639bfb80 1 -- 172.21.15.7:0/3389148440 wait complete. 2024-01-11T18:02:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.193+0000 7f97639bfb80 1 -- start start 2024-01-11T18:02:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 Processor -- start 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d2a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545d2add0 con 0x563545d1bc10 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545d2af40 con 0x563545d1c5d0 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.194+0000 7f97639bfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563545d2b100 con 0x563545d1af20 2024-01-11T18:02:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.195+0000 7f97639bfb80 10 client.0 _ll_register_callbacks cb 0x563545c0f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.195+0000 7f974d99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.195+0000 7f974e19c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d2a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.195+0000 7f974d99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d273a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48878/0 (socket says 172.21.15.7:48878) 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.195+0000 7f974d99b700 1 -- 172.21.15.7:0/943523369 learned_addr learned my addr 172.21.15.7:0/943523369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f974d99b700 1 -- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 msgr2=0x563545d2a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f974d99b700 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d2a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f974d99b700 1 -- 172.21.15.7:0/943523369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 msgr2=0x563545d278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f974d99b700 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f974d99b700 1 -- 172.21.15.7:0/943523369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9738017070 con 0x563545d1af20 2024-01-11T18:02:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f974d99b700 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d273a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9738005e50 tx=0x7f97380284b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6536b0fabe6107f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.196+0000 7f97317fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:02:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.197+0000 7f97317fa700 1 -- 172.21.15.7:0/943523369 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738022070 con 0x563545d1af20 2024-01-11T18:02:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.197+0000 7f97317fa700 1 -- 172.21.15.7:0/943523369 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9738003c60 con 0x563545d1af20 2024-01-11T18:02:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.197+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f974009e420 con 0x563545d1af20 2024-01-11T18:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.200+0000 7f97317fa700 1 -- 172.21.15.7:0/943523369 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9738003e20 con 0x563545d1af20 2024-01-11T18:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.200+0000 7f97639bfb80 10 client.25987 fetch_fsmap learned FSMap version 29 2024-01-11T18:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.200+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563545ae7c70 con 0x563545d1af20 2024-01-11T18:02:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.201+0000 7f97317fa700 1 -- 172.21.15.7:0/943523369 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f974009e420 con 0x563545d1af20 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97317fa700 1 -- 172.21.15.7:0/943523369 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9738005410 con 0x563545d1af20 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97639bfb80 10 client.25987 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97639bfb80 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97639bfb80 20 client.25987 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97639bfb80 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d87420 0x563545d89840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97639bfb80 4 client.25987 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.203+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563545c67880 con 0x563545d87420 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.204+0000 7f974d19a700 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d87420 0x563545d89840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.204+0000 7f974d19a700 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d87420 0x563545d89840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec616c2aae86c863 server_cookie=85c07c87a95faa68 in_seq=0 out_seq=0 2024-01-11T18:02:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:09.204+0000 7f97317fa700 10 client.25987 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:10.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:09 smithi007 ceph-mon[33361]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.0 MiB/s wr, 587 op/s 2024-01-11T18:02:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.221+0000 7f97317fa700 1 -- 172.21.15.7:0/943523369 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563545c67880 con 0x563545d87420 2024-01-11T18:02:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.221+0000 7f97317fa700 10 client.25987 handle_command_reply: tid=0 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.221+0000 7f97639bfb80 1 client.25987 shutdown 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.221+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 msgr2=0x563545d273a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.221+0000 7f97639bfb80 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d273a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9738005e50 tx=0x7f97380284b0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 shutdown_connections 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563545d87420 0x563545d89840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563545d1af20 0x563545d273a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97317fa700 0 client.25987 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563545d1c5d0 0x563545d2a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 --2- 172.21.15.7:0/943523369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563545d1bc10 0x563545d278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 >> 172.21.15.7:0/943523369 conn(0x563545c90740 msgr2=0x563545d373d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 shutdown_connections 2024-01-11T18:02:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 1 -- 172.21.15.7:0/943523369 wait complete. 2024-01-11T18:02:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.222+0000 7f97639bfb80 20 client.25987 trim_cache size 0 max 16384 2024-01-11T18:02:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:02:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:02:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:02:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.223+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f97340a20a0 msgr2=0x7f97340a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.223+0000 7f974fbff700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f97340a20a0 0x7f97340a4560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9738009cf0 tx=0x7f9738009310 comp rx=0 tx=0).stop 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.223+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9748105380 msgr2=0x7f9748115cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.223+0000 7f974fbff700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9748105380 0x7f9748115cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f974000d720 tx=0x7f974000da30 comp rx=0 tx=0).stop 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 shutdown_connections 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f97340a20a0 0x7f97340a4560 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97481037f0 0x7f9748110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9748105380 0x7f9748115cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 --2- 172.21.15.7:0/1608259410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97481049e0 0x7f9748110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 >> 172.21.15.7:0/1608259410 conn(0x7f97480fed60 msgr2=0x7f9748079190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 shutdown_connections 2024-01-11T18:02:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:10.224+0000 7f974fbff700 1 -- 172.21.15.7:0/1608259410 wait complete. 2024-01-11T18:02:10.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:09 smithi016 ceph-mon[43195]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.0 MiB/s wr, 587 op/s 2024-01-11T18:02:10.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:09 smithi038 ceph-mon[43582]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.0 MiB/s wr, 587 op/s 2024-01-11T18:02:11.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:10 smithi016 ceph-mon[43195]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-01-11T18:02:11.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:10 smithi038 ceph-mon[43582]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-01-11T18:02:11.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:10 smithi007 ceph-mon[33361]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-01-11T18:02:14.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:13 smithi007 ceph-mon[33361]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-01-11T18:02:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 225.0 s, 695.6 tps, lat 163.645 ms stddev 278.876 2024-01-11T18:02:14.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:13 smithi016 ceph-mon[43195]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-01-11T18:02:14.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:13 smithi038 ceph-mon[43582]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-01-11T18:02:15.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:15 smithi007 ceph-mon[33361]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-01-11T18:02:15.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:15 smithi016 ceph-mon[43195]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-01-11T18:02:16.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:15 smithi038 ceph-mon[43582]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-01-11T18:02:18.070 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:17 smithi007 ceph-mon[33361]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-01-11T18:02:18.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:17 smithi016 ceph-mon[43195]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-01-11T18:02:18.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:17 smithi038 ceph-mon[43582]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-01-11T18:02:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 230.0 s, 732.2 tps, lat 130.680 ms stddev 186.530 2024-01-11T18:02:20.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:19 smithi007 ceph-mon[33361]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.36k op/s 2024-01-11T18:02:20.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:19 smithi016 ceph-mon[43195]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.36k op/s 2024-01-11T18:02:20.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:19 smithi038 ceph-mon[43582]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.36k op/s 2024-01-11T18:02:22.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:21 smithi016 ceph-mon[43195]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-01-11T18:02:22.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:21 smithi038 ceph-mon[43582]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-01-11T18:02:22.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:21 smithi007 ceph-mon[33361]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-01-11T18:02:23.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:22 smithi016 ceph-mon[43195]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-01-11T18:02:23.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:22 smithi038 ceph-mon[43582]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-01-11T18:02:23.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:22 smithi007 ceph-mon[33361]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-01-11T18:02:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.499+0000 7f42461ff700 1 -- 172.21.15.7:0/2618642031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 msgr2=0x7f4240104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.499+0000 7f42461ff700 1 --2- 172.21.15.7:0/2618642031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240104410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4230009f90 tx=0x7f423000b330 comp rx=0 tx=0).stop 2024-01-11T18:02:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 -- 172.21.15.7:0/2618642031 shutdown_connections 2024-01-11T18:02:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 --2- 172.21.15.7:0/2618642031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240104410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 --2- 172.21.15.7:0/2618642031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4240107280 0x7f42401096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 --2- 172.21.15.7:0/2618642031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4240104950 0x7f4240106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 -- 172.21.15.7:0/2618642031 >> 172.21.15.7:0/2618642031 conn(0x7f42400fba30 msgr2=0x7f42400fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 -- 172.21.15.7:0/2618642031 shutdown_connections 2024-01-11T18:02:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.500+0000 7f42461ff700 1 -- 172.21.15.7:0/2618642031 wait complete. 2024-01-11T18:02:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.504+0000 7f42461ff700 1 Processor -- start 2024-01-11T18:02:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.504+0000 7f42461ff700 1 -- start start 2024-01-11T18:02:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.505+0000 7f42461ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.505+0000 7f42461ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4240104950 0x7f42401979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.505+0000 7f42461ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4240107280 0x7f424019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f42461ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424019f010 con 0x7f4240104950 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f42461ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424019f150 con 0x7f4240107280 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f42461ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424019f2e0 con 0x7f4240102020 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42330/0 (socket says 172.21.15.7:42330) 2024-01-11T18:02:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 -- 172.21.15.7:0/2224794229 learned_addr learned my addr 172.21.15.7:0/2224794229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423ffff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4240107280 0x7f424019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 -- 172.21.15.7:0/2224794229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4240107280 msgr2=0x7f424019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4240107280 0x7f424019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 -- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4240104950 msgr2=0x7f42401979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4240104950 0x7f42401979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.506+0000 7f423f7fe700 1 -- 172.21.15.7:0/2224794229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280095f0 con 0x7f4240102020 2024-01-11T18:02:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.507+0000 7f423f7fe700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240197490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4230007c10 tx=0x7f4230004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.508+0000 7f423d7fa700 1 -- 172.21.15.7:0/2224794229 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000e040 con 0x7f4240102020 2024-01-11T18:02:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.508+0000 7f423d7fa700 1 -- 172.21.15.7:0/2224794229 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f423000be00 con 0x7f4240102020 2024-01-11T18:02:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.508+0000 7f423d7fa700 1 -- 172.21.15.7:0/2224794229 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230005610 con 0x7f4240102020 2024-01-11T18:02:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.508+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4230009c40 con 0x7f4240102020 2024-01-11T18:02:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.509+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4240100150 con 0x7f4240102020 2024-01-11T18:02:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.517+0000 7f4259fe4b80 1 Processor -- start 2024-01-11T18:02:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.518+0000 7f4259fe4b80 1 -- start start 2024-01-11T18:02:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 0x56326db3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326da2f030 con 0x56326db38c10 2024-01-11T18:02:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326da92ac0 con 0x56326db37f20 2024-01-11T18:02:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326da903e0 con 0x56326db395d0 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f423d7fa700 1 -- 172.21.15.7:0/2224794229 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4230005770 con 0x7f4240102020 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4237fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 0x56326db3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4237fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 0x56326db3fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42332/0 (socket says 172.21.15.7:42332) 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f4237fff700 1 -- 172.21.15.7:0/2239766955 learned_addr learned my addr 172.21.15.7:0/2239766955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.519+0000 7f423f7fe700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f423ffff700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f4237fff700 1 -- 172.21.15.7:0/2239766955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 msgr2=0x56326db38340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f4237fff700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f4237fff700 1 -- 172.21.15.7:0/2239766955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 msgr2=0x56326db39090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f4237fff700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db39090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f4237fff700 1 -- 172.21.15.7:0/2239766955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56326d900350 con 0x56326db395d0 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f423ffff700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.520+0000 7f423f7fe700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.521+0000 7f423d7fa700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f422c0a2010 0x7f422c0a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.522+0000 7f423d7fa700 1 -- 172.21.15.7:0/2224794229 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4230022030 con 0x7f4240102020 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.522+0000 7f4237fff700 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 0x56326db3fe60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4228009f30 tx=0x7f422800e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70473502e4e4fb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.522+0000 7f4235ffb700 1 -- 172.21.15.7:0/2239766955 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228009a80 con 0x56326db395d0 2024-01-11T18:02:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.523+0000 7f4235ffb700 1 -- 172.21.15.7:0/2239766955 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422800ee50 con 0x56326db395d0 2024-01-11T18:02:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.523+0000 7f4235ffb700 1 -- 172.21.15.7:0/2239766955 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228029930 con 0x56326db395d0 2024-01-11T18:02:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.523+0000 7f423ffff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f422c0a2010 0x7f422c0a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.524+0000 7f4259fe4b80 1 -- 172.21.15.7:0/2239766955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 msgr2=0x56326db3fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.524+0000 7f423ffff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f422c0a2010 0x7f422c0a44d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4238005fd0 tx=0x7f423800c700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.524+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 0x56326db3fe60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4228009f30 tx=0x7f422800e630 comp rx=0 tx=0).stop 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.525+0000 7f4259fe4b80 1 -- 172.21.15.7:0/2239766955 shutdown_connections 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.525+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db395d0 0x56326db3fe60 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4228009f30 tx=0x7f422800e630 comp rx=0 tx=0).stop 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.525+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.525+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/2239766955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.525+0000 7f4259fe4b80 1 -- 172.21.15.7:0/2239766955 >> 172.21.15.7:0/2239766955 conn(0x56326daad740 msgr2=0x56326db32950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.526+0000 7f4259fe4b80 1 -- 172.21.15.7:0/2239766955 shutdown_connections 2024-01-11T18:02:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.526+0000 7f4259fe4b80 1 -- 172.21.15.7:0/2239766955 wait complete. 2024-01-11T18:02:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.527+0000 7f4259fe4b80 1 -- start start 2024-01-11T18:02:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.528+0000 7f4259fe4b80 1 Processor -- start 2024-01-11T18:02:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.530+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db4a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.531+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db4a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.531+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.531+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326da8bcd0 con 0x56326db38c10 2024-01-11T18:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.531+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326d9fa340 con 0x56326db37f20 2024-01-11T18:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.531+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326d9f9fe0 con 0x56326db4f910 2024-01-11T18:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.532+0000 7f4237fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db4a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.532+0000 7f4259fe4b80 10 client.0 _ll_register_callbacks cb 0x56326da2c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.532+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db4a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.532+0000 7f423ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.532+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db4a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50882/0 (socket says 172.21.15.7:50882) 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.532+0000 7f423f7fe700 1 -- 172.21.15.7:0/875631144 learned_addr learned my addr 172.21.15.7:0/875631144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.533+0000 7f423f7fe700 1 -- 172.21.15.7:0/875631144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 msgr2=0x56326db51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.533+0000 7f423f7fe700 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.534+0000 7f423f7fe700 1 -- 172.21.15.7:0/875631144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 msgr2=0x56326db4a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.534+0000 7f423f7fe700 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db4a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.534+0000 7f423f7fe700 1 -- 172.21.15.7:0/875631144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4240100150 con 0x56326db37f20 2024-01-11T18:02:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.534+0000 7f423ffff700 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:02:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.535+0000 7f423f7fe700 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db4a3a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f42300a0780 tx=0x7f42300a0940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec1cce94fe17bab server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.535+0000 7f421effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:02:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.536+0000 7f421effd700 1 -- 172.21.15.7:0/875631144 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42300a03e0 con 0x56326db37f20 2024-01-11T18:02:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.536+0000 7f421effd700 1 -- 172.21.15.7:0/875631144 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f42300242e0 con 0x56326db37f20 2024-01-11T18:02:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.536+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56326db2ad60 con 0x56326db37f20 2024-01-11T18:02:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.537+0000 7f421effd700 1 -- 172.21.15.7:0/875631144 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42300a1180 con 0x56326db37f20 2024-01-11T18:02:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.537+0000 7f4259fe4b80 10 client.25970 fetch_fsmap learned FSMap version 29 2024-01-11T18:02:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.537+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4228009be0 con 0x56326db37f20 2024-01-11T18:02:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.538+0000 7f421effd700 1 -- 172.21.15.7:0/875631144 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42300b7030 con 0x56326db37f20 2024-01-11T18:02:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.538+0000 7f421effd700 1 -- 172.21.15.7:0/875631144 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4230019020 con 0x56326db37f20 2024-01-11T18:02:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4259fe4b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:02:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4259fe4b80 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4259fe4b80 20 client.25970 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326db58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4259fe4b80 4 client.25970 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:02:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56326da84b60 con 0x56326db56430 2024-01-11T18:02:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.540+0000 7f4237fff700 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326db58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.541+0000 7f4237fff700 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326db58850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbdd544d1025728 server_cookie=804441070103b023 in_seq=0 out_seq=0 2024-01-11T18:02:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.541+0000 7f421effd700 10 client.25970 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.543+0000 7f421effd700 1 -- 172.21.15.7:0/875631144 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56326da84b60 con 0x56326db56430 2024-01-11T18:02:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.543+0000 7f421effd700 10 client.25970 handle_command_reply: tid=0 2024-01-11T18:02:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.543+0000 7f4259fe4b80 1 client.25970 shutdown 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 msgr2=0x56326db4a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db4a3a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f42300a0780 tx=0x7f42300a0940 comp rx=0 tx=0).stop 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 shutdown_connections 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326db58850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f421effd700 0 client.25970 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db4a3a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/875631144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db4a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.544+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 >> 172.21.15.7:0/875631144 conn(0x56326daad740 msgr2=0x56326db327e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.545+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 shutdown_connections 2024-01-11T18:02:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.545+0000 7f4259fe4b80 1 -- 172.21.15.7:0/875631144 wait complete. 2024-01-11T18:02:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.546+0000 7f4259fe4b80 20 client.25970 trim_cache size 0 max 16384 2024-01-11T18:02:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.581+0000 7f4259fe4b80 1 Processor -- start 2024-01-11T18:02:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.581+0000 7f4259fe4b80 1 -- start start 2024-01-11T18:02:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326d902640 con 0x56326db38c10 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326db39aa0 con 0x56326db37f20 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326db52e90 con 0x56326db4f910 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.582+0000 7f4237fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50884/0 (socket says 172.21.15.7:50884) 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423f7fe700 1 -- 172.21.15.7:0/804161748 learned_addr learned my addr 172.21.15.7:0/804161748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423ffff700 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423ffff700 1 -- 172.21.15.7:0/804161748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 msgr2=0x56326db38ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423ffff700 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423ffff700 1 -- 172.21.15.7:0/804161748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 msgr2=0x56326db5d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423ffff700 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db5d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.583+0000 7f423ffff700 1 -- 172.21.15.7:0/804161748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56326d8acb00 con 0x56326db4f910 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.584+0000 7f423ffff700 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db666d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f423800c640 tx=0x7f4238013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5227a2460b1d51 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.584+0000 7f42357fa700 1 -- 172.21.15.7:0/804161748 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238004830 con 0x56326db4f910 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.584+0000 7f42357fa700 1 -- 172.21.15.7:0/804161748 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4238015070 con 0x56326db4f910 2024-01-11T18:02:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.584+0000 7f42357fa700 1 -- 172.21.15.7:0/804161748 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423801f690 con 0x56326db4f910 2024-01-11T18:02:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.585+0000 7f4259fe4b80 1 -- 172.21.15.7:0/804161748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 msgr2=0x56326db666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.585+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db666d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f423800c640 tx=0x7f4238013000 comp rx=0 tx=0).stop 2024-01-11T18:02:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.589+0000 7f4259fe4b80 1 -- 172.21.15.7:0/804161748 shutdown_connections 2024-01-11T18:02:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.589+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326db666d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.589+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db37f20 0x56326db38ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.589+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/804161748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db38c10 0x56326db5d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.589+0000 7f4259fe4b80 1 -- 172.21.15.7:0/804161748 >> 172.21.15.7:0/804161748 conn(0x56326daad740 msgr2=0x56326db49920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.590+0000 7f4259fe4b80 1 -- 172.21.15.7:0/804161748 shutdown_connections 2024-01-11T18:02:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.590+0000 7f4259fe4b80 1 -- 172.21.15.7:0/804161748 wait complete. 2024-01-11T18:02:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.590+0000 7f4259fe4b80 1 -- start start 2024-01-11T18:02:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 Processor -- start 2024-01-11T18:02:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db37f20 0x56326db65780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 0x56326db65cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326dbab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326db664a0 con 0x56326db37f20 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326db66610 con 0x56326db38c10 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56326dbab920 con 0x56326db4f910 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.591+0000 7f4259fe4b80 10 client.0 _ll_register_callbacks cb 0x56326da2c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.592+0000 7f4237fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 0x56326db65cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.592+0000 7f423ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326dbab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.592+0000 7f4237fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 0x56326db65cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50886/0 (socket says 172.21.15.7:50886) 2024-01-11T18:02:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.592+0000 7f4237fff700 1 -- 172.21.15.7:0/870211619 learned_addr learned my addr 172.21.15.7:0/870211619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f423f7fe700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db37f20 0x56326db65780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f4237fff700 1 -- 172.21.15.7:0/870211619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 msgr2=0x56326dbab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f4237fff700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326dbab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f4237fff700 1 -- 172.21.15.7:0/870211619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db37f20 msgr2=0x56326db65780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f4237fff700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db37f20 0x56326db65780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f4237fff700 1 -- 172.21.15.7:0/870211619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4238011440 con 0x56326db38c10 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.593+0000 7f423f7fe700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db37f20 0x56326db65780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.594+0000 7f4237fff700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 0x56326db65cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f422800e980 tx=0x7f4228009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d480071370a47099 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.594+0000 7f421effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.594+0000 7f421effd700 1 -- 172.21.15.7:0/870211619 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228004040 con 0x56326db38c10 2024-01-11T18:02:23.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.594+0000 7f421effd700 1 -- 172.21.15.7:0/870211619 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4228025070 con 0x56326db38c10 2024-01-11T18:02:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.594+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f422800ee50 con 0x56326db38c10 2024-01-11T18:02:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.595+0000 7f421effd700 1 -- 172.21.15.7:0/870211619 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42280041a0 con 0x56326db38c10 2024-01-11T18:02:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.595+0000 7f4259fe4b80 10 client.25976 fetch_fsmap learned FSMap version 29 2024-01-11T18:02:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.595+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56326da4d1d0 con 0x56326db38c10 2024-01-11T18:02:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.595+0000 7f421effd700 1 -- 172.21.15.7:0/870211619 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422800ee50 con 0x56326db38c10 2024-01-11T18:02:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f421effd700 1 -- 172.21.15.7:0/870211619 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f422800e350 con 0x56326db38c10 2024-01-11T18:02:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f4259fe4b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:02:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f4259fe4b80 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f4259fe4b80 20 client.25976 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326dbaf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f4259fe4b80 4 client.25976 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:02:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.596+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56326da84880 con 0x56326db56430 2024-01-11T18:02:23.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.597+0000 7f423f7fe700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326dbaf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.598+0000 7f423f7fe700 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326dbaf3d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98ed6d1fb3afe5a server_cookie=64faef76191449ef in_seq=0 out_seq=0 2024-01-11T18:02:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:23.598+0000 7f421effd700 10 client.25976 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 235.0 s, 788.6 tps, lat 131.201 ms stddev 193.229 2024-01-11T18:02:25.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:25 smithi007 ceph-mon[33361]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.3 MiB/s wr, 604 op/s 2024-01-11T18:02:25.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:25 smithi016 ceph-mon[43195]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.3 MiB/s wr, 604 op/s 2024-01-11T18:02:26.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:25 smithi038 ceph-mon[43582]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.3 MiB/s wr, 604 op/s 2024-01-11T18:02:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.527+0000 7f421effd700 1 -- 172.21.15.7:0/870211619 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56326da84880 con 0x56326db56430 2024-01-11T18:02:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.527+0000 7f421effd700 10 client.25976 handle_command_reply: tid=0 2024-01-11T18:02:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.527+0000 7f4259fe4b80 1 client.25976 shutdown 2024-01-11T18:02:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.527+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 msgr2=0x56326db65cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 0x56326db65cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f422800e980 tx=0x7f4228009320 comp rx=0 tx=0).stop 2024-01-11T18:02:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 shutdown_connections 2024-01-11T18:02:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x56326db56430 0x56326dbaf3d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56326db4f910 0x56326dbab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56326db37f20 0x56326db65780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f421effd700 0 client.25976 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 --2- 172.21.15.7:0/870211619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56326db38c10 0x56326db65cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 >> 172.21.15.7:0/870211619 conn(0x56326daad740 msgr2=0x56326db5d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 shutdown_connections 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.528+0000 7f4259fe4b80 1 -- 172.21.15.7:0/870211619 wait complete. 2024-01-11T18:02:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f4259fe4b80 20 client.25976 trim_cache size 0 max 16384 2024-01-11T18:02:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:02:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:02:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:02:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f422c0a2010 msgr2=0x7f422c0a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f422c0a2010 0x7f422c0a44d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4238005fd0 tx=0x7f423800c700 comp rx=0 tx=0).stop 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 msgr2=0x7f4240197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240197490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4230007c10 tx=0x7f4230004d50 comp rx=0 tx=0).stop 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 shutdown_connections 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f422c0a2010 0x7f422c0a44d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4240102020 0x7f4240197490 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4240107280 0x7f424019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 --2- 172.21.15.7:0/2224794229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4240104950 0x7f42401979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.529+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 >> 172.21.15.7:0/2224794229 conn(0x7f42400fba30 msgr2=0x7f4240102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.530+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 shutdown_connections 2024-01-11T18:02:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:26.530+0000 7f42461ff700 1 -- 172.21.15.7:0/2224794229 wait complete. 2024-01-11T18:02:26.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:26 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:02:26.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:26 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:02:27.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:26 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:02:27.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:27 smithi016 ceph-mon[43195]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-01-11T18:02:28.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:27 smithi038 ceph-mon[43582]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-01-11T18:02:28.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:27 smithi007 ceph-mon[33361]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-01-11T18:02:28.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:02:28.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:02:28.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:02:29.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:02:29.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:02:29.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:02:29.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:02:29.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:02:29.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:02:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 240.0 s, 664.2 tps, lat 147.685 ms stddev 190.740 2024-01-11T18:02:29.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:29 smithi016 ceph-mon[43195]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-01-11T18:02:30.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:29 smithi038 ceph-mon[43582]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-01-11T18:02:30.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:29 smithi007 ceph-mon[33361]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-01-11T18:02:30.493 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h[79869]: *** Caught signal (Segmentation fault) ** 2024-01-11T18:02:30.493 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h[79869]: in thread 7f785ee52700 thread_name:MR_Finisher 2024-01-11T18:02:32.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:31 smithi016 ceph-mon[43195]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 627 op/s 2024-01-11T18:02:32.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:31 smithi038 ceph-mon[43582]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 627 op/s 2024-01-11T18:02:32.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:31 smithi007 ceph-mon[33361]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 627 op/s 2024-01-11T18:02:33.203 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:32 smithi016 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-01-11T18:02:33.494 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:33 smithi016 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-01-11T18:02:34.118 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 245.0 s, 858.2 tps, lat 119.971 ms stddev 159.136 2024-01-11T18:02:34.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:33 smithi016 ceph-mon[43195]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.5 MiB/s wr, 623 op/s 2024-01-11T18:02:34.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:33 smithi038 ceph-mon[43582]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.5 MiB/s wr, 623 op/s 2024-01-11T18:02:34.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:33 smithi007 ceph-mon[33361]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.5 MiB/s wr, 623 op/s 2024-01-11T18:02:35.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:35 smithi007 ceph-mon[33361]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 605 op/s 2024-01-11T18:02:35.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:35 smithi016 ceph-mon[43195]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 605 op/s 2024-01-11T18:02:36.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:35 smithi038 ceph-mon[43582]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 605 op/s 2024-01-11T18:02:38.068 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:37 smithi007 ceph-mon[33361]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 655 op/s 2024-01-11T18:02:38.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:37 smithi016 ceph-mon[43195]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 655 op/s 2024-01-11T18:02:38.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:37 smithi038 ceph-mon[43582]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 655 op/s 2024-01-11T18:02:39.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 250.0 s, 619.4 tps, lat 156.011 ms stddev 184.912 2024-01-11T18:02:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/1242011497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4100770 msgr2=0x7fa6e4100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/1242011497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4100770 0x7fa6e4100bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009990 tx=0x7fa6d4009ca0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/1242011497 shutdown_connections 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/1242011497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e40ffe10 0x7fa6e4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/1242011497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e4101130 0x7fa6e41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/1242011497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4100770 0x7fa6e4100bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.773+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/1242011497 >> 172.21.15.7:0/1242011497 conn(0x7fa6e40fba30 msgr2=0x7fa6e40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.774+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/1242011497 shutdown_connections 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.774+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/1242011497 wait complete. 2024-01-11T18:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.775+0000 7fa6eb9ff700 1 Processor -- start 2024-01-11T18:02:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.775+0000 7fa6eb9ff700 1 -- start start 2024-01-11T18:02:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6eb9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e40ffe10 0x7fa6e4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6eb9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 0x7fa6e4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6eb9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4101130 0x7fa6e41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6eb9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41a1200 con 0x7fa6e40ffe10 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6eb9ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41a1340 con 0x7fa6e4101130 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6eb9ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41a14d0 con 0x7fa6e4100770 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6e8f9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 0x7fa6e4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6e8f9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 0x7fa6e4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45468/0 (socket says 172.21.15.7:45468) 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.776+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/105145109 learned_addr learned my addr 172.21.15.7:0/105145109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e9f9c700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4101130 0x7fa6e41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e979b700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e40ffe10 0x7fa6e4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/105145109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4101130 msgr2=0x7fa6e41a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4101130 0x7fa6e41a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e40ffe10 msgr2=0x7fa6e4199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e40ffe10 0x7fa6e4199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/105145109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40095f0 con 0x7fa6e4100770 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 0x7fa6e4199bc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40098e0 tx=0x7fa6d400f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6da7fc700 1 -- 172.21.15.7:0/105145109 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d401d070 con 0x7fa6e4100770 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6da7fc700 1 -- 172.21.15.7:0/105145109 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6d4005510 con 0x7fa6e4100770 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.777+0000 7fa6da7fc700 1 -- 172.21.15.7:0/105145109 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4004c40 con 0x7fa6e4100770 2024-01-11T18:02:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.778+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e41a1750 con 0x7fa6e4100770 2024-01-11T18:02:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.778+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e41a1b20 con 0x7fa6e4100770 2024-01-11T18:02:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.784+0000 7fa6da7fc700 1 -- 172.21.15.7:0/105145109 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa6d400fbd0 con 0x7fa6e4100770 2024-01-11T18:02:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.787+0000 7fa6da7fc700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa6d00a1fd0 0x7fa6d00a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.787+0000 7fa6da7fc700 1 -- 172.21.15.7:0/105145109 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa6d40cec90 con 0x7fa6e4100770 2024-01-11T18:02:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.787+0000 7fa6ff75eb80 1 Processor -- start 2024-01-11T18:02:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.788+0000 7fa6e979b700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa6d00a1fd0 0x7fa6d00a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.788+0000 7fa6ff75eb80 1 -- start start 2024-01-11T18:02:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.789+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b3103b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.789+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b31048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.789+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b310b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.789+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b2ff2030 con 0x55d8b3104420 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.789+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b3055ac0 con 0x55d8b3103730 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.789+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b30533e0 con 0x55d8b3104de0 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b31048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b31048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47898/0 (socket says 172.21.15.7:47898) 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/1589171922 learned_addr learned my addr 172.21.15.7:0/1589171922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b3103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e9f9c700 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b310b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/1589171922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 msgr2=0x55d8b310b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b310b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/1589171922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 msgr2=0x55d8b3103b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b3103b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.790+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/1589171922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8b2ec3350 con 0x55d8b3104420 2024-01-11T18:02:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.791+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b31048a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4098160 tx=0x7fa6d4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedc9b3def385a7a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.791+0000 7fa6e979b700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa6d00a1fd0 0x7fa6d00a4490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0005950 tx=0x7fa6e00138a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.791+0000 7fa6cf7fe700 1 -- 172.21.15.7:0/1589171922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4005680 con 0x55d8b3104420 2024-01-11T18:02:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.791+0000 7fa6cf7fe700 1 -- 172.21.15.7:0/1589171922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6d409bdf0 con 0x55d8b3104420 2024-01-11T18:02:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.791+0000 7fa6cf7fe700 1 -- 172.21.15.7:0/1589171922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d409abe0 con 0x55d8b3104420 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1589171922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 msgr2=0x55d8b31048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b31048a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4098160 tx=0x7fa6d4009ca0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1589171922 shutdown_connections 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b310b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b3103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1589171922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b31048a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.792+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1589171922 >> 172.21.15.7:0/1589171922 conn(0x55d8b3070740 msgr2=0x55d8b30f5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.793+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1589171922 shutdown_connections 2024-01-11T18:02:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.793+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1589171922 wait complete. 2024-01-11T18:02:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.794+0000 7fa6ff75eb80 1 -- start start 2024-01-11T18:02:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.794+0000 7fa6ff75eb80 1 Processor -- start 2024-01-11T18:02:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.795+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 0x55d8b310cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.795+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.795+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3104de0 0x55d8b3114350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.795+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b304ecd0 con 0x55d8b3104420 2024-01-11T18:02:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.795+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b2fbd340 con 0x55d8b3104de0 2024-01-11T18:02:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.795+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b2fbcfe0 con 0x55d8b3103730 2024-01-11T18:02:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.796+0000 7fa6ff75eb80 10 client.0 _ll_register_callbacks cb 0x55d8b2fef6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.796+0000 7fa6e979b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 0x55d8b310cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.796+0000 7fa6e979b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 0x55d8b310cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45474/0 (socket says 172.21.15.7:45474) 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.796+0000 7fa6e979b700 1 -- 172.21.15.7:0/1267137740 learned_addr learned my addr 172.21.15.7:0/1267137740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.796+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.796+0000 7fa6e9f9c700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3104de0 0x55d8b3114350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6e979b700 1 -- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3104de0 msgr2=0x55d8b3114350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3104de0 0x55d8b3114350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6e979b700 1 -- 172.21.15.7:0/1267137740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 msgr2=0x55d8b310d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6e979b700 1 -- 172.21.15.7:0/1267137740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e41a1b20 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 0x55d8b310cdd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0014fd0 tx=0x7fa6e00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecae50159d8b76b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6ccff9700 1 -- 172.21.15.7:0/1267137740 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0028670 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6ccff9700 1 -- 172.21.15.7:0/1267137740 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa6e00287d0 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.797+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8b30edd60 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.798+0000 7fa6ccff9700 1 -- 172.21.15.7:0/1267137740 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e0013280 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.798+0000 7fa6ff75eb80 10 client.26014 fetch_fsmap learned FSMap version 29 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.798+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d8b2e6fb00 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.798+0000 7fa6ccff9700 1 -- 172.21.15.7:0/1267137740 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8b30edd60 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.798+0000 7fa6ccff9700 1 -- 172.21.15.7:0/1267137740 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa6e0003680 con 0x55d8b3103730 2024-01-11T18:02:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.799+0000 7fa6ff75eb80 10 client.26014 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:02:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.799+0000 7fa6ff75eb80 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.799+0000 7fa6ff75eb80 20 client.26014 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.799+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3118cb0 0x55d8b311b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.799+0000 7fa6ff75eb80 4 client.26014 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:02:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.799+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8b3047b60 con 0x55d8b3118cb0 2024-01-11T18:02:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.800+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3118cb0 0x55d8b311b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.806+0000 7fa6ccff9700 10 client.26014 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.806+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3118cb0 0x55d8b311b0d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719e52e7dc326bda server_cookie=dd0a8dd74ba5ba04 in_seq=0 out_seq=0 2024-01-11T18:02:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.807+0000 7fa6ccff9700 1 -- 172.21.15.7:0/1267137740 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8b3047b60 con 0x55d8b3118cb0 2024-01-11T18:02:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.807+0000 7fa6ccff9700 10 client.26014 handle_command_reply: tid=0 2024-01-11T18:02:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.808+0000 7fa6ff75eb80 1 client.26014 shutdown 2024-01-11T18:02:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.809+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 msgr2=0x55d8b310cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.809+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 0x55d8b310cdd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0014fd0 tx=0x7fa6e00047c0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 shutdown_connections 2024-01-11T18:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3118cb0 0x55d8b311b0d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3103730 0x55d8b310cdd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3104de0 0x55d8b3114350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1267137740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 >> 172.21.15.7:0/1267137740 conn(0x55d8b3070740 msgr2=0x55d8b30f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ccff9700 0 client.26014 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 shutdown_connections 2024-01-11T18:02:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.810+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1267137740 wait complete. 2024-01-11T18:02:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.811+0000 7fa6ff75eb80 20 client.26014 trim_cache size 0 max 16384 2024-01-11T18:02:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.857+0000 7fa6ff75eb80 1 Processor -- start 2024-01-11T18:02:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.857+0000 7fa6ff75eb80 1 -- start start 2024-01-11T18:02:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.858+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b311f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.858+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104420 0x55d8b3114310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.858+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104de0 0x55d8b3107d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.858+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b2ec5640 con 0x55d8b3104de0 2024-01-11T18:02:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.858+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b30f7010 con 0x55d8b3103730 2024-01-11T18:02:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.858+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b30f2560 con 0x55d8b3104420 2024-01-11T18:02:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.859+0000 7fa6e8f9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104420 0x55d8b3114310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.859+0000 7fa6e979b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b311f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.860+0000 7fa6e979b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b311f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37720/0 (socket says 172.21.15.7:37720) 2024-01-11T18:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.860+0000 7fa6e979b700 1 -- 172.21.15.7:0/1690017238 learned_addr learned my addr 172.21.15.7:0/1690017238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.860+0000 7fa6e9f9c700 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104de0 0x55d8b3107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.861+0000 7fa6e979b700 1 -- 172.21.15.7:0/1690017238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104420 msgr2=0x55d8b3114310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.861+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104420 0x55d8b3114310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.861+0000 7fa6e979b700 1 -- 172.21.15.7:0/1690017238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104de0 msgr2=0x55d8b3107d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.861+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104de0 0x55d8b3107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.861+0000 7fa6e979b700 1 -- 172.21.15.7:0/1690017238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8b30101d0 con 0x55d8b3103730 2024-01-11T18:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.862+0000 7fa6e979b700 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b311f9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0003fd0 tx=0x7fa6e000bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcf10370f271a741 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.862+0000 7fa6ceffd700 1 -- 172.21.15.7:0/1690017238 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e001ec30 con 0x55d8b3103730 2024-01-11T18:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.862+0000 7fa6ceffd700 1 -- 172.21.15.7:0/1690017238 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e0019070 con 0x55d8b3103730 2024-01-11T18:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.862+0000 7fa6ceffd700 1 -- 172.21.15.7:0/1690017238 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00286d0 con 0x55d8b3103730 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.864+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1690017238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 msgr2=0x55d8b311f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.864+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b311f9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0003fd0 tx=0x7fa6e000bf20 comp rx=0 tx=0).stop 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.865+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1690017238 shutdown_connections 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.865+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104de0 0x55d8b3107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.865+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b311f9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.865+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/1690017238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104420 0x55d8b3114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.865+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1690017238 >> 172.21.15.7:0/1690017238 conn(0x55d8b3070740 msgr2=0x55d8b310c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.867+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1690017238 shutdown_connections 2024-01-11T18:02:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.867+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/1690017238 wait complete. 2024-01-11T18:02:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.868+0000 7fa6ff75eb80 1 -- start start 2024-01-11T18:02:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 Processor -- start 2024-01-11T18:02:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b310f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b3112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b3113100 con 0x55d8b3104420 2024-01-11T18:02:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b3113270 con 0x55d8b3103730 2024-01-11T18:02:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b3113430 con 0x55d8b3104de0 2024-01-11T18:02:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.869+0000 7fa6ff75eb80 10 client.0 _ll_register_callbacks cb 0x55d8b2fef6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.870+0000 7fa6e9f9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b3112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.870+0000 7fa6e9f9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b3112bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45480/0 (socket says 172.21.15.7:45480) 2024-01-11T18:02:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.870+0000 7fa6e9f9c700 1 -- 172.21.15.7:0/167711886 learned_addr learned my addr 172.21.15.7:0/167711886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.870+0000 7fa6e979b700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b310f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.871+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/167711886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 msgr2=0x55d8b3112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b3112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 msgr2=0x55d8b310f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b310f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e8f9a700 1 -- 172.21.15.7:0/167711886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8b2e6fb00 con 0x55d8b3104420 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e979b700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b310f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.872+0000 7fa6e8f9a700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310fd00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40a3bb0 tx=0x7fa6d40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac2ff60a51079b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.873+0000 7fa6ccff9700 1 -- 172.21.15.7:0/167711886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40a39f0 con 0x55d8b3104420 2024-01-11T18:02:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.873+0000 7fa6ccff9700 1 -- 172.21.15.7:0/167711886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa6d409f840 con 0x55d8b3104420 2024-01-11T18:02:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.873+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6d409bdf0 con 0x55d8b3104420 2024-01-11T18:02:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ccff9700 1 -- 172.21.15.7:0/167711886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6d409b6f0 con 0x55d8b3104420 2024-01-11T18:02:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ff75eb80 10 client.15894 fetch_fsmap learned FSMap version 29 2024-01-11T18:02:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d8b2ec7c70 con 0x55d8b3104420 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ccff9700 1 -- 172.21.15.7:0/167711886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6d409bdf0 con 0x55d8b3104420 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ccff9700 1 -- 172.21.15.7:0/167711886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa6d40974e0 con 0x55d8b3104420 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ff75eb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 29 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ff75eb80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ff75eb80 20 client.15894 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.874+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3180430 0x55d8b3182820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.875+0000 7fa6ff75eb80 4 client.15894 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:02:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.875+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8b3047880 con 0x55d8b3180430 2024-01-11T18:02:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.876+0000 7fa6e979b700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3180430 0x55d8b3182820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.879+0000 7fa6e979b700 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3180430 0x55d8b3182820 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff74c628992c1b06 server_cookie=f9bc9fbf3648fe29 in_seq=0 out_seq=0 2024-01-11T18:02:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.879+0000 7fa6ccff9700 10 client.15894 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.978+0000 7fa6ccff9700 1 -- 172.21.15.7:0/167711886 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8b3047880 con 0x55d8b3180430 2024-01-11T18:02:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.978+0000 7fa6ccff9700 10 client.15894 handle_command_reply: tid=0 2024-01-11T18:02:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.978+0000 7fa6ff75eb80 1 client.15894 shutdown 2024-01-11T18:02:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.979+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 msgr2=0x55d8b310fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.979+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310fd00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40a3bb0 tx=0x7fa6d40057a0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 shutdown_connections 2024-01-11T18:02:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d8b3180430 0x55d8b3182820 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8b3104de0 0x55d8b3112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ccff9700 0 client.15894 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8b3103730 0x55d8b310f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ff75eb80 1 --2- 172.21.15.7:0/167711886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d8b3104420 0x55d8b310fd00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.980+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 >> 172.21.15.7:0/167711886 conn(0x55d8b3070740 msgr2=0x55d8b311f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.981+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 shutdown_connections 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.981+0000 7fa6ff75eb80 1 -- 172.21.15.7:0/167711886 wait complete. 2024-01-11T18:02:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.981+0000 7fa6ff75eb80 20 client.15894 trim_cache size 0 max 16384 2024-01-11T18:02:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:02:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:02:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:02:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:02:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa6d00a1fd0 msgr2=0x7fa6d00a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa6d00a1fd0 0x7fa6d00a4490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0005950 tx=0x7fa6e00138a0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 msgr2=0x7fa6e4199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 0x7fa6e4199bc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40098e0 tx=0x7fa6d400f720 comp rx=0 tx=0).stop 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 shutdown_connections 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa6d00a1fd0 0x7fa6d00a4490 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6e4101130 0x7fa6e41a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e40ffe10 0x7fa6e4199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 --2- 172.21.15.7:0/105145109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e4100770 0x7fa6e4199bc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 >> 172.21.15.7:0/105145109 conn(0x7fa6e40fba30 msgr2=0x7fa6e40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 shutdown_connections 2024-01-11T18:02:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:39.985+0000 7fa6eb9ff700 1 -- 172.21.15.7:0/105145109 wait complete. 2024-01-11T18:02:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:39 smithi016 ceph-mon[43195]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.0 MiB/s wr, 569 op/s 2024-01-11T18:02:40.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:39 smithi038 ceph-mon[43582]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.0 MiB/s wr, 569 op/s 2024-01-11T18:02:40.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:39 smithi007 ceph-mon[33361]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.0 MiB/s wr, 569 op/s 2024-01-11T18:02:41.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:40 smithi016 ceph-mon[43195]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.6 MiB/s wr, 596 op/s 2024-01-11T18:02:41.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:40 smithi038 ceph-mon[43582]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.6 MiB/s wr, 596 op/s 2024-01-11T18:02:41.319 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:40 smithi007 ceph-mon[33361]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.6 MiB/s wr, 596 op/s 2024-01-11T18:02:43.566 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h.service: Service RestartSec=10s expired, scheduling restart. 2024-01-11T18:02:43.566 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-01-11T18:02:43.566 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 systemd[1]: Stopped Ceph mds.h for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T18:02:43.566 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 systemd[1]: Starting Ceph mds.h for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:02:44.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 255.0 s, 776.0 tps, lat 134.104 ms stddev 168.404 2024-01-11T18:02:44.245 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 bash[82858]: 07dbd4141823f5b33bf450bae81449ee08dbfcab246832bf7b9120c77a4dfa83 2024-01-11T18:02:44.245 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 systemd[1]: Started Ceph mds.h for a76d0c8a-b0a9-11ee-95ab-87774f69a715. 2024-01-11T18:02:44.245 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 ceph-mds[82895]: -- 172.21.15.16:0/1194929837 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55921b6e9520 con 0x55921b6bec00 2024-01-11T18:02:44.245 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:02:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h[82872]: starting mds.h at 2024-01-11T18:02:44.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:43 smithi016 ceph-mon[43195]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7.3 MiB/s wr, 546 op/s 2024-01-11T18:02:44.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:43 smithi016 ceph-mon[43195]: daemon mds.h restarted 2024-01-11T18:02:44.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:43 smithi038 ceph-mon[43582]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7.3 MiB/s wr, 546 op/s 2024-01-11T18:02:44.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:43 smithi038 ceph-mon[43582]: daemon mds.h restarted 2024-01-11T18:02:44.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:43 smithi007 ceph-mon[33361]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7.3 MiB/s wr, 546 op/s 2024-01-11T18:02:44.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:43 smithi007 ceph-mon[33361]: daemon mds.h restarted 2024-01-11T18:02:45.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:44 smithi016 ceph-mon[43195]: osdmap e89: 12 total, 12 up, 12 in 2024-01-11T18:02:45.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:44 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6836/3100471640,v1:172.21.15.16:6837/3100471640] up:boot 2024-01-11T18:02:45.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:44 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-01-11T18:02:45.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-11T18:02:45.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:44 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:02:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:44 smithi038 ceph-mon[43582]: osdmap e89: 12 total, 12 up, 12 in 2024-01-11T18:02:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:44 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6836/3100471640,v1:172.21.15.16:6837/3100471640] up:boot 2024-01-11T18:02:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:44 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-01-11T18:02:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-11T18:02:45.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:44 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:02:45.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:44 smithi007 ceph-mon[33361]: osdmap e89: 12 total, 12 up, 12 in 2024-01-11T18:02:45.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:44 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6836/3100471640,v1:172.21.15.16:6837/3100471640] up:boot 2024-01-11T18:02:45.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:44 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-01-11T18:02:45.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-11T18:02:45.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:44 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:02:46.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:45 smithi016 ceph-mon[43195]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 MiB/s wr, 555 op/s 2024-01-11T18:02:46.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:45 smithi038 ceph-mon[43582]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 MiB/s wr, 555 op/s 2024-01-11T18:02:46.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:45 smithi007 ceph-mon[33361]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.8 MiB/s wr, 555 op/s 2024-01-11T18:02:48.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:47 smithi016 ceph-mon[43195]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.4 MiB/s wr, 526 op/s 2024-01-11T18:02:48.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:47 smithi038 ceph-mon[43582]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.4 MiB/s wr, 526 op/s 2024-01-11T18:02:48.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:47 smithi007 ceph-mon[33361]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.4 MiB/s wr, 526 op/s 2024-01-11T18:02:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 260.0 s, 767.8 tps, lat 127.749 ms stddev 157.291 2024-01-11T18:02:49.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:48 smithi016 ceph-mon[43195]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.6 MiB/s wr, 519 op/s 2024-01-11T18:02:49.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:48 smithi038 ceph-mon[43582]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.6 MiB/s wr, 519 op/s 2024-01-11T18:02:49.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:48 smithi007 ceph-mon[33361]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.6 MiB/s wr, 519 op/s 2024-01-11T18:02:52.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:51 smithi016 ceph-mon[43195]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-01-11T18:02:52.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:51 smithi038 ceph-mon[43582]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-01-11T18:02:52.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:51 smithi007 ceph-mon[33361]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.233+0000 7f0949fff700 1 -- 172.21.15.7:0/471116917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ffc90 msgr2=0x7f0944106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.233+0000 7f0949fff700 1 --2- 172.21.15.7:0/471116917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ffc90 0x7f0944106c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0938009990 tx=0x7f0938009ca0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 -- 172.21.15.7:0/471116917 shutdown_connections 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 --2- 172.21.15.7:0/471116917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0944107260 0x7f0944109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 --2- 172.21.15.7:0/471116917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09440ff330 0x7f09440ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 --2- 172.21.15.7:0/471116917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ffc90 0x7f0944106c90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 -- 172.21.15.7:0/471116917 >> 172.21.15.7:0/471116917 conn(0x7f09440fad60 msgr2=0x7f09440fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 -- 172.21.15.7:0/471116917 shutdown_connections 2024-01-11T18:02:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.234+0000 7f0949fff700 1 -- 172.21.15.7:0/471116917 wait complete. 2024-01-11T18:02:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 Processor -- start 2024-01-11T18:02:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 -- start start 2024-01-11T18:02:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 0x7f094419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440ffc90 0x7f094419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0944107260 0x7f09441a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441a5670 con 0x7f09440ffc90 2024-01-11T18:02:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441a57b0 con 0x7f0944107260 2024-01-11T18:02:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.236+0000 7f0949fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441a5940 con 0x7f09440ff330 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 0x7f094419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 0x7f094419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55984/0 (socket says 172.21.15.7:55984) 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 -- 172.21.15.7:0/1794138807 learned_addr learned my addr 172.21.15.7:0/1794138807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f0943fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0944107260 0x7f09441a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f0942ffd700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440ffc90 0x7f094419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 -- 172.21.15.7:0/1794138807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0944107260 msgr2=0x7f09441a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0944107260 0x7f09441a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 -- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440ffc90 msgr2=0x7f094419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440ffc90 0x7f094419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f09437fe700 1 -- 172.21.15.7:0/1794138807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380095f0 con 0x7f09440ff330 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.237+0000 7f0942ffd700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440ffc90 0x7f094419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.238+0000 7f09437fe700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 0x7f094419daf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f093400d950 tx=0x7f093400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.239+0000 7f0940ff9700 1 -- 172.21.15.7:0/1794138807 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934009940 con 0x7f09440ff330 2024-01-11T18:02:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.239+0000 7f0940ff9700 1 -- 172.21.15.7:0/1794138807 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0934010460 con 0x7f09440ff330 2024-01-11T18:02:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.239+0000 7f0940ff9700 1 -- 172.21.15.7:0/1794138807 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093400f620 con 0x7f09440ff330 2024-01-11T18:02:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.239+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09441a5c20 con 0x7f09440ff330 2024-01-11T18:02:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.239+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09441a60e0 con 0x7f09440ff330 2024-01-11T18:02:53.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:52 smithi016 ceph-mon[43195]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 MiB/s wr, 548 op/s 2024-01-11T18:02:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.246+0000 7f0940ff9700 1 -- 172.21.15.7:0/1794138807 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f09340108d0 con 0x7f09440ff330 2024-01-11T18:02:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.248+0000 7f0940ff9700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f092c0a20a0 0x7f092c0a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.249+0000 7f0940ff9700 1 -- 172.21.15.7:0/1794138807 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f09340cda30 con 0x7f09440ff330 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.252+0000 7f0942ffd700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f092c0a20a0 0x7f092c0a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.254+0000 7f095dcf9b80 1 Processor -- start 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.255+0000 7f095dcf9b80 1 -- start start 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.255+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8016220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.255+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8016cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.255+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8017710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.256+0000 7f09437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8016220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.258+0000 7f09437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8016220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42912/0 (socket says 172.21.15.7:42912) 2024-01-11T18:02:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f09437fe700 1 -- 172.21.15.7:0/1057437581 learned_addr learned my addr 172.21.15.7:0/1057437581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7efc030 con 0x55e6a8015e00 2024-01-11T18:02:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7f5fac0 con 0x55e6a8017260 2024-01-11T18:02:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7f5d3e0 con 0x55e6a8016760 2024-01-11T18:02:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f0943fff700 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8017710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f09437fe700 1 -- 172.21.15.7:0/1057437581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 msgr2=0x55e6a8016cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f09437fe700 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8016cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f09437fe700 1 -- 172.21.15.7:0/1057437581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 msgr2=0x55e6a8017710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f09437fe700 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8017710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.259+0000 7f09437fe700 1 -- 172.21.15.7:0/1057437581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a7dcd350 con 0x55e6a8015e00 2024-01-11T18:02:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.260+0000 7f0942ffd700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f092c0a20a0 0x7f092c0a4560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f093800b680 tx=0x7f0938005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.260+0000 7f09437fe700 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8016220 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f09340a2260 tx=0x7f093409a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91d86d51e8b054a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.261+0000 7f0929ffb700 1 -- 172.21.15.7:0/1057437581 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093409e550 con 0x55e6a8015e00 2024-01-11T18:02:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.261+0000 7f0929ffb700 1 -- 172.21.15.7:0/1057437581 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09340a2790 con 0x55e6a8015e00 2024-01-11T18:02:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.261+0000 7f0929ffb700 1 -- 172.21.15.7:0/1057437581 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093409a8a0 con 0x55e6a8015e00 2024-01-11T18:02:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 msgr2=0x55e6a8016220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8016220 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f09340a2260 tx=0x7f093409a070 comp rx=0 tx=0).stop 2024-01-11T18:02:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 shutdown_connections 2024-01-11T18:02:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8017710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8016220 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1057437581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8016cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.263+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 >> 172.21.15.7:0/1057437581 conn(0x55e6a7f7a740 msgr2=0x55e6a7fff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.265+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 shutdown_connections 2024-01-11T18:02:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.265+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1057437581 wait complete. 2024-01-11T18:02:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.266+0000 7f095dcf9b80 1 -- start start 2024-01-11T18:02:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.267+0000 7f095dcf9b80 1 Processor -- start 2024-01-11T18:02:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.271+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a801f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.271+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a801fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.271+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8026f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f095dcf9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7f58cd0 con 0x55e6a8015e00 2024-01-11T18:02:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f095dcf9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7ec7340 con 0x55e6a8017260 2024-01-11T18:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f095dcf9b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7ec6fe0 con 0x55e6a8016760 2024-01-11T18:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f095dcf9b80 10 client.0 _ll_register_callbacks cb 0x55e6a7ef96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f0942ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a801fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f0942ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a801fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:56000/0 (socket says 172.21.15.7:56000) 2024-01-11T18:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f0942ffd700 1 -- 172.21.15.7:0/1152314653 learned_addr learned my addr 172.21.15.7:0/1152314653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f09437fe700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a801f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.272+0000 7f0943fff700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8026f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.273+0000 7f09437fe700 1 -- 172.21.15.7:0/1152314653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 msgr2=0x55e6a801fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.273+0000 7f09437fe700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a801fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.273+0000 7f09437fe700 1 -- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 msgr2=0x55e6a8026f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.273+0000 7f09437fe700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8026f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.273+0000 7f09437fe700 1 -- 172.21.15.7:0/1152314653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09441a60e0 con 0x55e6a8015e00 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.273+0000 7f09437fe700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a801f990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f093409a100 tx=0x7f093409a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f87f032ad36f055 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.274+0000 7f0922ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.274+0000 7f0922ffd700 1 -- 172.21.15.7:0/1152314653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093409e5d0 con 0x55e6a8015e00 2024-01-11T18:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.274+0000 7f0922ffd700 1 -- 172.21.15.7:0/1152314653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f09340998f0 con 0x55e6a8015e00 2024-01-11T18:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.274+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6a7ff7d60 con 0x55e6a8015e00 2024-01-11T18:02:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f0922ffd700 1 -- 172.21.15.7:0/1152314653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0934099ab0 con 0x55e6a8015e00 2024-01-11T18:02:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f095dcf9b80 10 client.15909 fetch_fsmap learned FSMap version 31 2024-01-11T18:02:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e6a7d79b00 con 0x55e6a8015e00 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f0922ffd700 1 -- 172.21.15.7:0/1152314653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6a7ff7d60 con 0x55e6a8015e00 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f0922ffd700 1 -- 172.21.15.7:0/1152314653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0934099360 con 0x55e6a8015e00 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f095dcf9b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 31 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.278+0000 7f095dcf9b80 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.279+0000 7f095dcf9b80 20 client.15909 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.279+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a802b2e0 0x55e6a802d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.279+0000 7f095dcf9b80 4 client.15909 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.279+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6a7f51b60 con 0x55e6a802b2e0 2024-01-11T18:02:53.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:52 smithi038 ceph-mon[43582]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 MiB/s wr, 548 op/s 2024-01-11T18:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.280+0000 7f0942ffd700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a802b2e0 0x55e6a802d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.290+0000 7f0942ffd700 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a802b2e0 0x55e6a802d700 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbd37e8984f24ee server_cookie=7ec29eb7bb7bd375 in_seq=0 out_seq=0 2024-01-11T18:02:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.292+0000 7f0922ffd700 10 client.15909 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.298+0000 7f0922ffd700 1 -- 172.21.15.7:0/1152314653 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6a7f51b60 con 0x55e6a802b2e0 2024-01-11T18:02:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.298+0000 7f0922ffd700 10 client.15909 handle_command_reply: tid=0 2024-01-11T18:02:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.298+0000 7f095dcf9b80 1 client.15909 shutdown 2024-01-11T18:02:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 msgr2=0x55e6a801f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a801f990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f093409a100 tx=0x7f093409a4c0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 shutdown_connections 2024-01-11T18:02:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a802b2e0 0x55e6a802d700 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8026f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a801f990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/1152314653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a801fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 >> 172.21.15.7:0/1152314653 conn(0x55e6a7f7a740 msgr2=0x55e6a7ffef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.299+0000 7f0922ffd700 0 client.15909 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.300+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 shutdown_connections 2024-01-11T18:02:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.300+0000 7f095dcf9b80 1 -- 172.21.15.7:0/1152314653 wait complete. 2024-01-11T18:02:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.300+0000 7f095dcf9b80 20 client.15909 trim_cache size 0 max 16384 2024-01-11T18:02:53.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:52 smithi007 ceph-mon[33361]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.6 MiB/s wr, 548 op/s 2024-01-11T18:02:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.354+0000 7f095dcf9b80 1 Processor -- start 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.354+0000 7f095dcf9b80 1 -- start start 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a80275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8026ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a7ffb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f095dcf9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a7dcf640 con 0x55e6a8015e00 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f095dcf9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a801e400 con 0x55e6a8017260 2024-01-11T18:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f095dcf9b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8000be0 con 0x55e6a8016760 2024-01-11T18:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f0942ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8026ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f0942ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8026ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:56014/0 (socket says 172.21.15.7:56014) 2024-01-11T18:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f0942ffd700 1 -- 172.21.15.7:0/298311832 learned_addr learned my addr 172.21.15.7:0/298311832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.355+0000 7f0942ffd700 1 -- 172.21.15.7:0/298311832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 msgr2=0x55e6a7ffb9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.356+0000 7f0942ffd700 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a7ffb9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.356+0000 7f0942ffd700 1 -- 172.21.15.7:0/298311832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 msgr2=0x55e6a80275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.356+0000 7f0942ffd700 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a80275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.356+0000 7f0942ffd700 1 -- 172.21.15.7:0/298311832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a7f1a1d0 con 0x55e6a8016760 2024-01-11T18:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.356+0000 7f0942ffd700 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8026ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0938000c00 tx=0x7f0938021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42018a035e12b82b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.356+0000 7f09297fa700 1 -- 172.21.15.7:0/298311832 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938005690 con 0x55e6a8016760 2024-01-11T18:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.357+0000 7f09297fa700 1 -- 172.21.15.7:0/298311832 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f093802c040 con 0x55e6a8016760 2024-01-11T18:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.357+0000 7f09297fa700 1 -- 172.21.15.7:0/298311832 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938032850 con 0x55e6a8016760 2024-01-11T18:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 -- 172.21.15.7:0/298311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 msgr2=0x55e6a8026ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8026ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0938000c00 tx=0x7f0938021000 comp rx=0 tx=0).stop 2024-01-11T18:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 -- 172.21.15.7:0/298311832 shutdown_connections 2024-01-11T18:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a7ffb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a80275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/298311832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8026ed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.358+0000 7f095dcf9b80 1 -- 172.21.15.7:0/298311832 >> 172.21.15.7:0/298311832 conn(0x55e6a7f7a740 msgr2=0x55e6a7fff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.359+0000 7f095dcf9b80 1 -- 172.21.15.7:0/298311832 shutdown_connections 2024-01-11T18:02:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.359+0000 7f095dcf9b80 1 -- 172.21.15.7:0/298311832 wait complete. 2024-01-11T18:02:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.360+0000 7f095dcf9b80 1 -- start start 2024-01-11T18:02:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.360+0000 7f095dcf9b80 1 Processor -- start 2024-01-11T18:02:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.361+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8024a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.361+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8024f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.361+0000 7f095dcf9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8025f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.363+0000 7f0942ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8024f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.364+0000 7f0943fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8025f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.364+0000 7f0943fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8025f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33408/0 (socket says 172.21.15.7:33408) 2024-01-11T18:02:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.364+0000 7f0943fff700 1 -- 172.21.15.7:0/2956864660 learned_addr learned my addr 172.21.15.7:0/2956864660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:02:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.364+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a80264b0 con 0x55e6a8015e00 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f09437fe700 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8024a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8026620 con 0x55e6a8017260 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a80267e0 con 0x55e6a8016760 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f0943fff700 1 -- 172.21.15.7:0/2956864660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 msgr2=0x55e6a8024f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f0943fff700 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8024f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f0943fff700 1 -- 172.21.15.7:0/2956864660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 msgr2=0x55e6a8024a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f0943fff700 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8024a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.365+0000 7f0943fff700 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0938020040 con 0x55e6a8017260 2024-01-11T18:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.366+0000 7f095dcf9b80 10 client.0 _ll_register_callbacks cb 0x55e6a7ef96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.366+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09340a2790 con 0x55e6a8017260 2024-01-11T18:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.366+0000 7f0943fff700 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8025f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f093c007aa0 tx=0x7f093c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7970b533c74fc1d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.366+0000 7f0922ffd700 10 client.26012 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.367+0000 7f0922ffd700 1 -- 172.21.15.7:0/2956864660 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c00e040 con 0x55e6a8017260 2024-01-11T18:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.368+0000 7f0922ffd700 1 -- 172.21.15.7:0/2956864660 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f093c004050 con 0x55e6a8017260 2024-01-11T18:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.368+0000 7f0922ffd700 1 -- 172.21.15.7:0/2956864660 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f093c004250 con 0x55e6a8017260 2024-01-11T18:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.368+0000 7f095dcf9b80 10 client.26012 fetch_fsmap learned FSMap version 31 2024-01-11T18:02:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.368+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=31}) v3 -- 0x55e6a7dd1c70 con 0x55e6a8017260 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f0922ffd700 1 -- 172.21.15.7:0/2956864660 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f093c0043b0 con 0x55e6a8017260 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f095dcf9b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 31 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f095dcf9b80 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f095dcf9b80 20 client.26012 populate_metadata read hostname 'smithi007' 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a8082510 0x55e6a8084930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f095dcf9b80 4 client.26012 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.369+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e6a7f51880 con 0x55e6a8082510 2024-01-11T18:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.370+0000 7f09437fe700 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a8082510 0x55e6a8084930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.371+0000 7f09437fe700 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a8082510 0x55e6a8084930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb04269e6f22ffe server_cookie=7cb7368acde0ad5a in_seq=0 out_seq=0 2024-01-11T18:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.371+0000 7f0922ffd700 10 client.26012 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f0922ffd700 1 -- 172.21.15.7:0/2956864660 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e6a7f51880 con 0x55e6a8082510 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f0922ffd700 10 client.26012 handle_command_reply: tid=0 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 client.26012 shutdown 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 msgr2=0x55e6a8025f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8025f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f093c007aa0 tx=0x7f093c009510 comp rx=0 tx=0).stop 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 shutdown_connections 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55e6a8082510 0x55e6a8084930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6a8017260 0x55e6a8025f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f0922ffd700 0 client.26012 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e6a8015e00 0x55e6a8024a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 --2- 172.21.15.7:0/2956864660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e6a8016760 0x55e6a8024f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.447+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 >> 172.21.15.7:0/2956864660 conn(0x55e6a7f7a740 msgr2=0x55e6a801d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.448+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 shutdown_connections 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.448+0000 7f095dcf9b80 1 -- 172.21.15.7:0/2956864660 wait complete. 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.448+0000 7f095dcf9b80 20 client.26012 trim_cache size 0 max 16384 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:02:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f092c0a20a0 msgr2=0x7f092c0a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f092c0a20a0 0x7f092c0a4560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f093800b680 tx=0x7f0938005c00 comp rx=0 tx=0).stop 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 msgr2=0x7f094419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 0x7f094419daf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f093400d950 tx=0x7f093400dc60 comp rx=0 tx=0).stop 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 shutdown_connections 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f092c0a20a0 0x7f092c0a4560 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09440ff330 0x7f094419daf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0944107260 0x7f09441a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 --2- 172.21.15.7:0/1794138807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440ffc90 0x7f094419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:02:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 >> 172.21.15.7:0/1794138807 conn(0x7f09440fad60 msgr2=0x7f0944100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:02:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 shutdown_connections 2024-01-11T18:02:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:02:53.449+0000 7f0949fff700 1 -- 172.21.15.7:0/1794138807 wait complete. 2024-01-11T18:02:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 265.0 s, 819.8 tps, lat 121.358 ms stddev 166.445 2024-01-11T18:02:55.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:55 smithi007 ceph-mon[33361]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 MiB/s wr, 558 op/s 2024-01-11T18:02:55.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:55 smithi016 ceph-mon[43195]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 MiB/s wr, 558 op/s 2024-01-11T18:02:56.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:55 smithi038 ceph-mon[43582]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 MiB/s wr, 558 op/s 2024-01-11T18:02:57.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:57 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:02:57.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:57 smithi016 ceph-mon[43195]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 566 op/s 2024-01-11T18:02:58.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:57 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:02:58.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:57 smithi038 ceph-mon[43582]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 566 op/s 2024-01-11T18:02:58.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:57 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:02:58.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:57 smithi007 ceph-mon[33361]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 566 op/s 2024-01-11T18:02:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 270.0 s, 773.0 tps, lat 130.064 ms stddev 154.323 2024-01-11T18:03:00.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:02:59 smithi016 ceph-mon[43195]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 MiB/s wr, 562 op/s 2024-01-11T18:03:00.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:02:59 smithi038 ceph-mon[43582]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 MiB/s wr, 562 op/s 2024-01-11T18:03:00.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:02:59 smithi007 ceph-mon[33361]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 MiB/s wr, 562 op/s 2024-01-11T18:03:02.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:01 smithi016 ceph-mon[43195]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-01-11T18:03:02.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:01 smithi038 ceph-mon[43582]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-01-11T18:03:02.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:01 smithi007 ceph-mon[33361]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 606 op/s 2024-01-11T18:03:04.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 275.0 s, 965.3 tps, lat 104.762 ms stddev 135.941 2024-01-11T18:03:04.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:03 smithi016 ceph-mon[43195]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-01-11T18:03:04.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:03 smithi038 ceph-mon[43582]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-01-11T18:03:04.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:03 smithi007 ceph-mon[33361]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-01-11T18:03:05.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:05 smithi016 ceph-mon[43195]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-01-11T18:03:06.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:05 smithi038 ceph-mon[43582]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-01-11T18:03:06.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:05 smithi007 ceph-mon[33361]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-01-11T18:03:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 -- 172.21.15.7:0/4248530191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 msgr2=0x7ff314100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 --2- 172.21.15.7:0/4248530191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314100230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff3040098f0 tx=0x7ff304009c00 comp rx=0 tx=0).stop 2024-01-11T18:03:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 -- 172.21.15.7:0/4248530191 shutdown_connections 2024-01-11T18:03:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 --2- 172.21.15.7:0/4248530191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314100230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 --2- 172.21.15.7:0/4248530191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314101130 0x7ff3141015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 --2- 172.21.15.7:0/4248530191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff314100770 0x7ff314100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.720+0000 7ff31abff700 1 -- 172.21.15.7:0/4248530191 >> 172.21.15.7:0/4248530191 conn(0x7ff3140fba30 msgr2=0x7ff3140fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.721+0000 7ff31abff700 1 -- 172.21.15.7:0/4248530191 shutdown_connections 2024-01-11T18:03:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.721+0000 7ff31abff700 1 -- 172.21.15.7:0/4248530191 wait complete. 2024-01-11T18:03:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.723+0000 7ff31abff700 1 Processor -- start 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.723+0000 7ff31abff700 1 -- start start 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31abff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314105990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31abff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100770 0x7ff314105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31abff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff314101130 0x7ff314102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31abff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314102cc0 con 0x7ff314100770 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31abff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314102e30 con 0x7ff314101130 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31abff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314102ff0 con 0x7ff3140ffe10 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31899b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31899b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35174/0 (socket says 172.21.15.7:35174) 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31899b700 1 -- 172.21.15.7:0/1725416722 learned_addr learned my addr 172.21.15.7:0/1725416722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff31919c700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff314101130 0x7ff314102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.724+0000 7ff313fff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100770 0x7ff314105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff31899b700 1 -- 172.21.15.7:0/1725416722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff314101130 msgr2=0x7ff314102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff31899b700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff314101130 0x7ff314102520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff31899b700 1 -- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100770 msgr2=0x7ff314105ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff31899b700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100770 0x7ff314105ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff31899b700 1 -- 172.21.15.7:0/1725416722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c0095f0 con 0x7ff3140ffe10 2024-01-11T18:03:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff313fff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100770 0x7ff314105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:03:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.725+0000 7ff31899b700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314105990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff304009230 tx=0x7ff30400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.726+0000 7ff311ffb700 1 -- 172.21.15.7:0/1725416722 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30401c070 con 0x7ff3140ffe10 2024-01-11T18:03:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.726+0000 7ff311ffb700 1 -- 172.21.15.7:0/1725416722 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30400f840 con 0x7ff3140ffe10 2024-01-11T18:03:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.726+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff304009520 con 0x7ff3140ffe10 2024-01-11T18:03:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.726+0000 7ff311ffb700 1 -- 172.21.15.7:0/1725416722 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304004c00 con 0x7ff3140ffe10 2024-01-11T18:03:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.727+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141997b0 con 0x7ff3140ffe10 2024-01-11T18:03:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.732+0000 7ff311ffb700 1 -- 172.21.15.7:0/1725416722 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7ff304020070 con 0x7ff3140ffe10 2024-01-11T18:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.734+0000 7ff32e941b80 1 Processor -- start 2024-01-11T18:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.735+0000 7ff32e941b80 1 -- start start 2024-01-11T18:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.736+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f3a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.736+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f41e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff32e941b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7e31030 con 0x559dc7f39f20 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff32e941b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7e94ac0 con 0x559dc7f3ac10 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff32e941b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7e923e0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff311ffb700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2f40a1f90 0x7ff2f40a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff31919c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f3a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff31919c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f3a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41326/0 (socket says 172.21.15.7:41326) 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff31919c700 1 -- 172.21.15.7:0/2504590457 learned_addr learned my addr 172.21.15.7:0/2504590457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.737+0000 7ff311ffb700 1 -- 172.21.15.7:0/1725416722 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff3040ceb00 con 0x7ff3140ffe10 2024-01-11T18:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff31919c700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2f40a1f90 0x7ff2f40a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff313fff700 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f41e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff31919c700 1 -- 172.21.15.7:0/2504590457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 msgr2=0x559dc7f41e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff31919c700 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f41e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff31919c700 1 -- 172.21.15.7:0/2504590457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 msgr2=0x559dc7f3b090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff31919c700 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.738+0000 7ff31919c700 1 -- 172.21.15.7:0/2504590457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dc7d02350 con 0x559dc7f39f20 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.739+0000 7ff31919c700 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f3a340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff30c015da0 tx=0x7ff30c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808b02c6ca9cb046 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.739+0000 7ff31919c700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2f40a1f90 0x7ff2f40a4450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009fd0 tx=0x7ff30c01d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.739+0000 7ff30affd700 1 -- 172.21.15.7:0/2504590457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c012d30 con 0x559dc7f39f20 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.739+0000 7ff30affd700 1 -- 172.21.15.7:0/2504590457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30c01b040 con 0x559dc7f39f20 2024-01-11T18:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.739+0000 7ff30affd700 1 -- 172.21.15.7:0/2504590457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004c90 con 0x559dc7f39f20 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.740+0000 7ff32e941b80 1 -- 172.21.15.7:0/2504590457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 msgr2=0x559dc7f3a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.740+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f3a340 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff30c015da0 tx=0x7ff30c012670 comp rx=0 tx=0).stop 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 -- 172.21.15.7:0/2504590457 shutdown_connections 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f41e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f3a340 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2504590457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 -- 172.21.15.7:0/2504590457 >> 172.21.15.7:0/2504590457 conn(0x559dc7eaf740 msgr2=0x559dc7f34950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 -- 172.21.15.7:0/2504590457 shutdown_connections 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 -- 172.21.15.7:0/2504590457 wait complete. 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.741+0000 7ff32e941b80 1 -- start start 2024-01-11T18:03:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 Processor -- start 2024-01-11T18:03:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7e8dcd0 con 0x559dc7f39f20 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7dfc340 con 0x559dc7f3b5d0 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7dfbfe0 con 0x559dc7f3ac10 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.742+0000 7ff32e941b80 10 client.0 _ll_register_callbacks cb 0x559dc7e2e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f43fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35192/0 (socket says 172.21.15.7:35192) 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 -- 172.21.15.7:0/4067356967 learned_addr learned my addr 172.21.15.7:0/4067356967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff31899b700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff31919c700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 -- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 msgr2=0x559dc7f4af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 -- 172.21.15.7:0/4067356967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 msgr2=0x559dc7f43a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f43a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff313fff700 1 -- 172.21.15.7:0/4067356967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c015980 con 0x559dc7f3ac10 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.743+0000 7ff31899b700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff313fff700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f43fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e8f0 tx=0x7ff2fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a9f2426818179c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff303fff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff303fff700 1 -- 172.21.15.7:0/4067356967 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00cb60 con 0x559dc7f3ac10 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dc7f2cd60 con 0x559dc7f3ac10 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff303fff700 1 -- 172.21.15.7:0/4067356967 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff2fc00ccc0 con 0x559dc7f3ac10 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff303fff700 1 -- 172.21.15.7:0/4067356967 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2fc00ce80 con 0x559dc7f3ac10 2024-01-11T18:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff32e941b80 10 client.26053 fetch_fsmap learned FSMap version 33 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.744+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559dc7caeb00 con 0x559dc7f3ac10 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff303fff700 1 -- 172.21.15.7:0/4067356967 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559dc7f2cd60 con 0x559dc7f3ac10 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff303fff700 1 -- 172.21.15.7:0/4067356967 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff2fc01a450 con 0x559dc7f3ac10 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff32e941b80 10 client.26053 fetch_fsmap finished waiting for FSMap version 33 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff32e941b80 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff32e941b80 20 client.26053 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff32e941b80 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7f4f960 0x559dc7f51d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff32e941b80 4 client.26053 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dc7e86b60 con 0x559dc7f4f960 2024-01-11T18:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.745+0000 7ff31899b700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7f4f960 0x559dc7f51d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.747+0000 7ff31899b700 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7f4f960 0x559dc7f51d80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6b1e987da2b59c server_cookie=fd572f2e585c44f1 in_seq=0 out_seq=0 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.747+0000 7ff303fff700 10 client.26053 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff303fff700 1 -- 172.21.15.7:0/4067356967 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559dc7e86b60 con 0x559dc7f4f960 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff303fff700 10 client.26053 handle_command_reply: tid=0 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 client.26053 shutdown 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 msgr2=0x559dc7f43fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f43fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e8f0 tx=0x7ff2fc00ecb0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 shutdown_connections 2024-01-11T18:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7f4f960 0x559dc7f51d80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 --2- 172.21.15.7:0/4067356967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f43fa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff303fff700 0 client.26053 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.748+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 >> 172.21.15.7:0/4067356967 conn(0x559dc7eaf740 msgr2=0x559dc7f33f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.749+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 shutdown_connections 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.749+0000 7ff32e941b80 1 -- 172.21.15.7:0/4067356967 wait complete. 2024-01-11T18:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.749+0000 7ff32e941b80 20 client.26053 trim_cache size 0 max 16384 2024-01-11T18:03:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.774+0000 7ff32e941b80 1 Processor -- start 2024-01-11T18:03:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.774+0000 7ff32e941b80 1 -- start start 2024-01-11T18:03:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff32e941b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7d04640 con 0x559dc7f39f20 2024-01-11T18:03:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff32e941b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7f3a7a0 con 0x559dc7f3ac10 2024-01-11T18:03:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff32e941b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7f435d0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff31919c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff31919c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35202/0 (socket says 172.21.15.7:35202) 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.775+0000 7ff31919c700 1 -- 172.21.15.7:0/2747550318 learned_addr learned my addr 172.21.15.7:0/2747550318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff313fff700 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff31919c700 1 -- 172.21.15.7:0/2747550318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 msgr2=0x559dc7f3de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff31919c700 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff31919c700 1 -- 172.21.15.7:0/2747550318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 msgr2=0x559dc7f51d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff31919c700 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff31919c700 1 -- 172.21.15.7:0/2747550318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dc7e4f1d0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.776+0000 7ff31919c700 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0095c0 tx=0x7ff30c015c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91db7ca17a3b0c57 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.777+0000 7ff30a7fc700 1 -- 172.21.15.7:0/2747550318 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c005da0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.777+0000 7ff30a7fc700 1 -- 172.21.15.7:0/2747550318 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30c0046b0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.777+0000 7ff30a7fc700 1 -- 172.21.15.7:0/2747550318 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0316d0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 -- 172.21.15.7:0/2747550318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 msgr2=0x559dc7f4af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0095c0 tx=0x7ff30c015c50 comp rx=0 tx=0).stop 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 -- 172.21.15.7:0/2747550318 shutdown_connections 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3b5d0 0x559dc7f4af30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f51d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 --2- 172.21.15.7:0/2747550318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3ac10 0x559dc7f3de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.778+0000 7ff32e941b80 1 -- 172.21.15.7:0/2747550318 >> 172.21.15.7:0/2747550318 conn(0x559dc7eaf740 msgr2=0x559dc7f3e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.779+0000 7ff32e941b80 1 -- 172.21.15.7:0/2747550318 shutdown_connections 2024-01-11T18:03:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.779+0000 7ff32e941b80 1 -- 172.21.15.7:0/2747550318 wait complete. 2024-01-11T18:03:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.780+0000 7ff32e941b80 1 -- start start 2024-01-11T18:03:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 Processor -- start 2024-01-11T18:03:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f48720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f48c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f49c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7f4a160 con 0x559dc7f39f20 2024-01-11T18:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7f4a2d0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.781+0000 7ff32e941b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dc7f4a490 con 0x559dc7f3ac10 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff32e941b80 10 client.0 _ll_register_callbacks cb 0x559dc7e2e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff313fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f48c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31919c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f49c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31899b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f48720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff313fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f48c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35216/0 (socket says 172.21.15.7:35216) 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff313fff700 1 -- 172.21.15.7:0/3797278004 learned_addr learned my addr 172.21.15.7:0/3797278004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31919c700 1 -- 172.21.15.7:0/3797278004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 msgr2=0x559dc7f48c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31919c700 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f48c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31919c700 1 -- 172.21.15.7:0/3797278004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 msgr2=0x559dc7f48720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31919c700 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f48720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.782+0000 7ff31919c700 1 -- 172.21.15.7:0/3797278004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dc7d02350 con 0x559dc7f3b5d0 2024-01-11T18:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.783+0000 7ff31919c700 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f49c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009f70 tx=0x7ff30c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc4ee7fb4fb42e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.783+0000 7ff303fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.783+0000 7ff303fff700 1 -- 172.21.15.7:0/3797278004 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00bd40 con 0x559dc7f3b5d0 2024-01-11T18:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.783+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dc7e86880 con 0x559dc7f3b5d0 2024-01-11T18:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.783+0000 7ff303fff700 1 -- 172.21.15.7:0/3797278004 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff30c005040 con 0x559dc7f3b5d0 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.784+0000 7ff303fff700 1 -- 172.21.15.7:0/3797278004 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff30c015780 con 0x559dc7f3b5d0 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.784+0000 7ff32e941b80 10 client.26021 fetch_fsmap learned FSMap version 33 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.784+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559dc7d06c70 con 0x559dc7f3b5d0 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.784+0000 7ff303fff700 1 -- 172.21.15.7:0/3797278004 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559dc7e86880 con 0x559dc7f3b5d0 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.784+0000 7ff303fff700 1 -- 172.21.15.7:0/3797278004 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff30c004be0 con 0x559dc7f3b5d0 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff32e941b80 10 client.26021 fetch_fsmap finished waiting for FSMap version 33 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff32e941b80 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff32e941b80 20 client.26021 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff32e941b80 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7fa6430 0x559dc7fa8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff32e941b80 4 client.26021 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559dc7ccfc90 con 0x559dc7fa6430 2024-01-11T18:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.785+0000 7ff31899b700 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7fa6430 0x559dc7fa8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.786+0000 7ff31899b700 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7fa6430 0x559dc7fa8850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a835ce0e2815f7b server_cookie=5036df39c4188da5 in_seq=0 out_seq=0 2024-01-11T18:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.786+0000 7ff303fff700 10 client.26021 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.801+0000 7ff303fff700 1 -- 172.21.15.7:0/3797278004 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559dc7ccfc90 con 0x559dc7fa6430 2024-01-11T18:03:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.801+0000 7ff303fff700 10 client.26021 handle_command_reply: tid=0 2024-01-11T18:03:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.801+0000 7ff32e941b80 1 client.26021 shutdown 2024-01-11T18:03:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 msgr2=0x559dc7f49c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f49c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009f70 tx=0x7ff30c005e40 comp rx=0 tx=0).stop 2024-01-11T18:03:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 shutdown_connections 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559dc7fa6430 0x559dc7fa8850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dc7f3b5d0 0x559dc7f49c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559dc7f39f20 0x559dc7f48720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 --2- 172.21.15.7:0/3797278004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559dc7f3ac10 0x559dc7f48c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 >> 172.21.15.7:0/3797278004 conn(0x559dc7eaf740 msgr2=0x559dc7f43a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff303fff700 0 client.26021 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 shutdown_connections 2024-01-11T18:03:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.802+0000 7ff32e941b80 1 -- 172.21.15.7:0/3797278004 wait complete. 2024-01-11T18:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.803+0000 7ff32e941b80 20 client.26021 trim_cache size 0 max 16384 2024-01-11T18:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.804+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2f40a1f90 msgr2=0x7ff2f40a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.804+0000 7ff31abff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2f40a1f90 0x7ff2f40a4450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009fd0 tx=0x7ff30c01d000 comp rx=0 tx=0).stop 2024-01-11T18:03:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.804+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 msgr2=0x7ff314105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.804+0000 7ff31abff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314105990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff304009230 tx=0x7ff30400b840 comp rx=0 tx=0).stop 2024-01-11T18:03:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.805+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 shutdown_connections 2024-01-11T18:03:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.805+0000 7ff31abff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff2f40a1f90 0x7ff2f40a4450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.805+0000 7ff31abff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3140ffe10 0x7ff314105990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.805+0000 7ff31abff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff314101130 0x7ff314102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.805+0000 7ff31abff700 1 --2- 172.21.15.7:0/1725416722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff314100770 0x7ff314105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.805+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 >> 172.21.15.7:0/1725416722 conn(0x7ff3140fba30 msgr2=0x7ff3140fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.806+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 shutdown_connections 2024-01-11T18:03:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:06.806+0000 7ff31abff700 1 -- 172.21.15.7:0/1725416722 wait complete. 2024-01-11T18:03:07.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:07 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:07.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:07 smithi016 ceph-mon[43195]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 595 op/s 2024-01-11T18:03:08.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:07 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:08.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:07 smithi038 ceph-mon[43582]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 595 op/s 2024-01-11T18:03:08.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:07 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:08.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:07 smithi007 ceph-mon[33361]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 595 op/s 2024-01-11T18:03:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 280.0 s, 721.8 tps, lat 137.773 ms stddev 160.796 2024-01-11T18:03:10.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:09 smithi016 ceph-mon[43195]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-01-11T18:03:10.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:09 smithi038 ceph-mon[43582]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-01-11T18:03:10.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:09 smithi007 ceph-mon[33361]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-01-11T18:03:12.248 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:11 smithi016 ceph-mon[43195]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-01-11T18:03:12.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:11 smithi038 ceph-mon[43582]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-01-11T18:03:12.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:11 smithi007 ceph-mon[33361]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-01-11T18:03:14.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 285.0 s, 565.6 tps, lat 175.900 ms stddev 175.483 2024-01-11T18:03:14.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:13 smithi016 ceph-mon[43195]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 557 op/s 2024-01-11T18:03:14.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:13 smithi038 ceph-mon[43582]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 557 op/s 2024-01-11T18:03:14.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:13 smithi007 ceph-mon[33361]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 557 op/s 2024-01-11T18:03:15.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:14 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:15.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:14 smithi016 ceph-mon[43195]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 MiB/s wr, 566 op/s 2024-01-11T18:03:15.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:14 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:15.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:14 smithi038 ceph-mon[43582]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 MiB/s wr, 566 op/s 2024-01-11T18:03:15.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:14 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:15.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:14 smithi007 ceph-mon[33361]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 MiB/s wr, 566 op/s 2024-01-11T18:03:18.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:17 smithi016 ceph-mon[43195]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-01-11T18:03:18.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:17 smithi038 ceph-mon[43582]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-01-11T18:03:18.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:17 smithi007 ceph-mon[33361]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-01-11T18:03:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 290.0 s, 915.3 tps, lat 109.039 ms stddev 131.638 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.080+0000 7f1a869ff700 1 -- 172.21.15.7:0/456089949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010dc90 msgr2=0x7f1a80110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.080+0000 7f1a869ff700 1 --2- 172.21.15.7:0/456089949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010dc90 0x7f1a80110150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800c310 tx=0x7f1a7800c620 comp rx=0 tx=0).stop 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.080+0000 7f1a869ff700 1 -- 172.21.15.7:0/456089949 shutdown_connections 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.080+0000 7f1a869ff700 1 --2- 172.21.15.7:0/456089949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010dc90 0x7f1a80110150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.081+0000 7f1a869ff700 1 --2- 172.21.15.7:0/456089949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a801087f0 0x7f1a8010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.081+0000 7f1a869ff700 1 --2- 172.21.15.7:0/456089949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010b1b0 0x7f1a8010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.081+0000 7f1a869ff700 1 -- 172.21.15.7:0/456089949 >> 172.21.15.7:0/456089949 conn(0x7f1a8006e500 msgr2=0x7f1a80070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.082+0000 7f1a869ff700 1 -- 172.21.15.7:0/456089949 shutdown_connections 2024-01-11T18:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.082+0000 7f1a869ff700 1 -- 172.21.15.7:0/456089949 wait complete. 2024-01-11T18:03:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.084+0000 7f1a869ff700 1 Processor -- start 2024-01-11T18:03:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.084+0000 7f1a869ff700 1 -- start start 2024-01-11T18:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.085+0000 7f1a869ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a801087f0 0x7f1a8019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.085+0000 7f1a869ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010b1b0 0x7f1a8019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.085+0000 7f1a869ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010dc90 0x7f1a801a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.085+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010b1b0 0x7f1a8019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010b1b0 0x7f1a8019e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:56600/0 (socket says 172.21.15.7:56600) 2024-01-11T18:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a801087f0 0x7f1a8019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/2307799061 learned_addr learned my addr 172.21.15.7:0/2307799061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010dc90 0x7f1a801a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a801a9cf0 con 0x7f1a801087f0 2024-01-11T18:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a801a9e30 con 0x7f1a8010dc90 2024-01-11T18:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.086+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a801a9f70 con 0x7f1a8010b1b0 2024-01-11T18:03:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.087+0000 7f1a84f9c700 1 -- 172.21.15.7:0/2307799061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010b1b0 msgr2=0x7f1a8019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.087+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010b1b0 0x7f1a8019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.087+0000 7f1a84f9c700 1 -- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a801087f0 msgr2=0x7f1a8019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.087+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a801087f0 0x7f1a8019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.087+0000 7f1a84f9c700 1 -- 172.21.15.7:0/2307799061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7800c050 con 0x7f1a8010dc90 2024-01-11T18:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.087+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010dc90 0x7f1a801a97b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a78016000 tx=0x7f1a78014400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.088+0000 7f1a7d7fa700 1 -- 172.21.15.7:0/2307799061 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7800f040 con 0x7f1a8010dc90 2024-01-11T18:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.088+0000 7f1a7d7fa700 1 -- 172.21.15.7:0/2307799061 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a78003e10 con 0x7f1a8010dc90 2024-01-11T18:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.088+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a801aa170 con 0x7f1a8010dc90 2024-01-11T18:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.088+0000 7f1a7d7fa700 1 -- 172.21.15.7:0/2307799061 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78005680 con 0x7f1a8010dc90 2024-01-11T18:03:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.088+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a801aa590 con 0x7f1a8010dc90 2024-01-11T18:03:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.096+0000 7f1a7d7fa700 1 -- 172.21.15.7:0/2307799061 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1a78014c60 con 0x7f1a8010dc90 2024-01-11T18:03:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.096+0000 7f1a9a762b80 1 Processor -- start 2024-01-11T18:03:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 -- start start 2024-01-11T18:03:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c45f20 0x558075c46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c46c10 0x558075c47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c4de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075b3d030 con 0x558075c45f20 2024-01-11T18:03:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075ba0ac0 con 0x558075c46c10 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.098+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075b9e3e0 con 0x558075c475d0 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c4de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c4de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:56612/0 (socket says 172.21.15.7:56612) 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/2949035192 learned_addr learned my addr 172.21.15.7:0/2949035192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/2949035192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c46c10 msgr2=0x558075c47090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c46c10 0x558075c47090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/2949035192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c45f20 msgr2=0x558075c46340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c45f20 0x558075c46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.100+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/2949035192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558075a0e350 con 0x558075c475d0 2024-01-11T18:03:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.101+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c4de60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000ad80 tx=0x7f1a70011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a0fd8c164b4bba server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.101+0000 7f1a7d7fa700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1a680a1f10 0x7f1a680a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.101+0000 7f1a7d7fa700 1 -- 172.21.15.7:0/2307799061 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1a780cf2d0 con 0x7f1a8010dc90 2024-01-11T18:03:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.102+0000 7f1a7ffff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1a680a1f10 0x7f1a680a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.103+0000 7f1a7ffff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1a680a1f10 0x7f1a680a43d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x558075c427f0 tx=0x7f1a7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.103+0000 7f1a667fc700 1 -- 172.21.15.7:0/2949035192 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70004220 con 0x558075c475d0 2024-01-11T18:03:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.103+0000 7f1a667fc700 1 -- 172.21.15.7:0/2949035192 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7000aed0 con 0x558075c475d0 2024-01-11T18:03:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.103+0000 7f1a667fc700 1 -- 172.21.15.7:0/2949035192 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a700054e0 con 0x558075c475d0 2024-01-11T18:03:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.104+0000 7f1a9a762b80 1 -- 172.21.15.7:0/2949035192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 msgr2=0x558075c4de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.104+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c4de60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000ad80 tx=0x7f1a70011720 comp rx=0 tx=0).stop 2024-01-11T18:03:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 -- 172.21.15.7:0/2949035192 shutdown_connections 2024-01-11T18:03:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c4de60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c45f20 0x558075c46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/2949035192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c46c10 0x558075c47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 -- 172.21.15.7:0/2949035192 >> 172.21.15.7:0/2949035192 conn(0x558075bbb740 msgr2=0x558075c40950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 -- 172.21.15.7:0/2949035192 shutdown_connections 2024-01-11T18:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.105+0000 7f1a9a762b80 1 -- 172.21.15.7:0/2949035192 wait complete. 2024-01-11T18:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.106+0000 7f1a9a762b80 1 -- start start 2024-01-11T18:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.106+0000 7f1a9a762b80 1 Processor -- start 2024-01-11T18:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c58730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075b99cd0 con 0x558075c46c10 2024-01-11T18:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075b08340 con 0x558075c45f20 2024-01-11T18:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075b07fe0 con 0x558075c475d0 2024-01-11T18:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a9a762b80 10 client.0 _ll_register_callbacks cb 0x558075b3a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c58730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a84f9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38100/0 (socket says 172.21.15.7:38100) 2024-01-11T18:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.107+0000 7f1a7ffff700 1 -- 172.21.15.7:0/1608100701 learned_addr learned my addr 172.21.15.7:0/1608100701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a84f9c700 1 -- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 msgr2=0x558075c581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a84f9c700 1 -- 172.21.15.7:0/1608100701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 msgr2=0x558075c58730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c58730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a84f9c700 1 -- 172.21.15.7:0/1608100701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7000aa60 con 0x558075c475d0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c58730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5f770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a78012400 tx=0x7f1a7809bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec7456afe218bce server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.108+0000 7f1a5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1608100701 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a780cf170 con 0x558075c475d0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1608100701 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1a7809c490 con 0x558075c475d0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558075c38d60 con 0x558075c475d0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1608100701 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7809bd90 con 0x558075c475d0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a9a762b80 10 client.26071 fetch_fsmap learned FSMap version 34 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5580759bab00 con 0x558075c475d0 2024-01-11T18:03:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.109+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1608100701 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558075c38d60 con 0x558075c475d0 2024-01-11T18:03:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1608100701 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1a7809b310 con 0x558075c475d0 2024-01-11T18:03:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a9a762b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T18:03:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a9a762b80 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a9a762b80 20 client.26071 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075c63d30 0x558075c66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a9a762b80 4 client.26071 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:03:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.110+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558075b92b60 con 0x558075c63d30 2024-01-11T18:03:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.111+0000 7f1a7ffff700 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075c63d30 0x558075c66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.112+0000 7f1a7ffff700 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075c63d30 0x558075c66150 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cb796041eddeca server_cookie=6cffcc0ae75c543b in_seq=0 out_seq=0 2024-01-11T18:03:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.113+0000 7f1a5f7fe700 10 client.26071 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.113+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1608100701 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558075b92b60 con 0x558075c63d30 2024-01-11T18:03:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.113+0000 7f1a5f7fe700 10 client.26071 handle_command_reply: tid=0 2024-01-11T18:03:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.113+0000 7f1a9a762b80 1 client.26071 shutdown 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.113+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 msgr2=0x558075c5f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5f770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a78012400 tx=0x7f1a7809bb60 comp rx=0 tx=0).stop 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 shutdown_connections 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075c63d30 0x558075c66150 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5f770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a5f7fe700 0 client.26071 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1608100701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c58730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 >> 172.21.15.7:0/1608100701 conn(0x558075bbb740 msgr2=0x558075c3ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 shutdown_connections 2024-01-11T18:03:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1608100701 wait complete. 2024-01-11T18:03:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.114+0000 7f1a9a762b80 20 client.26071 trim_cache size 0 max 16384 2024-01-11T18:03:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 Processor -- start 2024-01-11T18:03:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 -- start start 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c45f20 0x558075c46ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c475d0 0x558075c5fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075a10640 con 0x558075c46c10 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075c46ad0 con 0x558075c475d0 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.153+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075c41810 con 0x558075c45f20 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.154+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c45f20 0x558075c46ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.154+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.154+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37926/0 (socket says 172.21.15.7:37926) 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.154+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/1696411474 learned_addr learned my addr 172.21.15.7:0/1696411474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.154+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/1696411474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c45f20 msgr2=0x558075c46ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.154+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c45f20 0x558075c46ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.155+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/1696411474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c475d0 msgr2=0x558075c5fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.155+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c475d0 0x558075c5fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.155+0000 7f1a7f7fe700 1 -- 172.21.15.7:0/1696411474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558075b5b1d0 con 0x558075c46c10 2024-01-11T18:03:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.155+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5f730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005950 tx=0x7f1a7000a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cf7b7ba42d409e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.159+0000 7f1a65ffb700 1 -- 172.21.15.7:0/1696411474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70020070 con 0x558075c46c10 2024-01-11T18:03:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.160+0000 7f1a65ffb700 1 -- 172.21.15.7:0/1696411474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a700054f0 con 0x558075c46c10 2024-01-11T18:03:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.160+0000 7f1a65ffb700 1 -- 172.21.15.7:0/1696411474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70004e70 con 0x558075c46c10 2024-01-11T18:03:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.161+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1696411474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 msgr2=0x558075c5f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.161+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5f730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005950 tx=0x7f1a7000a740 comp rx=0 tx=0).stop 2024-01-11T18:03:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.162+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1696411474 shutdown_connections 2024-01-11T18:03:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.162+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c45f20 0x558075c46ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.162+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c475d0 0x558075c5fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.162+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1696411474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5f730 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.162+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1696411474 >> 172.21.15.7:0/1696411474 conn(0x558075bbb740 msgr2=0x558075c4dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.162+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1696411474 shutdown_connections 2024-01-11T18:03:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.163+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1696411474 wait complete. 2024-01-11T18:03:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.163+0000 7f1a9a762b80 1 -- start start 2024-01-11T18:03:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.164+0000 7f1a9a762b80 1 Processor -- start 2024-01-11T18:03:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075c5ef70 con 0x558075c46c10 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075c5f0e0 con 0x558075c45f20 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558075c5f2a0 con 0x558075c475d0 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a9a762b80 10 client.0 _ll_register_callbacks cb 0x558075b3a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.165+0000 7f1a7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c5d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38124/0 (socket says 172.21.15.7:38124) 2024-01-11T18:03:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.166+0000 7f1a7ffff700 1 -- 172.21.15.7:0/1259871399 learned_addr learned my addr 172.21.15.7:0/1259871399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.166+0000 7f1a7f7fe700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.166+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.167+0000 7f1a84f9c700 1 -- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 msgr2=0x558075c5d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.167+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c5d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.167+0000 7f1a84f9c700 1 -- 172.21.15.7:0/1259871399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 msgr2=0x558075c5da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.167+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.168+0000 7f1a84f9c700 1 -- 172.21.15.7:0/1259871399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558075a0e350 con 0x558075c475d0 2024-01-11T18:03:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.168+0000 7f1a84f9c700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5ea30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a7809a0e0 tx=0x7f1a7809c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa00268a57de673 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.168+0000 7f1a5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:03:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.170+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1259871399 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a780cf170 con 0x558075c475d0 2024-01-11T18:03:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.170+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1259871399 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1a7809bd90 con 0x558075c475d0 2024-01-11T18:03:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.170+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a7000aed0 con 0x558075c475d0 2024-01-11T18:03:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.170+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1259871399 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a78007b60 con 0x558075c475d0 2024-01-11T18:03:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.170+0000 7f1a9a762b80 10 client.26077 fetch_fsmap learned FSMap version 34 2024-01-11T18:03:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.170+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558075a12c70 con 0x558075c475d0 2024-01-11T18:03:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.173+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1259871399 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7000aed0 con 0x558075c475d0 2024-01-11T18:03:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.173+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1259871399 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1a78097b80 con 0x558075c475d0 2024-01-11T18:03:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.173+0000 7f1a9a762b80 10 client.26077 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T18:03:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.173+0000 7f1a9a762b80 10 client.26077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.173+0000 7f1a9a762b80 20 client.26077 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.173+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075cbabd0 0x558075cbcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.174+0000 7f1a9a762b80 4 client.26077 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:03:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.174+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558075b92880 con 0x558075cbabd0 2024-01-11T18:03:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.174+0000 7f1a7ffff700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075cbabd0 0x558075cbcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.175+0000 7f1a5f7fe700 10 client.26077 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.177+0000 7f1a7ffff700 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075cbabd0 0x558075cbcfc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be663ae959cd2dd server_cookie=1efcb62446433f7 in_seq=0 out_seq=0 2024-01-11T18:03:20.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:19 smithi016 ceph-mon[43195]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-01-11T18:03:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.261+0000 7f1a5f7fe700 1 -- 172.21.15.7:0/1259871399 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558075b92880 con 0x558075cbabd0 2024-01-11T18:03:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.261+0000 7f1a5f7fe700 10 client.26077 handle_command_reply: tid=0 2024-01-11T18:03:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.261+0000 7f1a9a762b80 1 client.26077 shutdown 2024-01-11T18:03:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.261+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 msgr2=0x558075c5ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.261+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5ea30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a7809a0e0 tx=0x7f1a7809c220 comp rx=0 tx=0).stop 2024-01-11T18:03:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 shutdown_connections 2024-01-11T18:03:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558075cbabd0 0x558075cbcfc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558075c475d0 0x558075c5ea30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558075c45f20 0x558075c5d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 --2- 172.21.15.7:0/1259871399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558075c46c10 0x558075c5da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 >> 172.21.15.7:0/1259871399 conn(0x558075bbb740 msgr2=0x558075c4db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a5f7fe700 0 client.26077 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 shutdown_connections 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 1 -- 172.21.15.7:0/1259871399 wait complete. 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.262+0000 7f1a9a762b80 20 client.26077 trim_cache size 0 max 16384 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:03:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:03:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:03:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:03:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.263+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1a680a1f10 msgr2=0x7f1a680a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.263+0000 7f1a869ff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1a680a1f10 0x7f1a680a43d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x558075c427f0 tx=0x7f1a7400b410 comp rx=0 tx=0).stop 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.263+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010dc90 msgr2=0x7f1a801a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.263+0000 7f1a869ff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010dc90 0x7f1a801a97b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a78016000 tx=0x7f1a78014400 comp rx=0 tx=0).stop 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 shutdown_connections 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1a680a1f10 0x7f1a680a43d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8010dc90 0x7f1a801a97b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a801087f0 0x7f1a8019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 --2- 172.21.15.7:0/2307799061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a8010b1b0 0x7f1a8019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 >> 172.21.15.7:0/2307799061 conn(0x7f1a8006e500 msgr2=0x7f1a80108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 shutdown_connections 2024-01-11T18:03:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:20.264+0000 7f1a869ff700 1 -- 172.21.15.7:0/2307799061 wait complete. 2024-01-11T18:03:20.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:19 smithi038 ceph-mon[43582]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-01-11T18:03:20.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:19 smithi007 ceph-mon[33361]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.8 MiB/s wr, 542 op/s 2024-01-11T18:03:22.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:21 smithi016 ceph-mon[43195]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-01-11T18:03:22.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:21 smithi038 ceph-mon[43582]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-01-11T18:03:22.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:21 smithi007 ceph-mon[33361]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-01-11T18:03:23.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:22 smithi016 ceph-mon[43195]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 520 op/s 2024-01-11T18:03:23.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:22 smithi038 ceph-mon[43582]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 520 op/s 2024-01-11T18:03:23.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:22 smithi007 ceph-mon[33361]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 520 op/s 2024-01-11T18:03:24.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 295.0 s, 755.6 tps, lat 134.209 ms stddev 141.005 2024-01-11T18:03:25.991 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:25 smithi016 ceph-mon[43195]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-01-11T18:03:26.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:25 smithi038 ceph-mon[43582]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-01-11T18:03:26.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:25 smithi007 ceph-mon[33361]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-01-11T18:03:28.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:27 smithi016 ceph-mon[43195]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 548 op/s 2024-01-11T18:03:28.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:27 smithi038 ceph-mon[43582]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 548 op/s 2024-01-11T18:03:28.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:27 smithi007 ceph-mon[33361]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 548 op/s 2024-01-11T18:03:29.119 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 300.0 s, 808.2 tps, lat 120.738 ms stddev 147.325 2024-01-11T18:03:29.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:28 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:03:29.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:28 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:03:29.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:28 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:03:30.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:29 smithi016 ceph-mon[43195]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.2 MiB/s wr, 534 op/s 2024-01-11T18:03:30.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:03:30.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:03:30.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:29 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:03:30.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:29 smithi038 ceph-mon[43582]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.2 MiB/s wr, 534 op/s 2024-01-11T18:03:30.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:03:30.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:03:30.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:29 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:03:30.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:29 smithi007 ceph-mon[33361]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.2 MiB/s wr, 534 op/s 2024-01-11T18:03:30.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:03:30.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:03:30.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:29 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:03:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:31 smithi016 ceph-mon[43195]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 601 op/s 2024-01-11T18:03:32.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:31 smithi038 ceph-mon[43582]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 601 op/s 2024-01-11T18:03:32.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:31 smithi007 ceph-mon[33361]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 601 op/s 2024-01-11T18:03:33.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:32 smithi016 ceph-mon[43195]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-01-11T18:03:33.283 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:32 smithi038 ceph-mon[43582]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-01-11T18:03:33.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:32 smithi007 ceph-mon[33361]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 -- 172.21.15.7:0/740565695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7368074010 msgr2=0x7f7368074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 --2- 172.21.15.7:0/740565695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7368074010 0x7f7368074490 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f735c0098f0 tx=0x7f735c009c00 comp rx=0 tx=0).stop 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 -- 172.21.15.7:0/740565695 shutdown_connections 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 --2- 172.21.15.7:0/740565695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73681094a0 0x7f736810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 --2- 172.21.15.7:0/740565695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73680755f0 0x7f7368073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 --2- 172.21.15.7:0/740565695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7368074010 0x7f7368074490 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 -- 172.21.15.7:0/740565695 >> 172.21.15.7:0/740565695 conn(0x7f73680fcb60 msgr2=0x7f73680fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 -- 172.21.15.7:0/740565695 shutdown_connections 2024-01-11T18:03:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.520+0000 7f736e1ff700 1 -- 172.21.15.7:0/740565695 wait complete. 2024-01-11T18:03:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.522+0000 7f736e1ff700 1 Processor -- start 2024-01-11T18:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.522+0000 7f736e1ff700 1 -- start start 2024-01-11T18:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f736e1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7368074010 0x7f7368199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f736e1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 0x7f7368199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f736e1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73681094a0 0x7f73681a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f736e1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681a12a0 con 0x7f73680755f0 2024-01-11T18:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f736e1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681a13e0 con 0x7f73681094a0 2024-01-11T18:03:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f736e1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73681a1570 con 0x7f7368074010 2024-01-11T18:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.523+0000 7f7366ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 0x7f7368199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 0x7f7368199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34096/0 (socket says 172.21.15.7:34096) 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 -- 172.21.15.7:0/2288902038 learned_addr learned my addr 172.21.15.7:0/2288902038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7367fff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73681094a0 0x7f73681a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f73677fe700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7368074010 0x7f7368199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 -- 172.21.15.7:0/2288902038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7368074010 msgr2=0x7f7368199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7368074010 0x7f7368199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 -- 172.21.15.7:0/2288902038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73681094a0 msgr2=0x7f73681a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73681094a0 0x7f73681a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.524+0000 7f7366ffd700 1 -- 172.21.15.7:0/2288902038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c009520 con 0x7f73680755f0 2024-01-11T18:03:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.525+0000 7f7366ffd700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 0x7f7368199c60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f735c000c00 tx=0x7f735c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.525+0000 7f7364ff9700 1 -- 172.21.15.7:0/2288902038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c01c070 con 0x7f73680755f0 2024-01-11T18:03:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.525+0000 7f7364ff9700 1 -- 172.21.15.7:0/2288902038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f735c0054f0 con 0x7f73680755f0 2024-01-11T18:03:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.525+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73681a17f0 con 0x7f73680755f0 2024-01-11T18:03:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.525+0000 7f7364ff9700 1 -- 172.21.15.7:0/2288902038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c020410 con 0x7f73680755f0 2024-01-11T18:03:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.526+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73681a1c70 con 0x7f73680755f0 2024-01-11T18:03:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.534+0000 7f7381f26b80 1 Processor -- start 2024-01-11T18:03:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.535+0000 7f7381f26b80 1 -- start start 2024-01-11T18:03:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.535+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed893340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed894090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed89ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7381f26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed78a030 con 0x55f5ed8945d0 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7381f26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed7edac0 con 0x55f5ed893c10 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7381f26b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed7eb3e0 con 0x55f5ed892f20 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7367fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed893340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7367fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed893340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55238/0 (socket says 172.21.15.7:55238) 2024-01-11T18:03:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.538+0000 7f7367fff700 1 -- 172.21.15.7:0/864547976 learned_addr learned my addr 172.21.15.7:0/864547976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.539+0000 7f7364ff9700 1 -- 172.21.15.7:0/2288902038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f735c0046d0 con 0x7f73680755f0 2024-01-11T18:03:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.541+0000 7f7366ffd700 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed89ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7364ff9700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f73500a1f50 0x7f73500a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7364ff9700 1 -- 172.21.15.7:0/2288902038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f735c0cee80 con 0x7f73680755f0 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7366ffd700 1 -- 172.21.15.7:0/864547976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 msgr2=0x55f5ed893340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7366ffd700 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed893340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7366ffd700 1 -- 172.21.15.7:0/864547976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 msgr2=0x55f5ed894090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7366ffd700 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed894090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.542+0000 7f7366ffd700 1 -- 172.21.15.7:0/864547976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5ed65b350 con 0x55f5ed8945d0 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.543+0000 7f7366ffd700 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed89ae60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f735c0cc5c0 tx=0x7f735c00f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb518b537fe161ae server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.543+0000 7f734dffb700 1 -- 172.21.15.7:0/864547976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c005660 con 0x55f5ed8945d0 2024-01-11T18:03:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.547+0000 7f734dffb700 1 -- 172.21.15.7:0/864547976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f735c09b8e0 con 0x55f5ed8945d0 2024-01-11T18:03:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.547+0000 7f7367fff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f73500a1f50 0x7f73500a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.548+0000 7f7381f26b80 1 -- 172.21.15.7:0/864547976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 msgr2=0x55f5ed89ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.548+0000 7f734dffb700 1 -- 172.21.15.7:0/864547976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c0d23d0 con 0x55f5ed8945d0 2024-01-11T18:03:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.548+0000 7f7381f26b80 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed89ae60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f735c0cc5c0 tx=0x7f735c00f9e0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.550+0000 7f7381f26b80 1 -- 172.21.15.7:0/864547976 shutdown_connections 2024-01-11T18:03:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.550+0000 7f7381f26b80 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed893340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.550+0000 7f7381f26b80 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed89ae60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.550+0000 7f7381f26b80 1 --2- 172.21.15.7:0/864547976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed894090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.551+0000 7f7367fff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f73500a1f50 0x7f73500a4410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55f5ed892cc0 tx=0x7f736000a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.552+0000 7f7381f26b80 1 -- 172.21.15.7:0/864547976 >> 172.21.15.7:0/864547976 conn(0x55f5ed808740 msgr2=0x55f5ed88d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.552+0000 7f7381f26b80 1 -- 172.21.15.7:0/864547976 shutdown_connections 2024-01-11T18:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.552+0000 7f7381f26b80 1 -- 172.21.15.7:0/864547976 wait complete. 2024-01-11T18:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.553+0000 7f7381f26b80 1 -- start start 2024-01-11T18:03:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.553+0000 7f7381f26b80 1 Processor -- start 2024-01-11T18:03:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.556+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 0x55f5ed89c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f73677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 0x55f5ed89c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7381f26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed7e6cd0 con 0x55f5ed892f20 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7381f26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed755340 con 0x55f5ed893c10 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7381f26b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed754fe0 con 0x55f5ed8945d0 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f73677fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 0x55f5ed89c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34118/0 (socket says 172.21.15.7:34118) 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f73677fe700 1 -- 172.21.15.7:0/3584808633 learned_addr learned my addr 172.21.15.7:0/3584808633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7366ffd700 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.557+0000 7f7381f26b80 10 client.0 _ll_register_callbacks cb 0x55f5ed7876c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.558+0000 7f73677fe700 1 -- 172.21.15.7:0/3584808633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed8945d0 msgr2=0x55f5ed8a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.558+0000 7f73677fe700 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.558+0000 7f73677fe700 1 -- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 msgr2=0x55f5ed89c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.558+0000 7f73677fe700 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.558+0000 7f73677fe700 1 -- 172.21.15.7:0/3584808633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73681a1c70 con 0x55f5ed892f20 2024-01-11T18:03:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.559+0000 7f73677fe700 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 0x55f5ed89c3b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f735800dd80 tx=0x7f735800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada97127e7e4ece3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.559+0000 7f7346ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:03:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.559+0000 7f7346ffd700 1 -- 172.21.15.7:0/3584808633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358004230 con 0x55f5ed892f20 2024-01-11T18:03:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.559+0000 7f7346ffd700 1 -- 172.21.15.7:0/3584808633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f735800fe60 con 0x55f5ed892f20 2024-01-11T18:03:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.560+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5ed885d60 con 0x55f5ed892f20 2024-01-11T18:03:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.560+0000 7f7346ffd700 1 -- 172.21.15.7:0/3584808633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f735800ded0 con 0x55f5ed892f20 2024-01-11T18:03:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.560+0000 7f7381f26b80 10 client.15942 fetch_fsmap learned FSMap version 34 2024-01-11T18:03:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.560+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f5ed607b00 con 0x55f5ed892f20 2024-01-11T18:03:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.561+0000 7f7346ffd700 1 -- 172.21.15.7:0/3584808633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5ed885d60 con 0x55f5ed892f20 2024-01-11T18:03:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.561+0000 7f7346ffd700 1 -- 172.21.15.7:0/3584808633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f735801b600 con 0x55f5ed892f20 2024-01-11T18:03:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.561+0000 7f7381f26b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T18:03:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.562+0000 7f7381f26b80 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.562+0000 7f7381f26b80 20 client.15942 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.562+0000 7f7381f26b80 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed8a8260 0x55f5ed8aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.562+0000 7f7381f26b80 4 client.15942 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:03:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.562+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5ed7dfb60 con 0x55f5ed8a8260 2024-01-11T18:03:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.564+0000 7f7366ffd700 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed8a8260 0x55f5ed8aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.577+0000 7f7366ffd700 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed8a8260 0x55f5ed8aa680 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f5f3f19eefd0c2 server_cookie=55c9b47803a8f148 in_seq=0 out_seq=0 2024-01-11T18:03:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.577+0000 7f7346ffd700 10 client.15942 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.585+0000 7f7346ffd700 1 -- 172.21.15.7:0/3584808633 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5ed7dfb60 con 0x55f5ed8a8260 2024-01-11T18:03:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.585+0000 7f7346ffd700 10 client.15942 handle_command_reply: tid=0 2024-01-11T18:03:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.585+0000 7f7381f26b80 1 client.15942 shutdown 2024-01-11T18:03:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 msgr2=0x55f5ed89c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 0x55f5ed89c3b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f735800dd80 tx=0x7f735800f640 comp rx=0 tx=0).stop 2024-01-11T18:03:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 shutdown_connections 2024-01-11T18:03:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed8a8260 0x55f5ed8aa680 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed892f20 0x55f5ed89c3b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7346ffd700 0 client.15942 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 --2- 172.21.15.7:0/3584808633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 >> 172.21.15.7:0/3584808633 conn(0x55f5ed808740 msgr2=0x55f5ed88cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.586+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 shutdown_connections 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.587+0000 7f7381f26b80 1 -- 172.21.15.7:0/3584808633 wait complete. 2024-01-11T18:03:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.587+0000 7f7381f26b80 20 client.15942 trim_cache size 0 max 16384 2024-01-11T18:03:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.617+0000 7f7381f26b80 1 Processor -- start 2024-01-11T18:03:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.618+0000 7f7381f26b80 1 -- start start 2024-01-11T18:03:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.618+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed892f20 0x55f5ed897340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.618+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed893c10 0x55f5ed897880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.618+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.618+0000 7f7381f26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed65d640 con 0x55f5ed8945d0 2024-01-11T18:03:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.619+0000 7f7381f26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed88f690 con 0x55f5ed892f20 2024-01-11T18:03:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.619+0000 7f7381f26b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed88a560 con 0x55f5ed893c10 2024-01-11T18:03:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.619+0000 7f7367fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.619+0000 7f7367fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34126/0 (socket says 172.21.15.7:34126) 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.619+0000 7f7367fff700 1 -- 172.21.15.7:0/1790664389 learned_addr learned my addr 172.21.15.7:0/1790664389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.619+0000 7f73677fe700 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed892f20 0x55f5ed897340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.620+0000 7f7366ffd700 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed893c10 0x55f5ed897880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.620+0000 7f73677fe700 1 -- 172.21.15.7:0/1790664389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed893c10 msgr2=0x55f5ed897880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.620+0000 7f73677fe700 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed893c10 0x55f5ed897880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.620+0000 7f73677fe700 1 -- 172.21.15.7:0/1790664389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 msgr2=0x55f5ed8a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.620+0000 7f73677fe700 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.621+0000 7f73677fe700 1 -- 172.21.15.7:0/1790664389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5ed7a81d0 con 0x55f5ed892f20 2024-01-11T18:03:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.621+0000 7f73677fe700 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed892f20 0x55f5ed897340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f735801ba60 tx=0x7f735800fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ce3fe64451c219 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.626+0000 7f734d7fa700 1 -- 172.21.15.7:0/1790664389 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735802a070 con 0x55f5ed892f20 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.626+0000 7f734d7fa700 1 -- 172.21.15.7:0/1790664389 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f735800ded0 con 0x55f5ed892f20 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.626+0000 7f734d7fa700 1 -- 172.21.15.7:0/1790664389 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358005630 con 0x55f5ed892f20 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 -- 172.21.15.7:0/1790664389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed892f20 msgr2=0x55f5ed897340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed892f20 0x55f5ed897340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f735801ba60 tx=0x7f735800fb30 comp rx=0 tx=0).stop 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 -- 172.21.15.7:0/1790664389 shutdown_connections 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed892f20 0x55f5ed897340 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 --2- 172.21.15.7:0/1790664389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed893c10 0x55f5ed897880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 -- 172.21.15.7:0/1790664389 >> 172.21.15.7:0/1790664389 conn(0x55f5ed808740 msgr2=0x55f5ed89b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 -- 172.21.15.7:0/1790664389 shutdown_connections 2024-01-11T18:03:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.627+0000 7f7381f26b80 1 -- 172.21.15.7:0/1790664389 wait complete. 2024-01-11T18:03:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.628+0000 7f7381f26b80 1 -- start start 2024-01-11T18:03:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.628+0000 7f7381f26b80 1 Processor -- start 2024-01-11T18:03:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.629+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed89fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7381f26b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f73677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed89fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f73677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed89fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:55262/0 (socket says 172.21.15.7:55262) 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7381f26b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed8a3500 con 0x55f5ed8945d0 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7381f26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed8a3670 con 0x55f5ed893c10 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7381f26b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ed8a3830 con 0x55f5ed892f20 2024-01-11T18:03:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7366ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f7381f26b80 10 client.0 _ll_register_callbacks cb 0x55f5ed7876c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.630+0000 7f73677fe700 1 -- 172.21.15.7:0/32304177 learned_addr learned my addr 172.21.15.7:0/32304177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7367fff700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7366ffd700 1 -- 172.21.15.7:0/32304177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 msgr2=0x55f5ed89fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7366ffd700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed89fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7366ffd700 1 -- 172.21.15.7:0/32304177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 msgr2=0x55f5ed8a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7366ffd700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7366ffd700 1 -- 172.21.15.7:0/32304177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5ed607b00 con 0x55f5ed893c10 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.631+0000 7f7367fff700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.632+0000 7f7346ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:03:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.632+0000 7f7366ffd700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89ffa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f735c09fbf0 tx=0x7f735c09b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9835650c522702c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.633+0000 7f7346ffd700 1 -- 172.21.15.7:0/32304177 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c099a20 con 0x55f5ed893c10 2024-01-11T18:03:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.633+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f735c09b8e0 con 0x55f5ed893c10 2024-01-11T18:03:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.633+0000 7f7346ffd700 1 -- 172.21.15.7:0/32304177 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f735c005660 con 0x55f5ed893c10 2024-01-11T18:03:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.633+0000 7f7346ffd700 1 -- 172.21.15.7:0/32304177 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f735c099a20 con 0x55f5ed893c10 2024-01-11T18:03:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.633+0000 7f7381f26b80 10 client.26042 fetch_fsmap learned FSMap version 34 2024-01-11T18:03:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.633+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f5ed65fc70 con 0x55f5ed893c10 2024-01-11T18:03:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7346ffd700 1 -- 172.21.15.7:0/32304177 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f735c09b8e0 con 0x55f5ed893c10 2024-01-11T18:03:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7346ffd700 1 -- 172.21.15.7:0/32304177 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f735c09aef0 con 0x55f5ed893c10 2024-01-11T18:03:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7381f26b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T18:03:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7381f26b80 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7381f26b80 20 client.26042 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7381f26b80 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed907eb0 0x55f5ed90a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7381f26b80 4 client.26042 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:03:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.634+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f5ed7df880 con 0x55f5ed907eb0 2024-01-11T18:03:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.636+0000 7f73677fe700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed907eb0 0x55f5ed90a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.637+0000 7f73677fe700 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed907eb0 0x55f5ed90a2d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9187f4cba3934625 server_cookie=e17f5602d7d5dda9 in_seq=0 out_seq=0 2024-01-11T18:03:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.637+0000 7f7346ffd700 10 client.26042 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.680+0000 7f7346ffd700 1 -- 172.21.15.7:0/32304177 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f5ed7df880 con 0x55f5ed907eb0 2024-01-11T18:03:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.680+0000 7f7346ffd700 10 client.26042 handle_command_reply: tid=0 2024-01-11T18:03:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 client.26042 shutdown 2024-01-11T18:03:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 msgr2=0x55f5ed89ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89ffa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f735c09fbf0 tx=0x7f735c09b340 comp rx=0 tx=0).stop 2024-01-11T18:03:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 shutdown_connections 2024-01-11T18:03:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f5ed907eb0 0x55f5ed90a2d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f5ed892f20 0x55f5ed89fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5ed8945d0 0x55f5ed8a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 --2- 172.21.15.7:0/32304177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5ed893c10 0x55f5ed89ffa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 >> 172.21.15.7:0/32304177 conn(0x55f5ed808740 msgr2=0x55f5ed89c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.681+0000 7f7346ffd700 0 client.26042 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.683+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 shutdown_connections 2024-01-11T18:03:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.683+0000 7f7381f26b80 1 -- 172.21.15.7:0/32304177 wait complete. 2024-01-11T18:03:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.683+0000 7f7381f26b80 20 client.26042 trim_cache size 0 max 16384 2024-01-11T18:03:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:03:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:03:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:03:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:03:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.684+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f73500a1f50 msgr2=0x7f73500a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.684+0000 7f736e1ff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f73500a1f50 0x7f73500a4410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55f5ed892cc0 tx=0x7f736000a040 comp rx=0 tx=0).stop 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.684+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 msgr2=0x7f7368199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.684+0000 7f736e1ff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 0x7f7368199c60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f735c000c00 tx=0x7f735c00b8f0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 shutdown_connections 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f73500a1f50 0x7f73500a4410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7368074010 0x7f7368199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73681094a0 0x7f73681a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 --2- 172.21.15.7:0/2288902038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680755f0 0x7f7368199c60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 >> 172.21.15.7:0/2288902038 conn(0x7f73680fcb60 msgr2=0x7f73681053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 shutdown_connections 2024-01-11T18:03:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:33.685+0000 7f736e1ff700 1 -- 172.21.15.7:0/2288902038 wait complete. 2024-01-11T18:03:34.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 305.0 s, 594.9 tps, lat 166.434 ms stddev 206.719 2024-01-11T18:03:35.991 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:35 smithi016 ceph-mon[43195]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 601 op/s 2024-01-11T18:03:36.026 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:35 smithi038 ceph-mon[43582]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 601 op/s 2024-01-11T18:03:36.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:35 smithi007 ceph-mon[33361]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 601 op/s 2024-01-11T18:03:38.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:37 smithi016 ceph-mon[43195]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 615 op/s 2024-01-11T18:03:38.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:37 smithi038 ceph-mon[43582]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 615 op/s 2024-01-11T18:03:38.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:37 smithi007 ceph-mon[33361]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 615 op/s 2024-01-11T18:03:39.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 310.0 s, 506.0 tps, lat 192.030 ms stddev 245.130 2024-01-11T18:03:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:39 smithi016 ceph-mon[43195]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 568 op/s 2024-01-11T18:03:40.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:39 smithi038 ceph-mon[43582]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 568 op/s 2024-01-11T18:03:40.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:39 smithi007 ceph-mon[33361]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 568 op/s 2024-01-11T18:03:41.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:40 smithi016 ceph-mon[43195]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 657 op/s 2024-01-11T18:03:41.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:40 smithi038 ceph-mon[43582]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 657 op/s 2024-01-11T18:03:41.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:40 smithi007 ceph-mon[33361]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 MiB/s wr, 657 op/s 2024-01-11T18:03:44.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 315.0 s, 519.2 tps, lat 194.114 ms stddev 282.386 2024-01-11T18:03:44.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:43 smithi016 ceph-mon[43195]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 620 op/s 2024-01-11T18:03:44.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:43 smithi038 ceph-mon[43582]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 620 op/s 2024-01-11T18:03:44.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:43 smithi007 ceph-mon[33361]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 620 op/s 2024-01-11T18:03:45.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:45 smithi016 ceph-mon[43195]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 MiB/s wr, 646 op/s 2024-01-11T18:03:46.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:45 smithi038 ceph-mon[43582]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 MiB/s wr, 646 op/s 2024-01-11T18:03:46.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:45 smithi007 ceph-mon[33361]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 MiB/s wr, 646 op/s 2024-01-11T18:03:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 -- 172.21.15.7:0/2540634220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 msgr2=0x7fc7941019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/2540634220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941019d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00b0d0 tx=0x7fc78c00b3e0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 -- 172.21.15.7:0/2540634220 shutdown_connections 2024-01-11T18:03:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/2540634220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc794103140 0x7fc7941099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/2540634220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941019d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/2540634220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7941027a0 0x7fc794102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.895+0000 7fc79c1ff700 1 -- 172.21.15.7:0/2540634220 >> 172.21.15.7:0/2540634220 conn(0x7fc7940fcb00 msgr2=0x7fc7940fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.896+0000 7fc79c1ff700 1 -- 172.21.15.7:0/2540634220 shutdown_connections 2024-01-11T18:03:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.896+0000 7fc79c1ff700 1 -- 172.21.15.7:0/2540634220 wait complete. 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.897+0000 7fc79c1ff700 1 Processor -- start 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.897+0000 7fc79c1ff700 1 -- start start 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.898+0000 7fc79c1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.898+0000 7fc79c1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 0x7fc79419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.898+0000 7fc79c1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794103140 0x7fc794195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.898+0000 7fc79c1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941ae030 con 0x7fc794103140 2024-01-11T18:03:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.898+0000 7fc79c1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941ae170 con 0x7fc7941015b0 2024-01-11T18:03:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.898+0000 7fc79c1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941ae330 con 0x7fc7941027a0 2024-01-11T18:03:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.899+0000 7fc79979a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 0x7fc79419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.899+0000 7fc79979a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 0x7fc79419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42478/0 (socket says 172.21.15.7:42478) 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.899+0000 7fc79979a700 1 -- 172.21.15.7:0/4155060254 learned_addr learned my addr 172.21.15.7:0/4155060254 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.899+0000 7fc799f9b700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.900+0000 7fc79979a700 1 -- 172.21.15.7:0/4155060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 msgr2=0x7fc7941954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.900+0000 7fc79979a700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.900+0000 7fc79979a700 1 -- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794103140 msgr2=0x7fc794195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.900+0000 7fc79979a700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794103140 0x7fc794195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.900+0000 7fc79979a700 1 -- 172.21.15.7:0/4155060254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c009d00 con 0x7fc7941027a0 2024-01-11T18:03:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.900+0000 7fc799f9b700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.901+0000 7fc79979a700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 0x7fc79419aa60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc79000b580 tx=0x7fc79000b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.902+0000 7fc786ffd700 1 -- 172.21.15.7:0/4155060254 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790011830 con 0x7fc7941027a0 2024-01-11T18:03:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.902+0000 7fc786ffd700 1 -- 172.21.15.7:0/4155060254 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc790011e50 con 0x7fc7941027a0 2024-01-11T18:03:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.902+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941ae610 con 0x7fc7941027a0 2024-01-11T18:03:46.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.902+0000 7fc786ffd700 1 -- 172.21.15.7:0/4155060254 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000f2e0 con 0x7fc7941027a0 2024-01-11T18:03:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.902+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7941aeba0 con 0x7fc7941027a0 2024-01-11T18:03:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.907+0000 7fc786ffd700 1 -- 172.21.15.7:0/4155060254 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc7900105f0 con 0x7fc7941027a0 2024-01-11T18:03:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.910+0000 7fc7afe72b80 1 Processor -- start 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.911+0000 7fc7afe72b80 1 -- start start 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.911+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.911+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 0x555d11162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.911+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d111625d0 0x555d11168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.911+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d11058030 con 0x555d11160f20 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.911+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d110bbac0 con 0x555d111625d0 2024-01-11T18:03:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.912+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d110b93e0 con 0x555d11161c10 2024-01-11T18:03:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.912+0000 7fc799f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 0x555d11162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.912+0000 7fc799f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 0x555d11162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42480/0 (socket says 172.21.15.7:42480) 2024-01-11T18:03:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.912+0000 7fc799f9b700 1 -- 172.21.15.7:0/110977596 learned_addr learned my addr 172.21.15.7:0/110977596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.913+0000 7fc79979a700 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d111625d0 0x555d11168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.913+0000 7fc786ffd700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7800a2060 0x7fc7800a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.913+0000 7fc79a79c700 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.913+0000 7fc786ffd700 1 -- 172.21.15.7:0/4155060254 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc7900cd690 con 0x7fc7941027a0 2024-01-11T18:03:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.914+0000 7fc799f9b700 1 -- 172.21.15.7:0/110977596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d111625d0 msgr2=0x555d11168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.914+0000 7fc799f9b700 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d111625d0 0x555d11168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.914+0000 7fc799f9b700 1 -- 172.21.15.7:0/110977596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 msgr2=0x555d11161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.914+0000 7fc799f9b700 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.914+0000 7fc799f9b700 1 -- 172.21.15.7:0/110977596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d10f29350 con 0x555d11161c10 2024-01-11T18:03:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.914+0000 7fc799f9b700 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 0x555d11162090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00b0d0 tx=0x7fc78c005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c774b6b4617424 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.915+0000 7fc77ffff700 1 -- 172.21.15.7:0/110977596 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00e040 con 0x555d11161c10 2024-01-11T18:03:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.915+0000 7fc77ffff700 1 -- 172.21.15.7:0/110977596 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc78c007910 con 0x555d11161c10 2024-01-11T18:03:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.915+0000 7fc77ffff700 1 -- 172.21.15.7:0/110977596 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c012760 con 0x555d11161c10 2024-01-11T18:03:46.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.915+0000 7fc79a79c700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7800a2060 0x7fc7800a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.916+0000 7fc7afe72b80 1 -- 172.21.15.7:0/110977596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 msgr2=0x555d11162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.916+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 0x555d11162090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00b0d0 tx=0x7fc78c005b30 comp rx=0 tx=0).stop 2024-01-11T18:03:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 -- 172.21.15.7:0/110977596 shutdown_connections 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d111625d0 0x555d11168e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/110977596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11161c10 0x555d11162090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 -- 172.21.15.7:0/110977596 >> 172.21.15.7:0/110977596 conn(0x555d110d6740 msgr2=0x555d1115b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc79a79c700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7800a2060 0x7fc7800a4520 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x555d11160cc0 tx=0x7fc788008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 -- 172.21.15.7:0/110977596 shutdown_connections 2024-01-11T18:03:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.917+0000 7fc7afe72b80 1 -- 172.21.15.7:0/110977596 wait complete. 2024-01-11T18:03:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.918+0000 7fc7afe72b80 1 -- start start 2024-01-11T18:03:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.918+0000 7fc7afe72b80 1 Processor -- start 2024-01-11T18:03:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d1116a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d11171e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d110b4cd0 con 0x555d11160f20 2024-01-11T18:03:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d11023340 con 0x555d11161c10 2024-01-11T18:03:46.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d11022fe0 con 0x555d111625d0 2024-01-11T18:03:46.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc7afe72b80 10 client.0 _ll_register_callbacks cb 0x555d110556c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:46.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.919+0000 7fc79979a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52808/0 (socket says 172.21.15.7:52808) 2024-01-11T18:03:46.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 -- 172.21.15.7:0/3539865445 learned_addr learned my addr 172.21.15.7:0/3539865445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 -- 172.21.15.7:0/3539865445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 msgr2=0x555d11171e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:03:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d11171e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 -- 172.21.15.7:0/3539865445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 msgr2=0x555d1116a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d1116a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.920+0000 7fc79979a700 1 -- 172.21.15.7:0/3539865445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79410a340 con 0x555d11161c10 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.921+0000 7fc79979a700 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116ae60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7900965c0 tx=0x7fc790010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608509e47a863327 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.921+0000 7fc77d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.921+0000 7fc77d7fa700 1 -- 172.21.15.7:0/3539865445 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900adea0 con 0x555d11161c10 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.921+0000 7fc77d7fa700 1 -- 172.21.15.7:0/3539865445 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc79009e210 con 0x555d11161c10 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.922+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d11153d60 con 0x555d11161c10 2024-01-11T18:03:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.922+0000 7fc77d7fa700 1 -- 172.21.15.7:0/3539865445 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7900a2450 con 0x555d11161c10 2024-01-11T18:03:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.922+0000 7fc7afe72b80 10 client.26054 fetch_fsmap learned FSMap version 34 2024-01-11T18:03:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.922+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555d10ed5b00 con 0x555d11161c10 2024-01-11T18:03:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc77d7fa700 1 -- 172.21.15.7:0/3539865445 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d11153d60 con 0x555d11161c10 2024-01-11T18:03:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc77d7fa700 1 -- 172.21.15.7:0/3539865445 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc7900a1fc0 con 0x555d11161c10 2024-01-11T18:03:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc7afe72b80 10 client.26054 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc7afe72b80 10 client.26054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc7afe72b80 20 client.26054 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d11176770 0x555d11178b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc7afe72b80 4 client.26054 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.923+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d110adb60 con 0x555d11176770 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.924+0000 7fc799f9b700 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d11176770 0x555d11178b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.925+0000 7fc77d7fa700 10 client.26054 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.925+0000 7fc799f9b700 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d11176770 0x555d11178b90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0bcfdcbcc848a0b server_cookie=dcd861d690a0c02c in_seq=0 out_seq=0 2024-01-11T18:03:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.927+0000 7fc77d7fa700 1 -- 172.21.15.7:0/3539865445 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d110adb60 con 0x555d11176770 2024-01-11T18:03:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.927+0000 7fc77d7fa700 10 client.26054 handle_command_reply: tid=0 2024-01-11T18:03:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.927+0000 7fc7afe72b80 1 client.26054 shutdown 2024-01-11T18:03:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.928+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 msgr2=0x555d1116ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:46.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.928+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116ae60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7900965c0 tx=0x7fc790010430 comp rx=0 tx=0).stop 2024-01-11T18:03:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 shutdown_connections 2024-01-11T18:03:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d11176770 0x555d11178b90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d11171e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d1116a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/3539865445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116ae60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 >> 172.21.15.7:0/3539865445 conn(0x555d110d6740 msgr2=0x555d1115af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.929+0000 7fc77d7fa700 0 client.26054 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.930+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 shutdown_connections 2024-01-11T18:03:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.930+0000 7fc7afe72b80 1 -- 172.21.15.7:0/3539865445 wait complete. 2024-01-11T18:03:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.931+0000 7fc7afe72b80 20 client.26054 trim_cache size 0 max 16384 2024-01-11T18:03:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.969+0000 7fc7afe72b80 1 Processor -- start 2024-01-11T18:03:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.969+0000 7fc7afe72b80 1 -- start start 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11178780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d11171e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d1117d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d10f2b640 con 0x555d11160f20 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1115d310 con 0x555d11161c10 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d11173c00 con 0x555d111625d0 2024-01-11T18:03:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.970+0000 7fc79a79c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d1117d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11178780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11178780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52822/0 (socket says 172.21.15.7:52822) 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 -- 172.21.15.7:0/383637036 learned_addr learned my addr 172.21.15.7:0/383637036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc79a79c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d1117d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42498/0 (socket says 172.21.15.7:42498) 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc79979a700 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d11171e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 -- 172.21.15.7:0/383637036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 msgr2=0x555d1117d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d1117d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 -- 172.21.15.7:0/383637036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 msgr2=0x555d11171e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d11171e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc799f9b700 1 -- 172.21.15.7:0/383637036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d110761d0 con 0x555d11160f20 2024-01-11T18:03:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.971+0000 7fc79979a700 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d11171e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:03:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.973+0000 7fc799f9b700 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11178780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x555d1115d7f0 tx=0x7fc78c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5569ee8565aea14d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.975+0000 7fc77f7fe700 1 -- 172.21.15.7:0/383637036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00e050 con 0x555d11160f20 2024-01-11T18:03:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.975+0000 7fc77f7fe700 1 -- 172.21.15.7:0/383637036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc78c00f040 con 0x555d11160f20 2024-01-11T18:03:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.975+0000 7fc77f7fe700 1 -- 172.21.15.7:0/383637036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c005330 con 0x555d11160f20 2024-01-11T18:03:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 -- 172.21.15.7:0/383637036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 msgr2=0x555d11178780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11178780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x555d1115d7f0 tx=0x7fc78c00b7e0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 -- 172.21.15.7:0/383637036 shutdown_connections 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d111625d0 0x555d1117d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d11160f20 0x555d11178780 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/383637036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d11171e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 -- 172.21.15.7:0/383637036 >> 172.21.15.7:0/383637036 conn(0x555d110d6740 msgr2=0x555d11169ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.976+0000 7fc7afe72b80 1 -- 172.21.15.7:0/383637036 shutdown_connections 2024-01-11T18:03:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.977+0000 7fc7afe72b80 1 -- 172.21.15.7:0/383637036 wait complete. 2024-01-11T18:03:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.977+0000 7fc7afe72b80 1 -- start start 2024-01-11T18:03:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.977+0000 7fc7afe72b80 1 Processor -- start 2024-01-11T18:03:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.978+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11160f20 0x555d1116eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.978+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.978+0000 7fc7afe72b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d111625d0 0x555d111cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.978+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d111cb1b0 con 0x555d111625d0 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.978+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d111cb2f0 con 0x555d11161c10 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.978+0000 7fc7afe72b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d111cb430 con 0x555d11160f20 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc7afe72b80 10 client.0 _ll_register_callbacks cb 0x555d110556c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc799f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11160f20 0x555d1116eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc799f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11160f20 0x555d1116eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42512/0 (socket says 172.21.15.7:42512) 2024-01-11T18:03:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc799f9b700 1 -- 172.21.15.7:0/4056725287 learned_addr learned my addr 172.21.15.7:0/4056725287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79979a700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79a79c700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d111625d0 0x555d111cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79979a700 1 -- 172.21.15.7:0/4056725287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11160f20 msgr2=0x555d1116eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79979a700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11160f20 0x555d1116eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79979a700 1 -- 172.21.15.7:0/4056725287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d111625d0 msgr2=0x555d111cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79979a700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d111625d0 0x555d111cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.979+0000 7fc79979a700 1 -- 172.21.15.7:0/4056725287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d10f29350 con 0x555d11161c10 2024-01-11T18:03:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.980+0000 7fc79979a700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116f400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc790099dc0 tx=0x7fc790098670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3b265861babc85 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:03:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.980+0000 7fc77d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.980+0000 7fc77d7fa700 1 -- 172.21.15.7:0/4056725287 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900adea0 con 0x555d11161c10 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.981+0000 7fc77d7fa700 1 -- 172.21.15.7:0/4056725287 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc79009e210 con 0x555d11161c10 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.981+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc78c007910 con 0x555d11161c10 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.981+0000 7fc77d7fa700 1 -- 172.21.15.7:0/4056725287 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc790096400 con 0x555d11161c10 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.981+0000 7fc7afe72b80 10 client.26057 fetch_fsmap learned FSMap version 34 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.981+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555d10f2dc70 con 0x555d11161c10 2024-01-11T18:03:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.981+0000 7fc77d7fa700 1 -- 172.21.15.7:0/4056725287 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc78c007910 con 0x555d11161c10 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc77d7fa700 1 -- 172.21.15.7:0/4056725287 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc790011990 con 0x555d11161c10 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc7afe72b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc7afe72b80 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc7afe72b80 20 client.26057 populate_metadata read hostname 'smithi007' 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d111cca90 0x555d111ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc7afe72b80 4 client.26057 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:03:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.983+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d110ad880 con 0x555d111cca90 2024-01-11T18:03:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.984+0000 7fc799f9b700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d111cca90 0x555d111ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:03:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.985+0000 7fc799f9b700 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d111cca90 0x555d111ceeb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc27bf7abab4002 server_cookie=704df473daa1c0f9 in_seq=0 out_seq=0 2024-01-11T18:03:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:46.985+0000 7fc77d7fa700 10 client.26057 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.021+0000 7fc77d7fa700 1 -- 172.21.15.7:0/4056725287 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d110ad880 con 0x555d111cca90 2024-01-11T18:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.021+0000 7fc77d7fa700 10 client.26057 handle_command_reply: tid=0 2024-01-11T18:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.022+0000 7fc7afe72b80 1 client.26057 shutdown 2024-01-11T18:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 msgr2=0x555d1116f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116f400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc790099dc0 tx=0x7fc790098670 comp rx=0 tx=0).stop 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 shutdown_connections 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555d111cca90 0x555d111ceeb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555d11160f20 0x555d1116eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d111625d0 0x555d111cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 --2- 172.21.15.7:0/4056725287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d11161c10 0x555d1116f400 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 >> 172.21.15.7:0/4056725287 conn(0x555d110d6740 msgr2=0x555d11169e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.023+0000 7fc77d7fa700 0 client.26057 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.024+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 shutdown_connections 2024-01-11T18:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.024+0000 7fc7afe72b80 1 -- 172.21.15.7:0/4056725287 wait complete. 2024-01-11T18:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.024+0000 7fc7afe72b80 20 client.26057 trim_cache size 0 max 16384 2024-01-11T18:03:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:03:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:03:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:03:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7800a2060 msgr2=0x7fc7800a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7800a2060 0x7fc7800a4520 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x555d11160cc0 tx=0x7fc788008040 comp rx=0 tx=0).stop 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 msgr2=0x7fc79419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 0x7fc79419aa60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc79000b580 tx=0x7fc79000b890 comp rx=0 tx=0).stop 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 shutdown_connections 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc7800a2060 0x7fc7800a4520 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794103140 0x7fc794195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7941015b0 0x7fc7941954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 --2- 172.21.15.7:0/4155060254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7941027a0 0x7fc79419aa60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.025+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 >> 172.21.15.7:0/4155060254 conn(0x7fc7940fcb00 msgr2=0x7fc7941039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.026+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 shutdown_connections 2024-01-11T18:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:03:47.026+0000 7fc79c1ff700 1 -- 172.21.15.7:0/4155060254 wait complete. 2024-01-11T18:03:48.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:47 smithi016 ceph-mon[43195]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.7 MiB/s wr, 621 op/s 2024-01-11T18:03:48.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:47 smithi038 ceph-mon[43582]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.7 MiB/s wr, 621 op/s 2024-01-11T18:03:48.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:47 smithi007 ceph-mon[33361]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.7 MiB/s wr, 621 op/s 2024-01-11T18:03:49.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 320.0 s, 624.2 tps, lat 167.353 ms stddev 223.620 2024-01-11T18:03:50.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:49 smithi016 ceph-mon[43195]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.4 MiB/s wr, 582 op/s 2024-01-11T18:03:50.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:49 smithi038 ceph-mon[43582]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.4 MiB/s wr, 582 op/s 2024-01-11T18:03:50.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:49 smithi007 ceph-mon[33361]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.4 MiB/s wr, 582 op/s 2024-01-11T18:03:51.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:50 smithi016 ceph-mon[43195]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 632 op/s 2024-01-11T18:03:51.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:50 smithi038 ceph-mon[43582]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 632 op/s 2024-01-11T18:03:51.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:50 smithi007 ceph-mon[33361]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 632 op/s 2024-01-11T18:03:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 325.0 s, 711.8 tps, lat 138.636 ms stddev 159.620 2024-01-11T18:03:54.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:53 smithi016 ceph-mon[43195]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.5 MiB/s wr, 543 op/s 2024-01-11T18:03:54.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:53 smithi038 ceph-mon[43582]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.5 MiB/s wr, 543 op/s 2024-01-11T18:03:54.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:53 smithi007 ceph-mon[33361]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.5 MiB/s wr, 543 op/s 2024-01-11T18:03:55.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:54 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:55.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:54 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:55.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:54 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:03:56.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:55 smithi016 ceph-mon[43195]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 535 op/s 2024-01-11T18:03:56.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:55 smithi038 ceph-mon[43582]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 535 op/s 2024-01-11T18:03:56.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:55 smithi007 ceph-mon[33361]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 535 op/s 2024-01-11T18:03:57.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:56 smithi016 ceph-mon[43195]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 531 op/s 2024-01-11T18:03:57.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:56 smithi038 ceph-mon[43582]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 531 op/s 2024-01-11T18:03:57.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:56 smithi007 ceph-mon[33361]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 531 op/s 2024-01-11T18:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 330.0 s, 649.0 tps, lat 153.950 ms stddev 187.618 2024-01-11T18:04:00.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:03:59 smithi016 ceph-mon[43195]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 495 op/s 2024-01-11T18:04:00.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:03:59 smithi038 ceph-mon[43582]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 495 op/s 2024-01-11T18:04:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 -- 172.21.15.7:0/4016799556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198074070 msgr2=0x7f91980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 --2- 172.21.15.7:0/4016799556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198074070 0x7f91980744f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f918c0098f0 tx=0x7f918c009c00 comp rx=0 tx=0).stop 2024-01-11T18:04:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 -- 172.21.15.7:0/4016799556 shutdown_connections 2024-01-11T18:04:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 --2- 172.21.15.7:0/4016799556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f919810db80 0x7f919810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 --2- 172.21.15.7:0/4016799556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198075650 0x7f9198073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 --2- 172.21.15.7:0/4016799556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198074070 0x7f91980744f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 -- 172.21.15.7:0/4016799556 >> 172.21.15.7:0/4016799556 conn(0x7f91980fcb00 msgr2=0x7f91980fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.294+0000 7f919d3ff700 1 -- 172.21.15.7:0/4016799556 shutdown_connections 2024-01-11T18:04:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.295+0000 7f919d3ff700 1 -- 172.21.15.7:0/4016799556 wait complete. 2024-01-11T18:04:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.296+0000 7f919d3ff700 1 Processor -- start 2024-01-11T18:04:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.296+0000 7f919d3ff700 1 -- start start 2024-01-11T18:04:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.296+0000 7f919d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198074070 0x7f919819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.296+0000 7f919d3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 0x7f919819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.297+0000 7f91967fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 0x7f919819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.297+0000 7f91967fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 0x7f919819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34270/0 (socket says 172.21.15.7:34270) 2024-01-11T18:04:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.298+0000 7f919d3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919810db80 0x7f91981a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.298+0000 7f919d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981a54c0 con 0x7f9198074070 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.298+0000 7f919d3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981a5600 con 0x7f9198075650 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.298+0000 7f919d3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981a5790 con 0x7f919810db80 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.298+0000 7f91967fc700 1 -- 172.21.15.7:0/1416058954 learned_addr learned my addr 172.21.15.7:0/1416058954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.298+0000 7f91977fe700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919810db80 0x7f91981a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91967fc700 1 -- 172.21.15.7:0/1416058954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919810db80 msgr2=0x7f91981a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91967fc700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919810db80 0x7f91981a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91967fc700 1 -- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198074070 msgr2=0x7f919819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91967fc700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198074070 0x7f919819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91967fc700 1 -- 172.21.15.7:0/1416058954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880095f0 con 0x7f9198075650 2024-01-11T18:04:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91977fe700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919810db80 0x7f91981a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.299+0000 7f91967fc700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 0x7f919819de80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f918c009230 tx=0x7f918c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.300+0000 7f917ffff700 1 -- 172.21.15.7:0/1416058954 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c01c070 con 0x7f9198075650 2024-01-11T18:04:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.302+0000 7f917ffff700 1 -- 172.21.15.7:0/1416058954 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f918c005500 con 0x7f9198075650 2024-01-11T18:04:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.302+0000 7f917ffff700 1 -- 172.21.15.7:0/1416058954 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c003ca0 con 0x7f9198075650 2024-01-11T18:04:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.302+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c009520 con 0x7f9198075650 2024-01-11T18:04:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.303+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91981a5c50 con 0x7f9198075650 2024-01-11T18:04:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.321+0000 7f917ffff700 1 -- 172.21.15.7:0/1416058954 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f918c020030 con 0x7f9198075650 2024-01-11T18:04:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.323+0000 7f917ffff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a20a0 0x7f91800a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.323+0000 7f917ffff700 1 -- 172.21.15.7:0/1416058954 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f918c0cef00 con 0x7f9198075650 2024-01-11T18:04:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.324+0000 7f91b117fb80 1 Processor -- start 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.324+0000 7f9196ffd700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a20a0 0x7f91800a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.324+0000 7f91b117fb80 1 -- start start 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdacf20 0x55f97cdad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 0x55f97cdb4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91b117fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cca4030 con 0x55f97cdacf20 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91b117fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cd07ac0 con 0x55f97cdadc10 2024-01-11T18:04:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91b117fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cd053e0 con 0x55f97cdae5d0 2024-01-11T18:04:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91977fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 0x55f97cdb4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91977fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 0x55f97cdb4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35116/0 (socket says 172.21.15.7:35116) 2024-01-11T18:04:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.325+0000 7f91977fe700 1 -- 172.21.15.7:0/3179586429 learned_addr learned my addr 172.21.15.7:0/3179586429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.326+0000 7f9196ffd700 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdacf20 0x55f97cdad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:03:59 smithi007 ceph-mon[33361]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 495 op/s 2024-01-11T18:04:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.327+0000 7f91977fe700 1 -- 172.21.15.7:0/3179586429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 msgr2=0x55f97cdae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.327+0000 7f91977fe700 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.327+0000 7f91977fe700 1 -- 172.21.15.7:0/3179586429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdacf20 msgr2=0x55f97cdad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.327+0000 7f91977fe700 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdacf20 0x55f97cdad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.327+0000 7f91977fe700 1 -- 172.21.15.7:0/3179586429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f97cb75350 con 0x55f97cdae5d0 2024-01-11T18:04:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.327+0000 7f9196ffd700 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdacf20 0x55f97cdad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.328+0000 7f9196ffd700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a20a0 0x7f91800a4560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91880095c0 tx=0x7f9188009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.328+0000 7f91977fe700 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 0x55f97cdb4e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9190009400 tx=0x7f9190009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a55c68f816730a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.329+0000 7f917d7fa700 1 -- 172.21.15.7:0/3179586429 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190014610 con 0x55f97cdae5d0 2024-01-11T18:04:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.329+0000 7f917d7fa700 1 -- 172.21.15.7:0/3179586429 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919000f050 con 0x55f97cdae5d0 2024-01-11T18:04:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.329+0000 7f917d7fa700 1 -- 172.21.15.7:0/3179586429 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91900139d0 con 0x55f97cdae5d0 2024-01-11T18:04:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.330+0000 7f91b117fb80 1 -- 172.21.15.7:0/3179586429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 msgr2=0x55f97cdb4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.330+0000 7f91b117fb80 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 0x55f97cdb4e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9190009400 tx=0x7f9190009920 comp rx=0 tx=0).stop 2024-01-11T18:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.331+0000 7f91b117fb80 1 -- 172.21.15.7:0/3179586429 shutdown_connections 2024-01-11T18:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.331+0000 7f91b117fb80 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdae5d0 0x55f97cdb4e60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.331+0000 7f91b117fb80 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdacf20 0x55f97cdad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.331+0000 7f91b117fb80 1 --2- 172.21.15.7:0/3179586429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.331+0000 7f91b117fb80 1 -- 172.21.15.7:0/3179586429 >> 172.21.15.7:0/3179586429 conn(0x55f97cd22740 msgr2=0x55f97cda7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.332+0000 7f91b117fb80 1 -- 172.21.15.7:0/3179586429 shutdown_connections 2024-01-11T18:04:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.332+0000 7f91b117fb80 1 -- 172.21.15.7:0/3179586429 wait complete. 2024-01-11T18:04:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.332+0000 7f91b117fb80 1 -- start start 2024-01-11T18:04:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.333+0000 7f91b117fb80 1 Processor -- start 2024-01-11T18:04:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.336+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.337+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdc7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.337+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.337+0000 7f91b117fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cd00cd0 con 0x55f97cdae5d0 2024-01-11T18:04:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.337+0000 7f91b117fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cc6f340 con 0x55f97cdadc10 2024-01-11T18:04:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.337+0000 7f91b117fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cc6efe0 con 0x55f97cdacf20 2024-01-11T18:04:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.337+0000 7f91977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.338+0000 7f91b117fb80 10 client.0 _ll_register_callbacks cb 0x55f97cca16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.338+0000 7f91977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54190/0 (socket says 172.21.15.7:54190) 2024-01-11T18:04:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.338+0000 7f91977fe700 1 -- 172.21.15.7:0/1712447072 learned_addr learned my addr 172.21.15.7:0/1712447072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.339+0000 7f91967fc700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdc7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.339+0000 7f91977fe700 1 -- 172.21.15.7:0/1712447072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 msgr2=0x55f97cdc7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.339+0000 7f91977fe700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.339+0000 7f91977fe700 1 -- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 msgr2=0x55f97cdc7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.339+0000 7f91977fe700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdc7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.339+0000 7f91977fe700 1 -- 172.21.15.7:0/1712447072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91900090b0 con 0x55f97cdae5d0 2024-01-11T18:04:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.340+0000 7f91967fc700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdc7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.340+0000 7f91977fe700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcefe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f919000b010 tx=0x7f9190013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c6d3c8c9dad428 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.340+0000 7f917aff5700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:04:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.341+0000 7f917aff5700 1 -- 172.21.15.7:0/1712447072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190005240 con 0x55f97cdae5d0 2024-01-11T18:04:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.341+0000 7f917aff5700 1 -- 172.21.15.7:0/1712447072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9190013e90 con 0x55f97cdae5d0 2024-01-11T18:04:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.341+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f97cd9fd60 con 0x55f97cdae5d0 2024-01-11T18:04:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.342+0000 7f917aff5700 1 -- 172.21.15.7:0/1712447072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9190009650 con 0x55f97cdae5d0 2024-01-11T18:04:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.342+0000 7f91b117fb80 10 client.15957 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.342+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f97cb21b00 con 0x55f97cdae5d0 2024-01-11T18:04:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.343+0000 7f917aff5700 1 -- 172.21.15.7:0/1712447072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f97cd9fd60 con 0x55f97cdae5d0 2024-01-11T18:04:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.343+0000 7f917aff5700 1 -- 172.21.15.7:0/1712447072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9190014900 con 0x55f97cdae5d0 2024-01-11T18:04:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.345+0000 7f91b117fb80 10 client.15957 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.345+0000 7f91b117fb80 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.345+0000 7f91b117fb80 20 client.15957 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.345+0000 7f91b117fb80 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97cdd3810 0x55f97cdd5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.345+0000 7f91b117fb80 4 client.15957 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:04:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.345+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f97ccf9b60 con 0x55f97cdd3810 2024-01-11T18:04:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.346+0000 7f9196ffd700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97cdd3810 0x55f97cdd5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.347+0000 7f9196ffd700 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97cdd3810 0x55f97cdd5c30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d1864f844f426e server_cookie=d1f906caa078fde7 in_seq=0 out_seq=0 2024-01-11T18:04:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.347+0000 7f917aff5700 10 client.15957 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.348+0000 7f917aff5700 1 -- 172.21.15.7:0/1712447072 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f97ccf9b60 con 0x55f97cdd3810 2024-01-11T18:04:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.348+0000 7f917aff5700 10 client.15957 handle_command_reply: tid=0 2024-01-11T18:04:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.348+0000 7f91b117fb80 1 client.15957 shutdown 2024-01-11T18:04:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 msgr2=0x55f97cdcefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcefe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f919000b010 tx=0x7f9190013a30 comp rx=0 tx=0).stop 2024-01-11T18:04:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 shutdown_connections 2024-01-11T18:04:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97cdd3810 0x55f97cdd5c30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcefe0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 --2- 172.21.15.7:0/1712447072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdc7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f917aff5700 0 client.15957 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 >> 172.21.15.7:0/1712447072 conn(0x55f97cd22740 msgr2=0x55f97cda6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.349+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 shutdown_connections 2024-01-11T18:04:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.350+0000 7f91b117fb80 1 -- 172.21.15.7:0/1712447072 wait complete. 2024-01-11T18:04:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.350+0000 7f91b117fb80 20 client.15957 trim_cache size 0 max 16384 2024-01-11T18:04:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.381+0000 7f91b117fb80 1 Processor -- start 2024-01-11T18:04:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.381+0000 7f91b117fb80 1 -- start start 2024-01-11T18:04:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.381+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.381+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91b117fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cb77640 con 0x55f97cdae5d0 2024-01-11T18:04:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91b117fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cdc87f0 con 0x55f97cdadc10 2024-01-11T18:04:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91b117fb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cda9690 con 0x55f97cdacf20 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91967fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91967fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34296/0 (socket says 172.21.15.7:34296) 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.382+0000 7f91967fc700 1 -- 172.21.15.7:0/148800476 learned_addr learned my addr 172.21.15.7:0/148800476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.383+0000 7f9196ffd700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.383+0000 7f91967fc700 1 -- 172.21.15.7:0/148800476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 msgr2=0x55f97cdc6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.383+0000 7f91967fc700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.383+0000 7f91967fc700 1 -- 172.21.15.7:0/148800476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 msgr2=0x55f97cdcf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.383+0000 7f91977fe700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.384+0000 7f91967fc700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.384+0000 7f91967fc700 1 -- 172.21.15.7:0/148800476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f97ccc21d0 con 0x55f97cdadc10 2024-01-11T18:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.384+0000 7f9196ffd700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.385+0000 7f91977fe700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.385+0000 7f91967fc700 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcefa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f918c0a3730 tx=0x7f918c003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec29317b5869dcd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.385+0000 7f917cff9700 1 -- 172.21.15.7:0/148800476 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c09b960 con 0x55f97cdadc10 2024-01-11T18:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.386+0000 7f917cff9700 1 -- 172.21.15.7:0/148800476 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f918c09fab0 con 0x55f97cdadc10 2024-01-11T18:04:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.386+0000 7f917cff9700 1 -- 172.21.15.7:0/148800476 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c098070 con 0x55f97cdadc10 2024-01-11T18:04:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.388+0000 7f91b117fb80 1 -- 172.21.15.7:0/148800476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 msgr2=0x55f97cdcefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.388+0000 7f91b117fb80 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcefa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f918c0a3730 tx=0x7f918c003710 comp rx=0 tx=0).stop 2024-01-11T18:04:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.388+0000 7f91b117fb80 1 -- 172.21.15.7:0/148800476 shutdown_connections 2024-01-11T18:04:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.388+0000 7f91b117fb80 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdc6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.389+0000 7f91b117fb80 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.389+0000 7f91b117fb80 1 --2- 172.21.15.7:0/148800476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcefa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.389+0000 7f91b117fb80 1 -- 172.21.15.7:0/148800476 >> 172.21.15.7:0/148800476 conn(0x55f97cd22740 msgr2=0x55f97cda39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.389+0000 7f91b117fb80 1 -- 172.21.15.7:0/148800476 shutdown_connections 2024-01-11T18:04:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.389+0000 7f91b117fb80 1 -- 172.21.15.7:0/148800476 wait complete. 2024-01-11T18:04:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.390+0000 7f91b117fb80 1 -- start start 2024-01-11T18:04:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.391+0000 7f91b117fb80 1 Processor -- start 2024-01-11T18:04:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.393+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.393+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.393+0000 7f91b117fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91967fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91967fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34298/0 (socket says 172.21.15.7:34298) 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91967fc700 1 -- 172.21.15.7:0/2313102699 learned_addr learned my addr 172.21.15.7:0/2313102699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cdcdff0 con 0x55f97cdae5d0 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cdce160 con 0x55f97cdadc10 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f97cdce320 con 0x55f97cdacf20 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91b117fb80 10 client.0 _ll_register_callbacks cb 0x55f97cca16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.394+0000 7f91977fe700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.395+0000 7f9196ffd700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.396+0000 7f91967fc700 1 -- 172.21.15.7:0/2313102699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 msgr2=0x55f97cdca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.396+0000 7f91967fc700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.396+0000 7f91967fc700 1 -- 172.21.15.7:0/2313102699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 msgr2=0x55f97cdcdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.396+0000 7f91967fc700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.396+0000 7f91967fc700 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91981a5c50 con 0x55f97cdadc10 2024-01-11T18:04:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.399+0000 7f91967fc700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcaae0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f918c0098c0 tx=0x7f918c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc603986f17878e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.399+0000 7f917aff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.399+0000 7f917aff5700 1 -- 172.21.15.7:0/2313102699 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c0cc780 con 0x55f97cdadc10 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.399+0000 7f917aff5700 1 -- 172.21.15.7:0/2313102699 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f918c09b960 con 0x55f97cdadc10 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.399+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f919000f050 con 0x55f97cdadc10 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f917aff5700 1 -- 172.21.15.7:0/2313102699 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918c0a3c60 con 0x55f97cdadc10 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 10 client.26069 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f97cb79c70 con 0x55f97cdadc10 2024-01-11T18:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f917aff5700 1 -- 172.21.15.7:0/2313102699 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919000f050 con 0x55f97cdadc10 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f917aff5700 1 -- 172.21.15.7:0/2313102699 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f918c0bb0b0 con 0x55f97cdadc10 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 10 client.26069 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 20 client.26069 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97ce3b0d0 0x55f97ce3d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 4 client.26069 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.400+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f97ccf9880 con 0x55f97ce3b0d0 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.401+0000 7f9196ffd700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97ce3b0d0 0x55f97ce3d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.401+0000 7f917aff5700 10 client.26069 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.403+0000 7f9196ffd700 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97ce3b0d0 0x55f97ce3d4c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350da18c4f9c51d5 server_cookie=99a6c28dbfc55033 in_seq=0 out_seq=0 2024-01-11T18:04:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f917aff5700 1 -- 172.21.15.7:0/2313102699 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f97ccf9880 con 0x55f97ce3b0d0 2024-01-11T18:04:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f917aff5700 10 client.26069 handle_command_reply: tid=0 2024-01-11T18:04:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 client.26069 shutdown 2024-01-11T18:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 msgr2=0x55f97cdcaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcaae0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f918c0098c0 tx=0x7f918c000bc0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 shutdown_connections 2024-01-11T18:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f97ce3b0d0 0x55f97ce3d4c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f97cdacf20 0x55f97cdca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f97cdae5d0 0x55f97cdcdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 --2- 172.21.15.7:0/2313102699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f97cdadc10 0x55f97cdcaae0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 >> 172.21.15.7:0/2313102699 conn(0x55f97cd22740 msgr2=0x55f97cda39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.429+0000 7f917aff5700 0 client.26069 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.430+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 shutdown_connections 2024-01-11T18:04:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.430+0000 7f91b117fb80 1 -- 172.21.15.7:0/2313102699 wait complete. 2024-01-11T18:04:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.430+0000 7f91b117fb80 20 client.26069 trim_cache size 0 max 16384 2024-01-11T18:04:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:04:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:04:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:04:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.432+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a20a0 msgr2=0x7f91800a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.432+0000 7f919d3ff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a20a0 0x7f91800a4560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91880095c0 tx=0x7f9188009940 comp rx=0 tx=0).stop 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.432+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 msgr2=0x7f919819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.432+0000 7f919d3ff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 0x7f919819de80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f918c009230 tx=0x7f918c00b840 comp rx=0 tx=0).stop 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 shutdown_connections 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a20a0 0x7f91800a4560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f919810db80 0x7f91981a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198074070 0x7f919819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 --2- 172.21.15.7:0/1416058954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198075650 0x7f919819de80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 >> 172.21.15.7:0/1416058954 conn(0x7f91980fcb00 msgr2=0x7f9198100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 shutdown_connections 2024-01-11T18:04:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:00.433+0000 7f919d3ff700 1 -- 172.21.15.7:0/1416058954 wait complete. 2024-01-11T18:04:02.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:01 smithi016 ceph-mon[43195]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 594 op/s 2024-01-11T18:04:02.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:01 smithi038 ceph-mon[43582]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 594 op/s 2024-01-11T18:04:02.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:01 smithi007 ceph-mon[33361]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 594 op/s 2024-01-11T18:04:04.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 335.0 s, 617.8 tps, lat 162.843 ms stddev 183.701 2024-01-11T18:04:04.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:03 smithi016 ceph-mon[43195]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 561 op/s 2024-01-11T18:04:04.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:03 smithi038 ceph-mon[43582]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 561 op/s 2024-01-11T18:04:04.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:03 smithi007 ceph-mon[33361]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 MiB/s wr, 561 op/s 2024-01-11T18:04:05.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:05 smithi016 ceph-mon[43195]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 579 op/s 2024-01-11T18:04:06.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:05 smithi038 ceph-mon[43582]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 579 op/s 2024-01-11T18:04:06.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:05 smithi007 ceph-mon[33361]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 579 op/s 2024-01-11T18:04:08.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:07 smithi016 ceph-mon[43195]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 592 op/s 2024-01-11T18:04:08.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:07 smithi038 ceph-mon[43582]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 592 op/s 2024-01-11T18:04:08.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:07 smithi007 ceph-mon[33361]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 592 op/s 2024-01-11T18:04:09.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 340.0 s, 467.2 tps, lat 211.320 ms stddev 246.091 2024-01-11T18:04:09.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:08 smithi016 ceph-mon[43195]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 530 op/s 2024-01-11T18:04:09.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:08 smithi038 ceph-mon[43582]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 530 op/s 2024-01-11T18:04:09.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:08 smithi007 ceph-mon[33361]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 530 op/s 2024-01-11T18:04:12.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:11 smithi016 ceph-mon[43195]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 10 MiB/s wr, 626 op/s 2024-01-11T18:04:12.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:11 smithi038 ceph-mon[43582]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 10 MiB/s wr, 626 op/s 2024-01-11T18:04:12.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:11 smithi007 ceph-mon[33361]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 10 MiB/s wr, 626 op/s 2024-01-11T18:04:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 -- 172.21.15.7:0/3167952430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 msgr2=0x7f9080104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 --2- 172.21.15.7:0/3167952430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080104410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f90700098f0 tx=0x7f9070009c00 comp rx=0 tx=0).stop 2024-01-11T18:04:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 -- 172.21.15.7:0/3167952430 shutdown_connections 2024-01-11T18:04:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 --2- 172.21.15.7:0/3167952430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080104410 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 --2- 172.21.15.7:0/3167952430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107280 0x7f90801096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 --2- 172.21.15.7:0/3167952430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9080104950 0x7f9080106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.706+0000 7f9086fff700 1 -- 172.21.15.7:0/3167952430 >> 172.21.15.7:0/3167952430 conn(0x7f90800fba30 msgr2=0x7f90800fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.708+0000 7f9086fff700 1 -- 172.21.15.7:0/3167952430 shutdown_connections 2024-01-11T18:04:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.708+0000 7f9086fff700 1 -- 172.21.15.7:0/3167952430 wait complete. 2024-01-11T18:04:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.710+0000 7f9086fff700 1 Processor -- start 2024-01-11T18:04:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.711+0000 7f9086fff700 1 -- start start 2024-01-11T18:04:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.712+0000 7f9086fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.712+0000 7f9086fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107280 0x7f9080195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.712+0000 7f9086fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908019a790 0x7f908019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.712+0000 7f9084d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.712+0000 7f9084d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080195240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46286/0 (socket says 172.21.15.7:46286) 2024-01-11T18:04:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.712+0000 7f9086fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908019d160 con 0x7f9080107280 2024-01-11T18:04:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.713+0000 7f9086fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908019d2d0 con 0x7f908019a790 2024-01-11T18:04:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.713+0000 7f9086fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908019d490 con 0x7f9080102020 2024-01-11T18:04:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.713+0000 7f9077fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107280 0x7f9080195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.713+0000 7f9084d9b700 1 -- 172.21.15.7:0/3320583441 learned_addr learned my addr 172.21.15.7:0/3320583441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.713+0000 7f908559c700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908019a790 0x7f908019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.714+0000 7f9084d9b700 1 -- 172.21.15.7:0/3320583441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908019a790 msgr2=0x7f908019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.714+0000 7f9084d9b700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908019a790 0x7f908019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.714+0000 7f9084d9b700 1 -- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107280 msgr2=0x7f9080195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.714+0000 7f9084d9b700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107280 0x7f9080195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.714+0000 7f9084d9b700 1 -- 172.21.15.7:0/3320583441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070009520 con 0x7f9080102020 2024-01-11T18:04:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.715+0000 7f9084d9b700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080195240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9070009230 tx=0x7f907000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.715+0000 7f907e7fc700 1 -- 172.21.15.7:0/3320583441 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907001c070 con 0x7f9080102020 2024-01-11T18:04:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.717+0000 7f907e7fc700 1 -- 172.21.15.7:0/3320583441 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9070005500 con 0x7f9080102020 2024-01-11T18:04:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.717+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908019d710 con 0x7f9080102020 2024-01-11T18:04:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.717+0000 7f907e7fc700 1 -- 172.21.15.7:0/3320583441 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070017400 con 0x7f9080102020 2024-01-11T18:04:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.717+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908019dc40 con 0x7f9080102020 2024-01-11T18:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.727+0000 7f907e7fc700 1 -- 172.21.15.7:0/3320583441 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f9070005040 con 0x7f9080102020 2024-01-11T18:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.731+0000 7f907e7fc700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90600a1fd0 0x7f90600a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.731+0000 7f909ad93b80 1 Processor -- start 2024-01-11T18:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.732+0000 7f909ad93b80 1 -- start start 2024-01-11T18:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.732+0000 7f907e7fc700 1 -- 172.21.15.7:0/3320583441 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f90700cf2e0 con 0x7f9080102020 2024-01-11T18:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.732+0000 7f9077fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90600a1fd0 0x7f90600a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d4334340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 0x5595d4335090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d433be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f909ad93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d422b030 con 0x5595d4333f20 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f909ad93b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d428eac0 con 0x5595d43355d0 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f909ad93b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d428c3e0 con 0x5595d4334c10 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f9077fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 0x5595d4335090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f908559c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d433be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f9077fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 0x5595d4335090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46302/0 (socket says 172.21.15.7:46302) 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f9077fff700 1 -- 172.21.15.7:0/1980220370 learned_addr learned my addr 172.21.15.7:0/1980220370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.733+0000 7f9084d9b700 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d4334340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.734+0000 7f9077fff700 1 -- 172.21.15.7:0/1980220370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 msgr2=0x5595d433be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.734+0000 7f9077fff700 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d433be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.734+0000 7f9077fff700 1 -- 172.21.15.7:0/1980220370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 msgr2=0x5595d4334340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.735+0000 7f9077fff700 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d4334340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.735+0000 7f9077fff700 1 -- 172.21.15.7:0/1980220370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d40fc350 con 0x5595d4334c10 2024-01-11T18:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.735+0000 7f9084d9b700 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d4334340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:04:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.736+0000 7f9077fff700 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 0x5595d4335090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9068007990 tx=0x7f9068007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1998bf549048d5d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.736+0000 7f9076ffd700 1 -- 172.21.15.7:0/1980220370 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800e070 con 0x5595d4334c10 2024-01-11T18:04:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.736+0000 7f9076ffd700 1 -- 172.21.15.7:0/1980220370 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906800f040 con 0x5595d4334c10 2024-01-11T18:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.737+0000 7f909ad93b80 1 -- 172.21.15.7:0/1980220370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 msgr2=0x5595d4335090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.738+0000 7f9076ffd700 1 -- 172.21.15.7:0/1980220370 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800e070 con 0x5595d4334c10 2024-01-11T18:04:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.738+0000 7f909ad93b80 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 0x5595d4335090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9068007990 tx=0x7f9068007ca0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.738+0000 7f9077fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90600a1fd0 0x7f90600a4490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9068021000 tx=0x7f906801c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 -- 172.21.15.7:0/1980220370 shutdown_connections 2024-01-11T18:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d433be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d4334340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 --2- 172.21.15.7:0/1980220370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4334c10 0x5595d4335090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 -- 172.21.15.7:0/1980220370 >> 172.21.15.7:0/1980220370 conn(0x5595d42a9740 msgr2=0x5595d432e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 -- 172.21.15.7:0/1980220370 shutdown_connections 2024-01-11T18:04:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.741+0000 7f909ad93b80 1 -- 172.21.15.7:0/1980220370 wait complete. 2024-01-11T18:04:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.742+0000 7f909ad93b80 1 -- start start 2024-01-11T18:04:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.742+0000 7f909ad93b80 1 Processor -- start 2024-01-11T18:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.743+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d433d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.743+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 0x5595d433dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.744+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 0x5595d4344c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.744+0000 7f9077fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 0x5595d433dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.744+0000 7f9084d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d433d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.744+0000 7f9084d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d433d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45254/0 (socket says 172.21.15.7:45254) 2024-01-11T18:04:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f908559c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 0x5595d4344c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f9077fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 0x5595d433dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51230/0 (socket says 172.21.15.7:51230) 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f908559c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 0x5595d4344c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46308/0 (socket says 172.21.15.7:46308) 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f9077fff700 1 -- 172.21.15.7:0/910966737 learned_addr learned my addr 172.21.15.7:0/910966737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d4287cd0 con 0x5595d4334c10 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d41f6340 con 0x5595d4333f20 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d41f5fe0 con 0x5595d43355d0 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.745+0000 7f909ad93b80 10 client.0 _ll_register_callbacks cb 0x5595d42286c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f9077fff700 1 -- 172.21.15.7:0/910966737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 msgr2=0x5595d4344c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f9077fff700 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 0x5595d4344c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f9077fff700 1 -- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 msgr2=0x5595d433d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f9077fff700 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d433d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f9077fff700 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908019dc40 con 0x5595d4334c10 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f908559c700 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 0x5595d4344c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f9077fff700 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 0x5595d433dc30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90680096c0 tx=0x7f90680096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3800961e7d3a83c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.746+0000 7f906ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.748+0000 7f906ffff700 1 -- 172.21.15.7:0/910966737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068029040 con 0x5595d4334c10 2024-01-11T18:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.748+0000 7f906ffff700 1 -- 172.21.15.7:0/910966737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f90680249b0 con 0x5595d4334c10 2024-01-11T18:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.748+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595d4326d60 con 0x5595d4334c10 2024-01-11T18:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.748+0000 7f906ffff700 1 -- 172.21.15.7:0/910966737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f906800e040 con 0x5595d4334c10 2024-01-11T18:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.748+0000 7f909ad93b80 10 client.15969 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.748+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f90680075c0 con 0x5595d4334c10 2024-01-11T18:04:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.749+0000 7f906ffff700 1 -- 172.21.15.7:0/910966737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595d4326d60 con 0x5595d4334c10 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f906ffff700 1 -- 172.21.15.7:0/910966737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9068005510 con 0x5595d4334c10 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f909ad93b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f909ad93b80 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f909ad93b80 20 client.15969 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f909ad93b80 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d4349470 0x5595d434b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f909ad93b80 4 client.15969 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.751+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595d4280b60 con 0x5595d4349470 2024-01-11T18:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.752+0000 7f9084d9b700 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d4349470 0x5595d434b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.752+0000 7f9084d9b700 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d4349470 0x5595d434b890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf4eb821351f5b5 server_cookie=717a95dc8820e3e2 in_seq=0 out_seq=0 2024-01-11T18:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.753+0000 7f906ffff700 10 client.15969 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.753+0000 7f906ffff700 1 -- 172.21.15.7:0/910966737 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595d4280b60 con 0x5595d4349470 2024-01-11T18:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.753+0000 7f906ffff700 10 client.15969 handle_command_reply: tid=0 2024-01-11T18:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 client.15969 shutdown 2024-01-11T18:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 msgr2=0x5595d433dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 0x5595d433dc30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90680096c0 tx=0x7f90680096f0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 shutdown_connections 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d4349470 0x5595d434b890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f906ffff700 0 client.15969 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d43355d0 0x5595d4344c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d433d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 --2- 172.21.15.7:0/910966737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4334c10 0x5595d433dc30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.754+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 >> 172.21.15.7:0/910966737 conn(0x5595d42a9740 msgr2=0x5595d432df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.756+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 shutdown_connections 2024-01-11T18:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.756+0000 7f909ad93b80 1 -- 172.21.15.7:0/910966737 wait complete. 2024-01-11T18:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.756+0000 7f909ad93b80 20 client.15969 trim_cache size 0 max 16384 2024-01-11T18:04:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.795+0000 7f909ad93b80 1 Processor -- start 2024-01-11T18:04:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 -- start start 2024-01-11T18:04:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d433c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d4344c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d433d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d40fe640 con 0x5595d4333f20 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d433e5f0 con 0x5595d43355d0 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.796+0000 7f909ad93b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d432b560 con 0x5595d4337750 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.797+0000 7f908559c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d433d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.797+0000 7f908559c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d433d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46322/0 (socket says 172.21.15.7:46322) 2024-01-11T18:04:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.797+0000 7f908559c700 1 -- 172.21.15.7:0/662783150 learned_addr learned my addr 172.21.15.7:0/662783150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.797+0000 7f9077fff700 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d4344c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.797+0000 7f9084d9b700 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d433c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.798+0000 7f908559c700 1 -- 172.21.15.7:0/662783150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 msgr2=0x5595d4344c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.798+0000 7f908559c700 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d4344c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.798+0000 7f908559c700 1 -- 172.21.15.7:0/662783150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 msgr2=0x5595d433c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.798+0000 7f908559c700 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d433c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.798+0000 7f908559c700 1 -- 172.21.15.7:0/662783150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d40a8b00 con 0x5595d4337750 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.798+0000 7f908559c700 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d433d6f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9078012550 tx=0x7f9078012860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6fe0da2666ce15e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.799+0000 7f90767fc700 1 -- 172.21.15.7:0/662783150 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078017070 con 0x5595d4337750 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.799+0000 7f90767fc700 1 -- 172.21.15.7:0/662783150 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9078012e90 con 0x5595d4337750 2024-01-11T18:04:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.799+0000 7f90767fc700 1 -- 172.21.15.7:0/662783150 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907801ba50 con 0x5595d4337750 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.800+0000 7f909ad93b80 1 -- 172.21.15.7:0/662783150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 msgr2=0x5595d433d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.800+0000 7f909ad93b80 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d433d6f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9078012550 tx=0x7f9078012860 comp rx=0 tx=0).stop 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.800+0000 7f909ad93b80 1 -- 172.21.15.7:0/662783150 shutdown_connections 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.801+0000 7f909ad93b80 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d433d6f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.801+0000 7f909ad93b80 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d4333f20 0x5595d433c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.801+0000 7f909ad93b80 1 --2- 172.21.15.7:0/662783150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d43355d0 0x5595d4344c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.801+0000 7f909ad93b80 1 -- 172.21.15.7:0/662783150 >> 172.21.15.7:0/662783150 conn(0x5595d42a9740 msgr2=0x5595d433cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.801+0000 7f909ad93b80 1 -- 172.21.15.7:0/662783150 shutdown_connections 2024-01-11T18:04:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.802+0000 7f909ad93b80 1 -- 172.21.15.7:0/662783150 wait complete. 2024-01-11T18:04:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.802+0000 7f909ad93b80 1 -- start start 2024-01-11T18:04:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.802+0000 7f909ad93b80 1 Processor -- start 2024-01-11T18:04:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.803+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d4340dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.803+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d43355d0 0x5595d4343310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.803+0000 7f909ad93b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d4343920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f909ad93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d4344030 con 0x5595d43355d0 2024-01-11T18:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f909ad93b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d43441a0 con 0x5595d4333f20 2024-01-11T18:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f908559c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d4343920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f908559c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d4343920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46338/0 (socket says 172.21.15.7:46338) 2024-01-11T18:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f908559c700 1 -- 172.21.15.7:0/145077777 learned_addr learned my addr 172.21.15.7:0/145077777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f9084d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d4340dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d4344360 con 0x5595d4337750 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f909ad93b80 10 client.0 _ll_register_callbacks cb 0x5595d42286c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.804+0000 7f9077fff700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d43355d0 0x5595d4343310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f908559c700 1 -- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 msgr2=0x5595d4340dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f908559c700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d4340dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f908559c700 1 -- 172.21.15.7:0/145077777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d43355d0 msgr2=0x5595d4343310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f908559c700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d43355d0 0x5595d4343310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f908559c700 1 -- 172.21.15.7:0/145077777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907800ada0 con 0x5595d4337750 2024-01-11T18:04:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f9077fff700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d43355d0 0x5595d4343310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f908559c700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d4343920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9078012c60 tx=0x7f907801bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb707ac53d5df8c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f906ffff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.805+0000 7f906ffff700 1 -- 172.21.15.7:0/145077777 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907802a070 con 0x5595d4337750 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.806+0000 7f906ffff700 1 -- 172.21.15.7:0/145077777 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9078004020 con 0x5595d4337750 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.806+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f906800f040 con 0x5595d4337750 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.806+0000 7f906ffff700 1 -- 172.21.15.7:0/145077777 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9078017050 con 0x5595d4337750 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.806+0000 7f909ad93b80 10 client.26128 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.806+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5595d42491d0 con 0x5595d4337750 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.806+0000 7f906ffff700 1 -- 172.21.15.7:0/145077777 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906800f040 con 0x5595d4337750 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f906ffff700 1 -- 172.21.15.7:0/145077777 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f907802d020 con 0x5595d4337750 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f909ad93b80 10 client.26128 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f909ad93b80 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f909ad93b80 20 client.26128 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f909ad93b80 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d43b1550 0x5595d43b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f909ad93b80 4 client.26128 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595d4280880 con 0x5595d43b1550 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.807+0000 7f9084d9b700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d43b1550 0x5595d43b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.808+0000 7f9084d9b700 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d43b1550 0x5595d43b3970 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e65a760ce0965df server_cookie=a5c2a4e904a99d92 in_seq=0 out_seq=0 2024-01-11T18:04:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.808+0000 7f906ffff700 10 client.26128 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.850+0000 7f906ffff700 1 -- 172.21.15.7:0/145077777 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595d4280880 con 0x5595d43b1550 2024-01-11T18:04:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.850+0000 7f906ffff700 10 client.26128 handle_command_reply: tid=0 2024-01-11T18:04:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.850+0000 7f909ad93b80 1 client.26128 shutdown 2024-01-11T18:04:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 msgr2=0x5595d4343920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d4343920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9078012c60 tx=0x7f907801bee0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 shutdown_connections 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5595d43b1550 0x5595d43b3970 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5595d4337750 0x5595d4343920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595d4333f20 0x5595d4340dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 --2- 172.21.15.7:0/145077777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595d43355d0 0x5595d4343310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 >> 172.21.15.7:0/145077777 conn(0x5595d42a9740 msgr2=0x5595d433b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.851+0000 7f906ffff700 0 client.26128 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.853+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 shutdown_connections 2024-01-11T18:04:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.853+0000 7f909ad93b80 1 -- 172.21.15.7:0/145077777 wait complete. 2024-01-11T18:04:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.853+0000 7f909ad93b80 20 client.26128 trim_cache size 0 max 16384 2024-01-11T18:04:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:04:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:04:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:04:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:04:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.854+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90600a1fd0 msgr2=0x7f90600a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.854+0000 7f9086fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90600a1fd0 0x7f90600a4490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9068021000 tx=0x7f906801c000 comp rx=0 tx=0).stop 2024-01-11T18:04:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.854+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 msgr2=0x7f9080195240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.854+0000 7f9086fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080195240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9070009230 tx=0x7f907000b790 comp rx=0 tx=0).stop 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 shutdown_connections 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f90600a1fd0 0x7f90600a4490 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9080102020 0x7f9080195240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908019a790 0x7f908019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 --2- 172.21.15.7:0/3320583441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9080107280 0x7f9080195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 >> 172.21.15.7:0/3320583441 conn(0x7f90800fba30 msgr2=0x7f9080100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 shutdown_connections 2024-01-11T18:04:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:13.855+0000 7f9086fff700 1 -- 172.21.15.7:0/3320583441 wait complete. 2024-01-11T18:04:14.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 345.0 s, 539.8 tps, lat 181.928 ms stddev 220.088 2024-01-11T18:04:14.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:13 smithi016 ceph-mon[43195]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.3 MiB/s wr, 552 op/s 2024-01-11T18:04:14.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:13 smithi038 ceph-mon[43582]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.3 MiB/s wr, 552 op/s 2024-01-11T18:04:14.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:13 smithi007 ceph-mon[33361]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.3 MiB/s wr, 552 op/s 2024-01-11T18:04:15.991 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:15 smithi016 ceph-mon[43195]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.5 MiB/s wr, 564 op/s 2024-01-11T18:04:16.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:15 smithi038 ceph-mon[43582]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.5 MiB/s wr, 564 op/s 2024-01-11T18:04:16.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:15 smithi007 ceph-mon[33361]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.5 MiB/s wr, 564 op/s 2024-01-11T18:04:18.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:17 smithi016 ceph-mon[43195]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 775 op/s 2024-01-11T18:04:18.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:17 smithi038 ceph-mon[43582]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 775 op/s 2024-01-11T18:04:18.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:17 smithi007 ceph-mon[33361]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 775 op/s 2024-01-11T18:04:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 350.0 s, 354.6 tps, lat 291.553 ms stddev 444.536 2024-01-11T18:04:20.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:19 smithi016 ceph-mon[43195]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 MiB/s wr, 807 op/s 2024-01-11T18:04:20.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:19 smithi038 ceph-mon[43582]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 MiB/s wr, 807 op/s 2024-01-11T18:04:20.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:19 smithi007 ceph-mon[33361]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 MiB/s wr, 807 op/s 2024-01-11T18:04:22.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:21 smithi016 ceph-mon[43195]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-01-11T18:04:22.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:21 smithi038 ceph-mon[43582]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-01-11T18:04:22.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:21 smithi007 ceph-mon[33361]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-01-11T18:04:23.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:22 smithi016 ceph-mon[43195]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2024-01-11T18:04:23.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:22 smithi038 ceph-mon[43582]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2024-01-11T18:04:23.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:22 smithi007 ceph-mon[33361]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2024-01-11T18:04:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 355.0 s, 612.6 tps, lat 165.546 ms stddev 191.585 2024-01-11T18:04:25.990 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:25 smithi016 ceph-mon[43195]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 20 MiB/s wr, 1.42k op/s 2024-01-11T18:04:26.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:25 smithi038 ceph-mon[43582]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 20 MiB/s wr, 1.42k op/s 2024-01-11T18:04:26.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:25 smithi007 ceph-mon[33361]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 20 MiB/s wr, 1.42k op/s 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.146+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/1029597766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4073450 msgr2=0x7fc0b4073870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.146+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/1029597766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4073450 0x7fc0b4073870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007570 tx=0x7fc0b0007880 comp rx=0 tx=0).stop 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/1029597766 shutdown_connections 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/1029597766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4073450 0x7fc0b4073870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/1029597766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0b4078d00 0x7fc0b407b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/1029597766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4076330 0x7fc0b40787c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/1029597766 >> 172.21.15.7:0/1029597766 conn(0x7fc0b406e500 msgr2=0x7fc0b4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/1029597766 shutdown_connections 2024-01-11T18:04:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.147+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/1029597766 wait complete. 2024-01-11T18:04:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 Processor -- start 2024-01-11T18:04:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 -- start start 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 0x7fc0b41b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4076330 0x7fc0b41b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0b4078d00 0x7fc0b41ba510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41bacb0 con 0x7fc0b4078d00 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41bae20 con 0x7fc0b4073450 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.149+0000 7fc0bc7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41bafe0 con 0x7fc0b4076330 2024-01-11T18:04:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 0x7fc0b41b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 0x7fc0b41b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35306/0 (socket says 172.21.15.7:35306) 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 -- 172.21.15.7:0/854822800 learned_addr learned my addr 172.21.15.7:0/854822800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0b9d9a700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4076330 0x7fc0b41b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0b4078d00 0x7fc0b41ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 -- 172.21.15.7:0/854822800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4076330 msgr2=0x7fc0b41b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4076330 0x7fc0b41b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 -- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0b4078d00 msgr2=0x7fc0b41ba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0b4078d00 0x7fc0b41ba510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.150+0000 7fc0ba59b700 1 -- 172.21.15.7:0/854822800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0007220 con 0x7fc0b4073450 2024-01-11T18:04:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.151+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 0x7fc0b41b5930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000a5c0 tx=0x7fc0b000a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.151+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/854822800 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000f070 con 0x7fc0b4073450 2024-01-11T18:04:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.151+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/854822800 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b000aa30 con 0x7fc0b4073450 2024-01-11T18:04:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.151+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/854822800 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000ca30 con 0x7fc0b4073450 2024-01-11T18:04:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.151+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b41bb260 con 0x7fc0b4073450 2024-01-11T18:04:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.151+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b41bb710 con 0x7fc0b4073450 2024-01-11T18:04:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.158+0000 7fc0d044ab80 1 Processor -- start 2024-01-11T18:04:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.158+0000 7fc0d044ab80 1 -- start start 2024-01-11T18:04:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.158+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.158+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb153de030 con 0x55eb154e7c10 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb15441ac0 con 0x55eb154e6f20 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb1543f3e0 con 0x55eb154e85d0 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0bad9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0bad9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154e7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35308/0 (socket says 172.21.15.7:35308) 2024-01-11T18:04:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0bad9c700 1 -- 172.21.15.7:0/1793130967 learned_addr learned my addr 172.21.15.7:0/1793130967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.159+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.160+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/854822800 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc0b000cb90 con 0x7fc0b4073450 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.160+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1793130967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 msgr2=0x55eb154eee60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.160+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154eee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.160+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1793130967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 msgr2=0x55eb154e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.161+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.161+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1793130967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb152af350 con 0x55eb154e7c10 2024-01-11T18:04:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.161+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:04:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.161+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154e8090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00cbc00 tx=0x7fc0b00d4430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb60e1b4bd894f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.162+0000 7fc0a8ff9700 1 -- 172.21.15.7:0/1793130967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00d4c90 con 0x55eb154e7c10 2024-01-11T18:04:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.162+0000 7fc0a8ff9700 1 -- 172.21.15.7:0/1793130967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b00d4df0 con 0x55eb154e7c10 2024-01-11T18:04:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.162+0000 7fc0a8ff9700 1 -- 172.21.15.7:0/1793130967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00d4c90 con 0x55eb154e7c10 2024-01-11T18:04:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.163+0000 7fc0ab7fe700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc0a00a20a0 0x7fc0a00a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.163+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/854822800 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc0b00ce4f0 con 0x7fc0b4073450 2024-01-11T18:04:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.163+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1793130967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 msgr2=0x55eb154e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.163+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154e8090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00cbc00 tx=0x7fc0b00d4430 comp rx=0 tx=0).stop 2024-01-11T18:04:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.164+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1793130967 shutdown_connections 2024-01-11T18:04:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.164+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.164+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.164+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1793130967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154e8090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.164+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1793130967 >> 172.21.15.7:0/1793130967 conn(0x55eb1545c740 msgr2=0x55eb154e1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.165+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1793130967 shutdown_connections 2024-01-11T18:04:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.166+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1793130967 wait complete. 2024-01-11T18:04:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.166+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc0a00a20a0 0x7fc0a00a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.166+0000 7fc0d044ab80 1 -- start start 2024-01-11T18:04:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.167+0000 7fc0d044ab80 1 Processor -- start 2024-01-11T18:04:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.167+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc0a00a20a0 0x7fc0a00a4560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55eb154e6cc0 tx=0x7fc0ac007720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e6f20 0x55eb154f0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e7c10 0x55eb154f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb1543acd0 con 0x55eb154e6f20 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb153a9340 con 0x55eb154e7c10 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb153a8fe0 con 0x55eb154e85d0 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.168+0000 7fc0d044ab80 10 client.0 _ll_register_callbacks cb 0x55eb153db6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0b9d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e7c10 0x55eb154f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0b9d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e7c10 0x55eb154f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35322/0 (socket says 172.21.15.7:35322) 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0bad9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0b9d9a700 1 -- 172.21.15.7:0/3846380569 learned_addr learned my addr 172.21.15.7:0/3846380569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e6f20 0x55eb154f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0bad9c700 1 -- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e7c10 msgr2=0x55eb154f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e7c10 0x55eb154f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0bad9c700 1 -- 172.21.15.7:0/3846380569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e6f20 msgr2=0x55eb154f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e6f20 0x55eb154f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.169+0000 7fc0bad9c700 1 -- 172.21.15.7:0/3846380569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac007a60 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f7af0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac010dc0 tx=0x7fc0ac004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddce07680b5c11b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc09dffb700 1 -- 172.21.15.7:0/3846380569 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac01bb60 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc09dffb700 1 -- 172.21.15.7:0/3846380569 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc0ac021070 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb154d9d60 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc09dffb700 1 -- 172.21.15.7:0/3846380569 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0ac01bcc0 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc0d044ab80 10 client.26137 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.170+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fc0b41bb710 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.171+0000 7fc09dffb700 1 -- 172.21.15.7:0/3846380569 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb154d9d60 con 0x55eb154e85d0 2024-01-11T18:04:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc09dffb700 1 -- 172.21.15.7:0/3846380569 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc0ac01d4b0 con 0x55eb154e85d0 2024-01-11T18:04:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0d044ab80 10 client.26137 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0d044ab80 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0d044ab80 20 client.26137 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb154fc410 0x55eb154fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0d044ab80 4 client.26137 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:04:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb15433b60 con 0x55eb154fc410 2024-01-11T18:04:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.172+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb154fc410 0x55eb154fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.173+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb154fc410 0x55eb154fe830 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912ee00e74369028 server_cookie=cb77e0afa8a4226d in_seq=0 out_seq=0 2024-01-11T18:04:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.173+0000 7fc09dffb700 10 client.26137 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.174+0000 7fc09dffb700 1 -- 172.21.15.7:0/3846380569 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb15433b60 con 0x55eb154fc410 2024-01-11T18:04:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.174+0000 7fc09dffb700 10 client.26137 handle_command_reply: tid=0 2024-01-11T18:04:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.174+0000 7fc0d044ab80 1 client.26137 shutdown 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 msgr2=0x55eb154f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f7af0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac010dc0 tx=0x7fc0ac004be0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 shutdown_connections 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb154fc410 0x55eb154fe830 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f7af0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e6f20 0x55eb154f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3846380569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e7c10 0x55eb154f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc09dffb700 0 client.26137 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 >> 172.21.15.7:0/3846380569 conn(0x55eb1545c740 msgr2=0x55eb154e0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.177+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 shutdown_connections 2024-01-11T18:04:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.178+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3846380569 wait complete. 2024-01-11T18:04:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.178+0000 7fc0d044ab80 20 client.26137 trim_cache size 0 max 16384 2024-01-11T18:04:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.210+0000 7fc0d044ab80 1 Processor -- start 2024-01-11T18:04:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.210+0000 7fc0d044ab80 1 -- start start 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.210+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.210+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.210+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0ba59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0ba59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35330/0 (socket says 172.21.15.7:35330) 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb152b1640 con 0x55eb154e7c10 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb154e3270 con 0x55eb154e6f20 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb154f9800 con 0x55eb154e85d0 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0ba59b700 1 -- 172.21.15.7:0/3988186958 learned_addr learned my addr 172.21.15.7:0/3988186958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.211+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.212+0000 7fc0ba59b700 1 -- 172.21.15.7:0/3988186958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 msgr2=0x55eb154dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.212+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.212+0000 7fc0ba59b700 1 -- 172.21.15.7:0/3988186958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 msgr2=0x55eb154f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.212+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.212+0000 7fc0ba59b700 1 -- 172.21.15.7:0/3988186958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb1525bb00 con 0x55eb154e6f20 2024-01-11T18:04:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.213+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:04:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.213+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f81a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc0b009ac50 tx=0x7fc0b00a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510c3b98b0099d50 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.213+0000 7fc09ffff700 1 -- 172.21.15.7:0/3988186958 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00057f0 con 0x55eb154e6f20 2024-01-11T18:04:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.214+0000 7fc09ffff700 1 -- 172.21.15.7:0/3988186958 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b000ae00 con 0x55eb154e6f20 2024-01-11T18:04:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.214+0000 7fc09ffff700 1 -- 172.21.15.7:0/3988186958 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00d1a60 con 0x55eb154e6f20 2024-01-11T18:04:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3988186958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 msgr2=0x55eb154f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f81a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc0b009ac50 tx=0x7fc0b00a1040 comp rx=0 tx=0).stop 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3988186958 shutdown_connections 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f81a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/3988186958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.215+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3988186958 >> 172.21.15.7:0/3988186958 conn(0x55eb1545c740 msgr2=0x55eb154e1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.216+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3988186958 shutdown_connections 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.216+0000 7fc0d044ab80 1 -- 172.21.15.7:0/3988186958 wait complete. 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.217+0000 7fc0d044ab80 1 -- start start 2024-01-11T18:04:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.217+0000 7fc0d044ab80 1 Processor -- start 2024-01-11T18:04:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb154f6700 con 0x55eb154e7c10 2024-01-11T18:04:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb154f6870 con 0x55eb154e6f20 2024-01-11T18:04:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb154f6a30 con 0x55eb154e85d0 2024-01-11T18:04:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.218+0000 7fc0d044ab80 10 client.0 _ll_register_callbacks cb 0x55eb153db6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.219+0000 7fc0ba59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.220+0000 7fc0ba59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35340/0 (socket says 172.21.15.7:35340) 2024-01-11T18:04:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.220+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1944319660 learned_addr learned my addr 172.21.15.7:0/1944319660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.220+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.220+0000 7fc0b9d9a700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.221+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1944319660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 msgr2=0x55eb154f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.221+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.221+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1944319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 msgr2=0x55eb154f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.221+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.221+0000 7fc0ba59b700 1 -- 172.21.15.7:0/1944319660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb152af350 con 0x55eb154e6f20 2024-01-11T18:04:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.221+0000 7fc0bad9c700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:04:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.222+0000 7fc0ba59b700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f4c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0b009aee0 tx=0x7fc0b0007880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ae25d789b492c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.222+0000 7fc09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:04:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.222+0000 7fc09dffb700 1 -- 172.21.15.7:0/1944319660 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00d4c90 con 0x55eb154e6f20 2024-01-11T18:04:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.223+0000 7fc09dffb700 1 -- 172.21.15.7:0/1944319660 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc0b000ce40 con 0x55eb154e6f20 2024-01-11T18:04:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.223+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb15433880 con 0x55eb154e6f20 2024-01-11T18:04:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.223+0000 7fc09dffb700 1 -- 172.21.15.7:0/1944319660 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0b00d4800 con 0x55eb154e6f20 2024-01-11T18:04:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.223+0000 7fc0d044ab80 10 client.26102 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.223+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55eb153fc1d0 con 0x55eb154e6f20 2024-01-11T18:04:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc09dffb700 1 -- 172.21.15.7:0/1944319660 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b00a9080 con 0x55eb154e6f20 2024-01-11T18:04:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc09dffb700 1 -- 172.21.15.7:0/1944319660 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc0b0097660 con 0x55eb154e6f20 2024-01-11T18:04:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc0d044ab80 10 client.26102 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc0d044ab80 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc0d044ab80 20 client.26102 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb1555b180 0x55eb1555d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc0d044ab80 4 client.26102 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:04:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.224+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eb1527cc90 con 0x55eb1555b180 2024-01-11T18:04:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.225+0000 7fc0b9d9a700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb1555b180 0x55eb1555d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.226+0000 7fc09dffb700 10 client.26102 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.226+0000 7fc0b9d9a700 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb1555b180 0x55eb1555d570 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2372637f47fb675e server_cookie=ee5c1fa70ad761e8 in_seq=0 out_seq=0 2024-01-11T18:04:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.269+0000 7fc09dffb700 1 -- 172.21.15.7:0/1944319660 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eb1527cc90 con 0x55eb1555b180 2024-01-11T18:04:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.269+0000 7fc09dffb700 10 client.26102 handle_command_reply: tid=0 2024-01-11T18:04:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.269+0000 7fc0d044ab80 1 client.26102 shutdown 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 msgr2=0x55eb154f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f4c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0b009aee0 tx=0x7fc0b0007880 comp rx=0 tx=0).stop 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 shutdown_connections 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55eb1555b180 0x55eb1555d570 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55eb154e85d0 0x55eb154f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb154e6f20 0x55eb154f4c70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 --2- 172.21.15.7:0/1944319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb154e7c10 0x55eb154f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 >> 172.21.15.7:0/1944319660 conn(0x55eb1545c740 msgr2=0x55eb155031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.270+0000 7fc09dffb700 0 client.26102 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.274+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 shutdown_connections 2024-01-11T18:04:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.274+0000 7fc0d044ab80 1 -- 172.21.15.7:0/1944319660 wait complete. 2024-01-11T18:04:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.274+0000 7fc0d044ab80 20 client.26102 trim_cache size 0 max 16384 2024-01-11T18:04:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:04:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:04:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:04:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:04:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.277+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc0a00a20a0 msgr2=0x7fc0a00a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.277+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc0a00a20a0 0x7fc0a00a4560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55eb154e6cc0 tx=0x7fc0ac007720 comp rx=0 tx=0).stop 2024-01-11T18:04:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.277+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 msgr2=0x7fc0b41b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.277+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 0x7fc0b41b5930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000a5c0 tx=0x7fc0b000a6a0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 shutdown_connections 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc0a00a20a0 0x7fc0a00a4560 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0b4078d00 0x7fc0b41ba510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b4073450 0x7fc0b41b5930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 --2- 172.21.15.7:0/854822800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b4076330 0x7fc0b41b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 >> 172.21.15.7:0/854822800 conn(0x7fc0b406e500 msgr2=0x7fc0b40767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 shutdown_connections 2024-01-11T18:04:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:27.278+0000 7fc0bc7ff700 1 -- 172.21.15.7:0/854822800 wait complete. 2024-01-11T18:04:28.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:27 smithi016 ceph-mon[43195]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-01-11T18:04:28.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:27 smithi038 ceph-mon[43582]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-01-11T18:04:28.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:27 smithi007 ceph-mon[33361]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-01-11T18:04:29.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 360.0 s, 619.8 tps, lat 157.676 ms stddev 211.126 2024-01-11T18:04:30.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:29 smithi016 ceph-mon[43195]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-01-11T18:04:30.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:29 smithi038 ceph-mon[43582]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-01-11T18:04:30.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:29 smithi007 ceph-mon[33361]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-01-11T18:04:31.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:30 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:04:31.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:30 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:04:31.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:30 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:04:32.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:31 smithi016 ceph-mon[43195]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-01-11T18:04:32.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:31 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:04:32.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:31 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:04:32.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:31 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:04:32.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:31 smithi038 ceph-mon[43582]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-01-11T18:04:32.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:31 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:04:32.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:31 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:04:32.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:31 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:04:32.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:31 smithi007 ceph-mon[33361]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-01-11T18:04:32.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:31 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:04:32.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:31 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:04:32.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:31 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:04:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:32 smithi016 ceph-mon[43195]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-01-11T18:04:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:32 smithi038 ceph-mon[43582]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-01-11T18:04:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:32 smithi007 ceph-mon[33361]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-01-11T18:04:34.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 365.0 s, 561.4 tps, lat 178.035 ms stddev 208.547 2024-01-11T18:04:36.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:35 smithi016 ceph-mon[43195]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 8.9 MiB/s wr, 622 op/s 2024-01-11T18:04:36.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:35 smithi038 ceph-mon[43582]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 8.9 MiB/s wr, 622 op/s 2024-01-11T18:04:36.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:35 smithi007 ceph-mon[33361]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 8.9 MiB/s wr, 622 op/s 2024-01-11T18:04:38.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:37 smithi016 ceph-mon[43195]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.4 MiB/s wr, 620 op/s 2024-01-11T18:04:38.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:37 smithi038 ceph-mon[43582]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.4 MiB/s wr, 620 op/s 2024-01-11T18:04:38.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:37 smithi007 ceph-mon[33361]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.4 MiB/s wr, 620 op/s 2024-01-11T18:04:39.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 370.0 s, 659.6 tps, lat 150.016 ms stddev 191.114 2024-01-11T18:04:40.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:39 smithi016 ceph-mon[43195]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.8 MiB/s wr, 542 op/s 2024-01-11T18:04:40.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:39 smithi038 ceph-mon[43582]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.8 MiB/s wr, 542 op/s 2024-01-11T18:04:40.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:39 smithi007 ceph-mon[33361]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.8 MiB/s wr, 542 op/s 2024-01-11T18:04:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 -- 172.21.15.7:0/3032699155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 msgr2=0x7fdc1010bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 --2- 172.21.15.7:0/3032699155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 0x7fdc1010bc10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc080098f0 tx=0x7fdc08009c00 comp rx=0 tx=0).stop 2024-01-11T18:04:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 -- 172.21.15.7:0/3032699155 shutdown_connections 2024-01-11T18:04:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 --2- 172.21.15.7:0/3032699155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 0x7fdc1010bc10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 --2- 172.21.15.7:0/3032699155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc10103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 --2- 172.21.15.7:0/3032699155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc101049c0 0x7fdc10104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.519+0000 7fdc153ff700 1 -- 172.21.15.7:0/3032699155 >> 172.21.15.7:0/3032699155 conn(0x7fdc100fed60 msgr2=0x7fdc10101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.520+0000 7fdc153ff700 1 -- 172.21.15.7:0/3032699155 shutdown_connections 2024-01-11T18:04:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.520+0000 7fdc153ff700 1 -- 172.21.15.7:0/3032699155 wait complete. 2024-01-11T18:04:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.521+0000 7fdc153ff700 1 Processor -- start 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 -- start start 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc1019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc101049c0 0x7fdc1019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 0x7fdc101a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc101a5650 con 0x7fdc101049c0 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc101a5790 con 0x7fdc101037d0 2024-01-11T18:04:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.522+0000 7fdc153ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc101a5920 con 0x7fdc10105360 2024-01-11T18:04:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc1019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc1019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50872/0 (socket says 172.21.15.7:50872) 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 -- 172.21.15.7:0/240429540 learned_addr learned my addr 172.21.15.7:0/240429540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc067fc700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc101049c0 0x7fdc1019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 -- 172.21.15.7:0/240429540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 msgr2=0x7fdc101a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 0x7fdc101a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 -- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc101049c0 msgr2=0x7fdc1019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc101049c0 0x7fdc1019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.523+0000 7fdc0effd700 1 -- 172.21.15.7:0/240429540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08009520 con 0x7fdc101037d0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.524+0000 7fdc0effd700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc1019dad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e9f0 tx=0x7fdc0000edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.524+0000 7fdc0cff9700 1 -- 172.21.15.7:0/240429540 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00004220 con 0x7fdc101037d0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.525+0000 7fdc0cff9700 1 -- 172.21.15.7:0/240429540 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0000ce10 con 0x7fdc101037d0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.525+0000 7fdc0cff9700 1 -- 172.21.15.7:0/240429540 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00003d00 con 0x7fdc101037d0 2024-01-11T18:04:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.525+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1006b090 con 0x7fdc101037d0 2024-01-11T18:04:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.525+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1006b4f0 con 0x7fdc101037d0 2024-01-11T18:04:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.535+0000 7fdc291bab80 1 Processor -- start 2024-01-11T18:04:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.537+0000 7fdc291bab80 1 -- start start 2024-01-11T18:04:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.538+0000 7fdc0cff9700 1 -- 172.21.15.7:0/240429540 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fdc00018430 con 0x7fdc101037d0 2024-01-11T18:04:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.538+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c20f20 0x55a725c21340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c28e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc291bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725b18030 con 0x55a725c20f20 2024-01-11T18:04:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc291bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725b7bac0 con 0x55a725c21c10 2024-01-11T18:04:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc291bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725b793e0 con 0x55a725c225d0 2024-01-11T18:04:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc0effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc0effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c22090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50886/0 (socket says 172.21.15.7:50886) 2024-01-11T18:04:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc0effd700 1 -- 172.21.15.7:0/1541525274 learned_addr learned my addr 172.21.15.7:0/1541525274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.539+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c20f20 0x55a725c21340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.540+0000 7fdc0effd700 1 -- 172.21.15.7:0/1541525274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 msgr2=0x55a725c28e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.540+0000 7fdc0effd700 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c28e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.540+0000 7fdc0effd700 1 -- 172.21.15.7:0/1541525274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c20f20 msgr2=0x55a725c21340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.540+0000 7fdc0effd700 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c20f20 0x55a725c21340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.540+0000 7fdc0effd700 1 -- 172.21.15.7:0/1541525274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7259e9350 con 0x55a725c21c10 2024-01-11T18:04:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.541+0000 7fdc0cff9700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdbfc0b2fa0 0x7fdbfc0b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.541+0000 7fdc0effd700 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c22090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc000d1df0 tx=0x7fdc000d1e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef63d6234973d874 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.542+0000 7fdc0cff9700 1 -- 172.21.15.7:0/240429540 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc00014070 con 0x7fdc101037d0 2024-01-11T18:04:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.542+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdbfc0b2fa0 0x7fdbfc0b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.542+0000 7fdc057fa700 1 -- 172.21.15.7:0/1541525274 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000d9070 con 0x55a725c21c10 2024-01-11T18:04:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.542+0000 7fdc057fa700 1 -- 172.21.15.7:0/1541525274 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc00096cf0 con 0x55a725c21c10 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.543+0000 7fdc291bab80 1 -- 172.21.15.7:0/1541525274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 msgr2=0x55a725c22090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.543+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c22090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc000d1df0 tx=0x7fdc000d1e20 comp rx=0 tx=0).stop 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.543+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdbfc0b2fa0 0x7fdbfc0b5460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdc080098c0 tx=0x7fdc08019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.544+0000 7fdc291bab80 1 -- 172.21.15.7:0/1541525274 shutdown_connections 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.544+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c28e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.544+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c20f20 0x55a725c21340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.544+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1541525274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c22090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.544+0000 7fdc291bab80 1 -- 172.21.15.7:0/1541525274 >> 172.21.15.7:0/1541525274 conn(0x55a725b96740 msgr2=0x55a725c1b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.546+0000 7fdc291bab80 1 -- 172.21.15.7:0/1541525274 shutdown_connections 2024-01-11T18:04:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.547+0000 7fdc291bab80 1 -- 172.21.15.7:0/1541525274 wait complete. 2024-01-11T18:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.547+0000 7fdc291bab80 1 -- start start 2024-01-11T18:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.548+0000 7fdc291bab80 1 Processor -- start 2024-01-11T18:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.549+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 0x55a725c2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.552+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c21c10 0x55a725c2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.552+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c31f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 0x55a725c2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc291bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725b74cd0 con 0x55a725c21c10 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc291bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725ae3340 con 0x55a725c20f20 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc291bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725ae2fe0 con 0x55a725c225d0 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 0x55a725c2a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50896/0 (socket says 172.21.15.7:50896) 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 -- 172.21.15.7:0/1667528911 learned_addr learned my addr 172.21.15.7:0/1667528911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc291bab80 10 client.0 _ll_register_callbacks cb 0x55a725b156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c31f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc067fc700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c21c10 0x55a725c2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 -- 172.21.15.7:0/1667528911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 msgr2=0x55a725c31f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c31f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 -- 172.21.15.7:0/1667528911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c21c10 msgr2=0x55a725c2af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c21c10 0x55a725c2af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.553+0000 7fdc0effd700 1 -- 172.21.15.7:0/1667528911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1006b4f0 con 0x55a725c20f20 2024-01-11T18:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.554+0000 7fdc067fc700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c21c10 0x55a725c2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.554+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c31f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.554+0000 7fdc0effd700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 0x55a725c2a9e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e9c0 tx=0x7fdc0009a4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fbee2b79b337c78 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.554+0000 7fdbf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.554+0000 7fdbf67fc700 1 -- 172.21.15.7:0/1667528911 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00003680 con 0x55a725c20f20 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.554+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a725c13d60 con 0x55a725c20f20 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.555+0000 7fdbf67fc700 1 -- 172.21.15.7:0/1667528911 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc0009b0b0 con 0x55a725c20f20 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.555+0000 7fdbf67fc700 1 -- 172.21.15.7:0/1667528911 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc00003680 con 0x55a725c20f20 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.555+0000 7fdc291bab80 10 client.26114 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.556+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a725995b00 con 0x55a725c20f20 2024-01-11T18:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.556+0000 7fdbf67fc700 1 -- 172.21.15.7:0/1667528911 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a725c13d60 con 0x55a725c20f20 2024-01-11T18:04:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.558+0000 7fdbf67fc700 1 -- 172.21.15.7:0/1667528911 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdc00085070 con 0x55a725c20f20 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.560+0000 7fdc291bab80 10 client.26114 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.560+0000 7fdc291bab80 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.560+0000 7fdc291bab80 20 client.26114 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.561+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c36420 0x55a725c38840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.561+0000 7fdc291bab80 4 client.26114 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.561+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a725b6db60 con 0x55a725c36420 2024-01-11T18:04:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.561+0000 7fdc067fc700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c36420 0x55a725c38840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.562+0000 7fdc067fc700 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c36420 0x55a725c38840 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76688d91baae2f8 server_cookie=b9c9608ac09fd4f1 in_seq=0 out_seq=0 2024-01-11T18:04:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.562+0000 7fdbf67fc700 10 client.26114 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.563+0000 7fdbf67fc700 1 -- 172.21.15.7:0/1667528911 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a725b6db60 con 0x55a725c36420 2024-01-11T18:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.563+0000 7fdbf67fc700 10 client.26114 handle_command_reply: tid=0 2024-01-11T18:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.563+0000 7fdc291bab80 1 client.26114 shutdown 2024-01-11T18:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 msgr2=0x55a725c2a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 0x55a725c2a9e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e9c0 tx=0x7fdc0009a4f0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 shutdown_connections 2024-01-11T18:04:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c36420 0x55a725c38840 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c225d0 0x55a725c31f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c20f20 0x55a725c2a9e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 --2- 172.21.15.7:0/1667528911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c21c10 0x55a725c2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 >> 172.21.15.7:0/1667528911 conn(0x55a725b96740 msgr2=0x55a725c1af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdbf67fc700 0 client.26114 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 shutdown_connections 2024-01-11T18:04:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.564+0000 7fdc291bab80 1 -- 172.21.15.7:0/1667528911 wait complete. 2024-01-11T18:04:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.565+0000 7fdc291bab80 20 client.26114 trim_cache size 0 max 16384 2024-01-11T18:04:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.601+0000 7fdc291bab80 1 Processor -- start 2024-01-11T18:04:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.601+0000 7fdc291bab80 1 -- start start 2024-01-11T18:04:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.602+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c29f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c31f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c1af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc291bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7259eb640 con 0x55a725c225d0 2024-01-11T18:04:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc291bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725b6db60 con 0x55a725c21c10 2024-01-11T18:04:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc291bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725c18560 con 0x55a725c20f20 2024-01-11T18:04:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c31f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c31f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50902/0 (socket says 172.21.15.7:50902) 2024-01-11T18:04:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.603+0000 7fdc067fc700 1 -- 172.21.15.7:0/4060291626 learned_addr learned my addr 172.21.15.7:0/4060291626 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.604+0000 7fdc067fc700 1 -- 172.21.15.7:0/4060291626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 msgr2=0x55a725c29f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:04:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.604+0000 7fdc067fc700 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c29f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.604+0000 7fdc067fc700 1 -- 172.21.15.7:0/4060291626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 msgr2=0x55a725c1af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.604+0000 7fdc067fc700 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c1af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.604+0000 7fdc067fc700 1 -- 172.21.15.7:0/4060291626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a725b361d0 con 0x55a725c21c10 2024-01-11T18:04:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.604+0000 7fdc067fc700 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c31f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55a725c39100 tx=0x7fdbf8009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195bf44f6299a925 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.606+0000 7fdc04ff9700 1 -- 172.21.15.7:0/4060291626 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8004bb0 con 0x55a725c21c10 2024-01-11T18:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.607+0000 7fdc04ff9700 1 -- 172.21.15.7:0/4060291626 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf8004d10 con 0x55a725c21c10 2024-01-11T18:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.607+0000 7fdc04ff9700 1 -- 172.21.15.7:0/4060291626 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8017410 con 0x55a725c21c10 2024-01-11T18:04:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 -- 172.21.15.7:0/4060291626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 msgr2=0x55a725c31f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c31f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55a725c39100 tx=0x7fdbf8009040 comp rx=0 tx=0).stop 2024-01-11T18:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 -- 172.21.15.7:0/4060291626 shutdown_connections 2024-01-11T18:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c29f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c1af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 --2- 172.21.15.7:0/4060291626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c31f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 -- 172.21.15.7:0/4060291626 >> 172.21.15.7:0/4060291626 conn(0x55a725b96740 msgr2=0x55a725c179b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 -- 172.21.15.7:0/4060291626 shutdown_connections 2024-01-11T18:04:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.608+0000 7fdc291bab80 1 -- 172.21.15.7:0/4060291626 wait complete. 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.609+0000 7fdc291bab80 1 -- start start 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.609+0000 7fdc291bab80 1 Processor -- start 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c2fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c2ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c30f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725c314a0 con 0x55a725c225d0 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c2ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725c31610 con 0x55a725c21c10 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a725c317d0 con 0x55a725c20f20 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c2ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50918/0 (socket says 172.21.15.7:50918) 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc067fc700 1 -- 172.21.15.7:0/2678135590 learned_addr learned my addr 172.21.15.7:0/2678135590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc291bab80 10 client.0 _ll_register_callbacks cb 0x55a725b156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc0effd700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c2fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c30f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc0effd700 1 -- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 msgr2=0x55a725c2ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.610+0000 7fdc0effd700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c2ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdc0effd700 1 -- 172.21.15.7:0/2678135590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 msgr2=0x55a725c30f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdc0effd700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c30f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdc0effd700 1 -- 172.21.15.7:0/2678135590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf8011040 con 0x55a725c20f20 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdc0f7fe700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c30f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdc0effd700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c2fa80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdc00098d00 tx=0x7fdc0009a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8b8b9748c54fa3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdbf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdbf67fc700 1 -- 172.21.15.7:0/2678135590 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000d4790 con 0x55a725c20f20 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.611+0000 7fdbf67fc700 1 -- 172.21.15.7:0/2678135590 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc0009b0b0 con 0x55a725c20f20 2024-01-11T18:04:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.612+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc00096cf0 con 0x55a725c20f20 2024-01-11T18:04:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.612+0000 7fdbf67fc700 1 -- 172.21.15.7:0/2678135590 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc000ce080 con 0x55a725c20f20 2024-01-11T18:04:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.613+0000 7fdc291bab80 10 client.26152 fetch_fsmap learned FSMap version 35 2024-01-11T18:04:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.613+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a7259edc70 con 0x55a725c20f20 2024-01-11T18:04:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.614+0000 7fdbf67fc700 1 -- 172.21.15.7:0/2678135590 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc00096cf0 con 0x55a725c20f20 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdbf67fc700 1 -- 172.21.15.7:0/2678135590 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdc00099ea0 con 0x55a725c20f20 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdc291bab80 10 client.26152 fetch_fsmap finished waiting for FSMap version 35 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdc291bab80 10 client.26152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdc291bab80 20 client.26152 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdc291bab80 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c8d010 0x55a725c8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdc291bab80 4 client.26152 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.615+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a725b6d880 con 0x55a725c8d010 2024-01-11T18:04:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.616+0000 7fdc067fc700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c8d010 0x55a725c8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.619+0000 7fdbf67fc700 10 client.26152 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.619+0000 7fdc067fc700 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c8d010 0x55a725c8f400 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7132ba43b1f191 server_cookie=3b2d8e6b6b0ced93 in_seq=0 out_seq=0 2024-01-11T18:04:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.633+0000 7fdbf67fc700 1 -- 172.21.15.7:0/2678135590 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a725b6d880 con 0x55a725c8d010 2024-01-11T18:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.633+0000 7fdbf67fc700 10 client.26152 handle_command_reply: tid=0 2024-01-11T18:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.633+0000 7fdc291bab80 1 client.26152 shutdown 2024-01-11T18:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 msgr2=0x55a725c2fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c2fa80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdc00098d00 tx=0x7fdc0009a410 comp rx=0 tx=0).stop 2024-01-11T18:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 shutdown_connections 2024-01-11T18:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a725c8d010 0x55a725c8f400 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a725c20f20 0x55a725c2fa80 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdc00098d00 tx=0x7fdc0009a410 comp rx=0 tx=0).stop 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a725c225d0 0x55a725c30f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdbf67fc700 0 client.26152 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 --2- 172.21.15.7:0/2678135590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a725c21c10 0x55a725c2ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.634+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 >> 172.21.15.7:0/2678135590 conn(0x55a725b96740 msgr2=0x55a725c1af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.635+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 shutdown_connections 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.635+0000 7fdc291bab80 1 -- 172.21.15.7:0/2678135590 wait complete. 2024-01-11T18:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.635+0000 7fdc291bab80 20 client.26152 trim_cache size 0 max 16384 2024-01-11T18:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.636+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdbfc0b2fa0 msgr2=0x7fdbfc0b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.636+0000 7fdc153ff700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdbfc0b2fa0 0x7fdbfc0b5460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdc080098c0 tx=0x7fdc08019040 comp rx=0 tx=0).stop 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.636+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 msgr2=0x7fdc1019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.636+0000 7fdc153ff700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc1019dad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e9f0 tx=0x7fdc0000edb0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 shutdown_connections 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdbfc0b2fa0 0x7fdbfc0b5460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc10105360 0x7fdc101a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc101037d0 0x7fdc1019dad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 --2- 172.21.15.7:0/240429540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc101049c0 0x7fdc1019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 >> 172.21.15.7:0/240429540 conn(0x7fdc100fed60 msgr2=0x7fdc100ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 shutdown_connections 2024-01-11T18:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:40.637+0000 7fdc153ff700 1 -- 172.21.15.7:0/240429540 wait complete. 2024-01-11T18:04:42.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:41 smithi016 ceph-mon[43195]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.0 MiB/s wr, 628 op/s 2024-01-11T18:04:42.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:41 smithi038 ceph-mon[43582]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.0 MiB/s wr, 628 op/s 2024-01-11T18:04:42.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:41 smithi007 ceph-mon[33361]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.0 MiB/s wr, 628 op/s 2024-01-11T18:04:43.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:42 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:04:43.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:42 smithi016 ceph-mon[43195]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.0 MiB/s wr, 619 op/s 2024-01-11T18:04:43.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:42 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:04:43.275 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:42 smithi038 ceph-mon[43582]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.0 MiB/s wr, 619 op/s 2024-01-11T18:04:43.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:42 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:04:43.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:42 smithi007 ceph-mon[33361]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.0 MiB/s wr, 619 op/s 2024-01-11T18:04:44.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 375.0 s, 518.0 tps, lat 194.709 ms stddev 238.763 2024-01-11T18:04:45.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:45 smithi016 ceph-mon[43195]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.6 MiB/s wr, 521 op/s 2024-01-11T18:04:46.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:45 smithi038 ceph-mon[43582]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.6 MiB/s wr, 521 op/s 2024-01-11T18:04:46.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:45 smithi007 ceph-mon[33361]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.6 MiB/s wr, 521 op/s 2024-01-11T18:04:47.990 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:47 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:04:47.990 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:47 smithi016 ceph-mon[43195]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 561 op/s 2024-01-11T18:04:48.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:47 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:04:48.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:47 smithi038 ceph-mon[43582]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 561 op/s 2024-01-11T18:04:48.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:47 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:04:48.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:47 smithi007 ceph-mon[33361]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 561 op/s 2024-01-11T18:04:49.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 380.0 s, 530.6 tps, lat 188.265 ms stddev 227.511 2024-01-11T18:04:50.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:49 smithi016 ceph-mon[43195]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 502 op/s 2024-01-11T18:04:50.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:49 smithi038 ceph-mon[43582]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 502 op/s 2024-01-11T18:04:50.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:49 smithi007 ceph-mon[33361]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 502 op/s 2024-01-11T18:04:52.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:51 smithi016 ceph-mon[43195]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 569 op/s 2024-01-11T18:04:52.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:51 smithi038 ceph-mon[43582]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 569 op/s 2024-01-11T18:04:52.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:51 smithi007 ceph-mon[33361]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 569 op/s 2024-01-11T18:04:53.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:52 smithi016 ceph-mon[43195]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 574 op/s 2024-01-11T18:04:53.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:52 smithi038 ceph-mon[43582]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 574 op/s 2024-01-11T18:04:53.318 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:52 smithi007 ceph-mon[33361]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 574 op/s 2024-01-11T18:04:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.990+0000 7fc27d1ff700 1 -- 172.21.15.7:0/748316653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 msgr2=0x7fc278108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.990+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/748316653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc278108610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc270009f90 tx=0x7fc27000b330 comp rx=0 tx=0).stop 2024-01-11T18:04:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 -- 172.21.15.7:0/748316653 shutdown_connections 2024-01-11T18:04:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/748316653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27810fa80 0x7fc278111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/748316653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc278108610 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/748316653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278108c00 0x7fc278109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 -- 172.21.15.7:0/748316653 >> 172.21.15.7:0/748316653 conn(0x7fc27806e500 msgr2=0x7fc278070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 -- 172.21.15.7:0/748316653 shutdown_connections 2024-01-11T18:04:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.993+0000 7fc27d1ff700 1 -- 172.21.15.7:0/748316653 wait complete. 2024-01-11T18:04:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 Processor -- start 2024-01-11T18:04:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 -- start start 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc2781a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278108c00 0x7fc2781a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27810fa80 0x7fc2781a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a9b90 con 0x7fc278073430 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a9cd0 con 0x7fc27810fa80 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.995+0000 7fc27d1ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a9e60 con 0x7fc278108c00 2024-01-11T18:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.996+0000 7fc276d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc2781a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.996+0000 7fc27759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27810fa80 0x7fc2781a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.996+0000 7fc276d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc2781a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41612/0 (socket says 172.21.15.7:41612) 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.996+0000 7fc276d9d700 1 -- 172.21.15.7:0/2491815354 learned_addr learned my addr 172.21.15.7:0/2491815354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.996+0000 7fc276d9d700 1 -- 172.21.15.7:0/2491815354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278108c00 msgr2=0x7fc2781a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.997+0000 7fc276d9d700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278108c00 0x7fc2781a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.997+0000 7fc276d9d700 1 -- 172.21.15.7:0/2491815354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27810fa80 msgr2=0x7fc2781a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.997+0000 7fc276d9d700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27810fa80 0x7fc2781a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.997+0000 7fc276d9d700 1 -- 172.21.15.7:0/2491815354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009c40 con 0x7fc278073430 2024-01-11T18:04:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.998+0000 7fc276d9d700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc2781a2010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc27000b6c0 tx=0x7fc270004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.999+0000 7fc25ffff700 1 -- 172.21.15.7:0/2491815354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000e040 con 0x7fc278073430 2024-01-11T18:04:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.999+0000 7fc25ffff700 1 -- 172.21.15.7:0/2491815354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27000be00 con 0x7fc278073430 2024-01-11T18:04:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.999+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2781aa0e0 con 0x7fc278073430 2024-01-11T18:04:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.999+0000 7fc25ffff700 1 -- 172.21.15.7:0/2491815354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270005400 con 0x7fc278073430 2024-01-11T18:04:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:53.999+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2781aa5b0 con 0x7fc278073430 2024-01-11T18:04:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 Processor -- start 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 -- start start 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a901ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901ff0 0x55842a9024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a902f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a7f0030 con 0x55842a901690 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a853ac0 con 0x55842a901ff0 2024-01-11T18:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.006+0000 7fc290f57b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a8513e0 con 0x55842a902ad0 2024-01-11T18:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a902f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a902f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:40294/0 (socket says 172.21.15.7:40294) 2024-01-11T18:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 -- 172.21.15.7:0/3040402683 learned_addr learned my addr 172.21.15.7:0/3040402683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27759e700 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a901ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 -- 172.21.15.7:0/3040402683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901ff0 msgr2=0x55842a9024f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901ff0 0x55842a9024f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 -- 172.21.15.7:0/3040402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 msgr2=0x55842a901ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a901ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 -- 172.21.15.7:0/3040402683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55842a6c1350 con 0x55842a902ad0 2024-01-11T18:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.008+0000 7fc27659c700 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a902f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009bc0 tx=0x7fc26c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455bdc4c8e4a215b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.009+0000 7fc25d7fa700 1 -- 172.21.15.7:0/3040402683 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00eb60 con 0x55842a902ad0 2024-01-11T18:04:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.009+0000 7fc25d7fa700 1 -- 172.21.15.7:0/3040402683 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc26c004cf0 con 0x55842a902ad0 2024-01-11T18:04:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.009+0000 7fc25d7fa700 1 -- 172.21.15.7:0/3040402683 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c005720 con 0x55842a902ad0 2024-01-11T18:04:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.010+0000 7fc290f57b80 1 -- 172.21.15.7:0/3040402683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 msgr2=0x55842a902f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.010+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a902f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009bc0 tx=0x7fc26c009ed0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.011+0000 7fc290f57b80 1 -- 172.21.15.7:0/3040402683 shutdown_connections 2024-01-11T18:04:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.011+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a902f50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.011+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a901ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.011+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3040402683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901ff0 0x55842a9024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.011+0000 7fc290f57b80 1 -- 172.21.15.7:0/3040402683 >> 172.21.15.7:0/3040402683 conn(0x55842a86e740 msgr2=0x55842a8f3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.012+0000 7fc25ffff700 1 -- 172.21.15.7:0/2491815354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc27001daa0 con 0x7fc278073430 2024-01-11T18:04:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.016+0000 7fc290f57b80 1 -- 172.21.15.7:0/3040402683 shutdown_connections 2024-01-11T18:04:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.016+0000 7fc290f57b80 1 -- 172.21.15.7:0/3040402683 wait complete. 2024-01-11T18:04:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.017+0000 7fc290f57b80 1 -- start start 2024-01-11T18:04:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.017+0000 7fc290f57b80 1 Processor -- start 2024-01-11T18:04:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.019+0000 7fc25ffff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2600a1e60 0x7fc2600a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.019+0000 7fc25ffff700 1 -- 172.21.15.7:0/2491815354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2700cf260 con 0x7fc278073430 2024-01-11T18:04:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.019+0000 7fc27759e700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2600a1e60 0x7fc2600a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.020+0000 7fc27759e700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2600a1e60 0x7fc2600a4320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc268005950 tx=0x7fc2680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a8fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a8fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a8ffb90 0x55842a900010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a84ccd0 con 0x55842a901690 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a7bb340 con 0x55842a902ad0 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a7bafe0 con 0x55842a8ffb90 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.022+0000 7fc290f57b80 10 client.0 _ll_register_callbacks cb 0x55842a7ed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a8fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a8ffb90 0x55842a900010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a8fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35862/0 (socket says 172.21.15.7:35862) 2024-01-11T18:04:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27659c700 1 -- 172.21.15.7:0/3487604741 learned_addr learned my addr 172.21.15.7:0/3487604741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27759e700 1 -- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 msgr2=0x55842a8fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27759e700 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a8fabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27759e700 1 -- 172.21.15.7:0/3487604741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 msgr2=0x55842a8fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27759e700 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a8fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.023+0000 7fc27759e700 1 -- 172.21.15.7:0/3487604741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c0098a0 con 0x55842a8ffb90 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.024+0000 7fc27659c700 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a8fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.024+0000 7fc27759e700 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a8ffb90 0x55842a900010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc268011ef0 tx=0x7fc268004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16a23a9f8f1c4d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.024+0000 7fc25aff5700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.025+0000 7fc25aff5700 1 -- 172.21.15.7:0/3487604741 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26801f620 con 0x55842a8ffb90 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.025+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55842a8ebd60 con 0x55842a8ffb90 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.025+0000 7fc25aff5700 1 -- 172.21.15.7:0/3487604741 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc268018070 con 0x55842a8ffb90 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.025+0000 7fc25aff5700 1 -- 172.21.15.7:0/3487604741 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26801f780 con 0x55842a8ffb90 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.025+0000 7fc290f57b80 10 client.26161 fetch_fsmap learned FSMap version 37 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.025+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55842a66db00 con 0x55842a8ffb90 2024-01-11T18:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.026+0000 7fc25aff5700 1 -- 172.21.15.7:0/3487604741 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55842a8ebd60 con 0x55842a8ffb90 2024-01-11T18:04:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.027+0000 7fc25aff5700 1 -- 172.21.15.7:0/3487604741 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc26802bbc0 con 0x55842a8ffb90 2024-01-11T18:04:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc290f57b80 10 client.26161 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:04:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc290f57b80 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc290f57b80 20 client.26161 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a9221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc290f57b80 4 client.26161 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:04:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55842a845b60 con 0x55842a91fdd0 2024-01-11T18:04:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.028+0000 7fc276d9d700 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a9221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.029+0000 7fc25aff5700 10 client.26161 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.030+0000 7fc276d9d700 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a9221f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290ecf3a725383d3 server_cookie=f19535ce2fadc17c in_seq=0 out_seq=0 2024-01-11T18:04:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.030+0000 7fc25aff5700 1 -- 172.21.15.7:0/3487604741 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55842a845b60 con 0x55842a91fdd0 2024-01-11T18:04:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.031+0000 7fc25aff5700 10 client.26161 handle_command_reply: tid=0 2024-01-11T18:04:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.031+0000 7fc290f57b80 1 client.26161 shutdown 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a8ffb90 msgr2=0x55842a900010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a8ffb90 0x55842a900010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc268011ef0 tx=0x7fc268004750 comp rx=0 tx=0).stop 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 shutdown_connections 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a9221f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a8ffb90 0x55842a900010 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a901690 0x55842a8fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc25aff5700 0 client.26161 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 --2- 172.21.15.7:0/3487604741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a8fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 >> 172.21.15.7:0/3487604741 conn(0x55842a86e740 msgr2=0x55842a8f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 shutdown_connections 2024-01-11T18:04:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 1 -- 172.21.15.7:0/3487604741 wait complete. 2024-01-11T18:04:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.032+0000 7fc290f57b80 20 client.26161 trim_cache size 0 max 16384 2024-01-11T18:04:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.067+0000 7fc290f57b80 1 Processor -- start 2024-01-11T18:04:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 -- start start 2024-01-11T18:04:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a902ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 0x55842a900620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a8f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a6c3640 con 0x55842a8ffb90 2024-01-11T18:04:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a8f0560 con 0x55842a901690 2024-01-11T18:04:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.071+0000 7fc290f57b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a8f3e60 con 0x55842a902ad0 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 0x55842a900620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 0x55842a900620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35868/0 (socket says 172.21.15.7:35868) 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 -- 172.21.15.7:0/1981150863 learned_addr learned my addr 172.21.15.7:0/1981150863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27759e700 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a8f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc276d9d700 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a902ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 -- 172.21.15.7:0/1981150863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 msgr2=0x55842a8f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a8f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 -- 172.21.15.7:0/1981150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 msgr2=0x55842a902ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a902ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.072+0000 7fc27659c700 1 -- 172.21.15.7:0/1981150863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55842a80e1d0 con 0x55842a901690 2024-01-11T18:04:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.073+0000 7fc276d9d700 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a902ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:04:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.073+0000 7fc27659c700 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 0x55842a900620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc26c000c00 tx=0x7fc26c005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac31ac2531756f83 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.073+0000 7fc25cff9700 1 -- 172.21.15.7:0/1981150863 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c005e20 con 0x55842a901690 2024-01-11T18:04:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.074+0000 7fc25cff9700 1 -- 172.21.15.7:0/1981150863 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc26c005370 con 0x55842a901690 2024-01-11T18:04:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.074+0000 7fc25cff9700 1 -- 172.21.15.7:0/1981150863 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c023480 con 0x55842a901690 2024-01-11T18:04:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 -- 172.21.15.7:0/1981150863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 msgr2=0x55842a900620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 0x55842a900620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc26c000c00 tx=0x7fc26c005990 comp rx=0 tx=0).stop 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 -- 172.21.15.7:0/1981150863 shutdown_connections 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a902ad0 0x55842a8f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a902ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 --2- 172.21.15.7:0/1981150863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a901690 0x55842a900620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 -- 172.21.15.7:0/1981150863 >> 172.21.15.7:0/1981150863 conn(0x55842a86e740 msgr2=0x55842a8f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.075+0000 7fc290f57b80 1 -- 172.21.15.7:0/1981150863 shutdown_connections 2024-01-11T18:04:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.076+0000 7fc290f57b80 1 -- 172.21.15.7:0/1981150863 wait complete. 2024-01-11T18:04:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.076+0000 7fc290f57b80 1 -- start start 2024-01-11T18:04:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.076+0000 7fc290f57b80 1 Processor -- start 2024-01-11T18:04:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.077+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a92d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.077+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a901690 0x55842a92d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.077+0000 7fc290f57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a974d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc290f57b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a92de30 con 0x55842a8ffb90 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc290f57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a92dfa0 con 0x55842a902ad0 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc290f57b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842a9752d0 con 0x55842a901690 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc290f57b80 10 client.0 _ll_register_callbacks cb 0x55842a7ed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a901690 0x55842a92d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a901690 0x55842a92d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27659c700 1 -- 172.21.15.7:0/2546349481 learned_addr learned my addr 172.21.15.7:0/2546349481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc276d9d700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a92d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27759e700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a974d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27759e700 1 -- 172.21.15.7:0/2546349481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a901690 msgr2=0x55842a92d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27759e700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a901690 0x55842a92d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27759e700 1 -- 172.21.15.7:0/2546349481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 msgr2=0x55842a92d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:04:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27759e700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a92d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.078+0000 7fc27759e700 1 -- 172.21.15.7:0/2546349481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c01c040 con 0x55842a902ad0 2024-01-11T18:04:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.079+0000 7fc27759e700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a974d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc268005d90 tx=0x7fc268027da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd910a7b044dcec9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:04:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.079+0000 7fc25aff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:04:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.079+0000 7fc25aff5700 1 -- 172.21.15.7:0/2546349481 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26802b620 con 0x55842a902ad0 2024-01-11T18:04:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.079+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc26c004cf0 con 0x55842a902ad0 2024-01-11T18:04:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.079+0000 7fc25aff5700 1 -- 172.21.15.7:0/2546349481 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc268018040 con 0x55842a902ad0 2024-01-11T18:04:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.081+0000 7fc25aff5700 1 -- 172.21.15.7:0/2546349481 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc268012ac0 con 0x55842a902ad0 2024-01-11T18:04:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.081+0000 7fc290f57b80 10 client.26135 fetch_fsmap learned FSMap version 37 2024-01-11T18:04:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.081+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55842a6c5c70 con 0x55842a902ad0 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc25aff5700 1 -- 172.21.15.7:0/2546349481 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc26c004cf0 con 0x55842a902ad0 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc25aff5700 1 -- 172.21.15.7:0/2546349481 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc26803a020 con 0x55842a902ad0 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc290f57b80 10 client.26135 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc290f57b80 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc290f57b80 20 client.26135 populate_metadata read hostname 'smithi007' 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc290f57b80 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a978c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc290f57b80 4 client.26135 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:04:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.082+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55842a845880 con 0x55842a91fdd0 2024-01-11T18:04:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.083+0000 7fc276d9d700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a978c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:04:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.083+0000 7fc276d9d700 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a978c20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dec15553529ae08 server_cookie=485fcc31f12fed4a in_seq=0 out_seq=0 2024-01-11T18:04:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.083+0000 7fc25aff5700 10 client.26135 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.097+0000 7fc25aff5700 1 -- 172.21.15.7:0/2546349481 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55842a845880 con 0x55842a91fdd0 2024-01-11T18:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.097+0000 7fc25aff5700 10 client.26135 handle_command_reply: tid=0 2024-01-11T18:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.097+0000 7fc290f57b80 1 client.26135 shutdown 2024-01-11T18:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.097+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 msgr2=0x55842a974d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.097+0000 7fc290f57b80 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a974d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc268005d90 tx=0x7fc268027da0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 shutdown_connections 2024-01-11T18:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55842a91fdd0 0x55842a978c20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55842a902ad0 0x55842a974d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55842a8ffb90 0x55842a92d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc25aff5700 0 client.26135 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 --2- 172.21.15.7:0/2546349481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55842a901690 0x55842a92d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 >> 172.21.15.7:0/2546349481 conn(0x55842a86e740 msgr2=0x55842a900620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 shutdown_connections 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 1 -- 172.21.15.7:0/2546349481 wait complete. 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.098+0000 7fc290f57b80 20 client.26135 trim_cache size 0 max 16384 2024-01-11T18:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2600a1e60 msgr2=0x7fc2600a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2600a1e60 0x7fc2600a4320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc268005950 tx=0x7fc2680058e0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 msgr2=0x7fc2781a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:04:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc2781a2010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc27000b6c0 tx=0x7fc270004d70 comp rx=0 tx=0).stop 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 shutdown_connections 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc2600a1e60 0x7fc2600a4320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27810fa80 0x7fc2781a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278073430 0x7fc2781a2010 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 --2- 172.21.15.7:0/2491815354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc278108c00 0x7fc2781a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.099+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 >> 172.21.15.7:0/2491815354 conn(0x7fc27806e500 msgr2=0x7fc278109720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.100+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 shutdown_connections 2024-01-11T18:04:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:04:54.100+0000 7fc27d1ff700 1 -- 172.21.15.7:0/2491815354 wait complete. 2024-01-11T18:04:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 385.0 s, 584.2 tps, lat 173.098 ms stddev 211.942 2024-01-11T18:04:55.991 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:55 smithi016 ceph-mon[43195]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 506 op/s 2024-01-11T18:04:56.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:55 smithi038 ceph-mon[43582]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 506 op/s 2024-01-11T18:04:56.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:55 smithi007 ceph-mon[33361]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 506 op/s 2024-01-11T18:04:58.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:57 smithi016 ceph-mon[43195]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 547 op/s 2024-01-11T18:04:58.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:57 smithi038 ceph-mon[43582]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 547 op/s 2024-01-11T18:04:58.872 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:57 smithi007 ceph-mon[33361]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 547 op/s 2024-01-11T18:04:59.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 390.0 s, 711.0 tps, lat 138.330 ms stddev 154.006 2024-01-11T18:05:00.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:04:59 smithi016 ceph-mon[43195]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2024-01-11T18:05:00.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:04:59 smithi038 ceph-mon[43582]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2024-01-11T18:05:00.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:04:59 smithi007 ceph-mon[33361]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2024-01-11T18:05:01.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:00 smithi016 ceph-mon[43195]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.6 MiB/s wr, 616 op/s 2024-01-11T18:05:01.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:00 smithi038 ceph-mon[43582]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.6 MiB/s wr, 616 op/s 2024-01-11T18:05:01.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:00 smithi007 ceph-mon[33361]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.6 MiB/s wr, 616 op/s 2024-01-11T18:05:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 395.0 s, 516.2 tps, lat 188.929 ms stddev 211.540 2024-01-11T18:05:04.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:03 smithi016 ceph-mon[43195]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-01-11T18:05:04.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:03 smithi038 ceph-mon[43582]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-01-11T18:05:04.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:03 smithi007 ceph-mon[33361]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-01-11T18:05:05.990 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:05 smithi016 ceph-mon[43195]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-01-11T18:05:06.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:05 smithi038 ceph-mon[43582]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-01-11T18:05:06.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:05 smithi007 ceph-mon[33361]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-01-11T18:05:08.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:07 smithi016 ceph-mon[43195]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.1 MiB/s wr, 590 op/s 2024-01-11T18:05:08.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:07 smithi038 ceph-mon[43582]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.1 MiB/s wr, 590 op/s 2024-01-11T18:05:08.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:07 smithi007 ceph-mon[33361]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.1 MiB/s wr, 590 op/s 2024-01-11T18:05:09.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 400.0 s, 460.4 tps, lat 226.608 ms stddev 280.375 2024-01-11T18:05:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.379+0000 7f82e9d9d700 1 -- 172.21.15.7:0/2701731617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc096890 msgr2=0x7f82dc098c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.379+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/2701731617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc096890 0x7f82dc098c80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009990 tx=0x7f82d8009ca0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.380+0000 7f82e9d9d700 1 -- 172.21.15.7:0/2701731617 shutdown_connections 2024-01-11T18:05:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.380+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/2701731617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc096890 0x7f82dc098c80 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009990 tx=0x7f82d8009ca0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.380+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/2701731617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc09baf0 0x7f82dc09df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.380+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/2701731617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc0991c0 0x7f82dc09b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.380+0000 7f82e9d9d700 1 -- 172.21.15.7:0/2701731617 >> 172.21.15.7:0/2701731617 conn(0x7f82dc090260 msgr2=0x7f82dc0926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.381+0000 7f82e9d9d700 1 -- 172.21.15.7:0/2701731617 shutdown_connections 2024-01-11T18:05:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.381+0000 7f82e9d9d700 1 -- 172.21.15.7:0/2701731617 wait complete. 2024-01-11T18:05:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.383+0000 7f82e9d9d700 1 Processor -- start 2024-01-11T18:05:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.383+0000 7f82e9d9d700 1 -- start start 2024-01-11T18:05:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e9d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 0x7f82dc12e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e9d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc09baf0 0x7f82dc12e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e9d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc133760 0x7f82dc135b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e9d9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc136120 con 0x7f82dc133760 2024-01-11T18:05:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e9d9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc136260 con 0x7f82dc09baf0 2024-01-11T18:05:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e9d9d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc1363f0 con 0x7f82dc0991c0 2024-01-11T18:05:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e8d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 0x7f82dc12e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.384+0000 7f82e8d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 0x7f82dc12e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:60052/0 (socket says 172.21.15.7:60052) 2024-01-11T18:05:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e8d9b700 1 -- 172.21.15.7:0/4204264538 learned_addr learned my addr 172.21.15.7:0/4204264538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e3fff700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc09baf0 0x7f82dc12e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e3fff700 1 -- 172.21.15.7:0/4204264538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 msgr2=0x7f82dc12e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e3fff700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 0x7f82dc12e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e3fff700 1 -- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc133760 msgr2=0x7f82dc135b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e3fff700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc133760 0x7f82dc135b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e3fff700 1 -- 172.21.15.7:0/4204264538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80095f0 con 0x7f82dc09baf0 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.385+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 0x7f82dc12e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.386+0000 7f82e3fff700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc09baf0 0x7f82dc12e700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82d000d950 tx=0x7f82d000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.386+0000 7f82e1ffb700 1 -- 172.21.15.7:0/4204264538 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0009930 con 0x7f82dc09baf0 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.386+0000 7f82e1ffb700 1 -- 172.21.15.7:0/4204264538 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82d0010460 con 0x7f82dc09baf0 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.386+0000 7f82e1ffb700 1 -- 172.21.15.7:0/4204264538 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000b5d0 con 0x7f82dc09baf0 2024-01-11T18:05:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.387+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc1366d0 con 0x7f82dc09baf0 2024-01-11T18:05:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.387+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc136b90 con 0x7f82dc09baf0 2024-01-11T18:05:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.395+0000 7f82fee1ab80 1 Processor -- start 2024-01-11T18:05:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.396+0000 7f82fee1ab80 1 -- start start 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 0x562fc26bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bceb0 0x562fc26bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc25b3030 con 0x562fc26bceb0 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc2616ac0 con 0x562fc26bc120 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc26143e0 con 0x562fc26bd870 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82e959c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 0x562fc26bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82e959c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 0x562fc26bc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35728/0 (socket says 172.21.15.7:35728) 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.397+0000 7f82e959c700 1 -- 172.21.15.7:0/548895827 learned_addr learned my addr 172.21.15.7:0/548895827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.398+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bceb0 0x562fc26bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.398+0000 7f82e1ffb700 1 -- 172.21.15.7:0/4204264538 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f82d0010820 con 0x7f82dc09baf0 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.398+0000 7f82e3fff700 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e959c700 1 -- 172.21.15.7:0/548895827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 msgr2=0x562fc26c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e959c700 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e959c700 1 -- 172.21.15.7:0/548895827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bceb0 msgr2=0x562fc26bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e959c700 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bceb0 0x562fc26bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e959c700 1 -- 172.21.15.7:0/548895827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fc2484350 con 0x562fc26bc120 2024-01-11T18:05:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bceb0 0x562fc26bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:05:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.399+0000 7f82e959c700 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 0x562fc26bc540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82e4066d50 tx=0x7f82e40503f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1582faea40ba3a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.400+0000 7f82ceffd700 1 -- 172.21.15.7:0/548895827 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4082b60 con 0x562fc26bc120 2024-01-11T18:05:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.400+0000 7f82ceffd700 1 -- 172.21.15.7:0/548895827 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e4082cc0 con 0x562fc26bc120 2024-01-11T18:05:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.400+0000 7f82ceffd700 1 -- 172.21.15.7:0/548895827 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e408a630 con 0x562fc26bc120 2024-01-11T18:05:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 -- 172.21.15.7:0/548895827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 msgr2=0x562fc26bc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 0x562fc26bc540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82e4066d50 tx=0x7f82e40503f0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 -- 172.21.15.7:0/548895827 shutdown_connections 2024-01-11T18:05:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bc120 0x562fc26bc540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/548895827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bceb0 0x562fc26bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 -- 172.21.15.7:0/548895827 >> 172.21.15.7:0/548895827 conn(0x562fc2631740 msgr2=0x562fc26b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 -- 172.21.15.7:0/548895827 shutdown_connections 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82fee1ab80 1 -- 172.21.15.7:0/548895827 wait complete. 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.401+0000 7f82e1ffb700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82d40a1ec0 0x7f82d40a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.402+0000 7f82e1ffb700 1 -- 172.21.15.7:0/4204264538 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f82d00cd130 con 0x7f82dc09baf0 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.402+0000 7f82fee1ab80 1 -- start start 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82fee1ab80 1 Processor -- start 2024-01-11T18:05:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82d40a1ec0 0x7f82d40a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc260fcd0 con 0x562fc26bc120 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.403+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc257e340 con 0x562fc26bceb0 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc257dfe0 con 0x562fc26bd870 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82fee1ab80 10 client.0 _ll_register_callbacks cb 0x562fc25b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35742/0 (socket says 172.21.15.7:35742) 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e3fff700 1 -- 172.21.15.7:0/3258631109 learned_addr learned my addr 172.21.15.7:0/3258631109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e959c700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e3fff700 1 -- 172.21.15.7:0/3258631109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 msgr2=0x562fc26cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.404+0000 7f82e3fff700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82e3fff700 1 -- 172.21.15.7:0/3258631109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 msgr2=0x562fc26c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82e3fff700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82e3fff700 1 -- 172.21.15.7:0/3258631109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e4050260 con 0x562fc26bceb0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82e3fff700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26c60a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009520 tx=0x7f82d0099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63aff12f59487315 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82c3fff700 1 -- 172.21.15.7:0/3258631109 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d009a260 con 0x562fc26bceb0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82c3fff700 1 -- 172.21.15.7:0/3258631109 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f82d0099b60 con 0x562fc26bceb0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fc26aefb0 con 0x562fc26bceb0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82d40a1ec0 0x7f82d40a4380 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82d800b5c0 tx=0x7f82d800bbe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82c3fff700 1 -- 172.21.15.7:0/3258631109 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82d009dcb0 con 0x562fc26bceb0 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82fee1ab80 10 client.26147 fetch_fsmap learned FSMap version 37 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.405+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562fc2430b00 con 0x562fc26bceb0 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82c3fff700 1 -- 172.21.15.7:0/3258631109 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562fc26aefb0 con 0x562fc26bceb0 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82c3fff700 1 -- 172.21.15.7:0/3258631109 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f82d00a1a60 con 0x562fc26bceb0 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82fee1ab80 10 client.26147 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82fee1ab80 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82fee1ab80 20 client.26147 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc26d19a0 0x562fc26d3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82fee1ab80 4 client.26147 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:05:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.406+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fc2608b60 con 0x562fc26d19a0 2024-01-11T18:05:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.407+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc26d19a0 0x562fc26d3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.408+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc26d19a0 0x562fc26d3dc0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd80fd20c8d3de1d server_cookie=6642bbe9b5747f22 in_seq=0 out_seq=0 2024-01-11T18:05:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.408+0000 7f82c3fff700 10 client.26147 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.409+0000 7f82c3fff700 1 -- 172.21.15.7:0/3258631109 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562fc2608b60 con 0x562fc26d19a0 2024-01-11T18:05:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.409+0000 7f82c3fff700 10 client.26147 handle_command_reply: tid=0 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.409+0000 7f82fee1ab80 1 client.26147 shutdown 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 msgr2=0x562fc26c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26c60a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009520 tx=0x7f82d0099a30 comp rx=0 tx=0).stop 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 shutdown_connections 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc26d19a0 0x562fc26d3dc0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/3258631109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26c60a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 >> 172.21.15.7:0/3258631109 conn(0x562fc2631740 msgr2=0x562fc26b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.410+0000 7f82c3fff700 0 client.26147 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.411+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 shutdown_connections 2024-01-11T18:05:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.411+0000 7f82fee1ab80 1 -- 172.21.15.7:0/3258631109 wait complete. 2024-01-11T18:05:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.411+0000 7f82fee1ab80 20 client.26147 trim_cache size 0 max 16384 2024-01-11T18:05:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.441+0000 7f82fee1ab80 1 Processor -- start 2024-01-11T18:05:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.441+0000 7f82fee1ab80 1 -- start start 2024-01-11T18:05:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.442+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.442+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.442+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.442+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc2486640 con 0x562fc26bc120 2024-01-11T18:05:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.442+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc26c6a60 con 0x562fc26bceb0 2024-01-11T18:05:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.442+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc26c56d0 con 0x562fc26bd870 2024-01-11T18:05:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e959c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57424/0 (socket says 172.21.15.7:57424) 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e8d9b700 1 -- 172.21.15.7:0/837517622 learned_addr learned my addr 172.21.15.7:0/837517622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e959c700 1 -- 172.21.15.7:0/837517622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 msgr2=0x562fc26cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e959c700 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e959c700 1 -- 172.21.15.7:0/837517622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 msgr2=0x562fc26aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e959c700 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e959c700 1 -- 172.21.15.7:0/837517622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fc25d11d0 con 0x562fc26bd870 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e3fff700 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.443+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.444+0000 7f82e959c700 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26d39b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f82e4050c70 tx=0x7f82e404f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf29da32acc3fd05 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.444+0000 7f82ce7fc700 1 -- 172.21.15.7:0/837517622 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4082960 con 0x562fc26bd870 2024-01-11T18:05:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.444+0000 7f82ce7fc700 1 -- 172.21.15.7:0/837517622 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e4082ac0 con 0x562fc26bd870 2024-01-11T18:05:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.444+0000 7f82ce7fc700 1 -- 172.21.15.7:0/837517622 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e40935c0 con 0x562fc26bd870 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.445+0000 7f82fee1ab80 1 -- 172.21.15.7:0/837517622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 msgr2=0x562fc26d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.445+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26d39b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f82e4050c70 tx=0x7f82e404f870 comp rx=0 tx=0).stop 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.445+0000 7f82fee1ab80 1 -- 172.21.15.7:0/837517622 shutdown_connections 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.445+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26d39b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.445+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.445+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/837517622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.446+0000 7f82fee1ab80 1 -- 172.21.15.7:0/837517622 >> 172.21.15.7:0/837517622 conn(0x562fc2631740 msgr2=0x562fc26c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.446+0000 7f82fee1ab80 1 -- 172.21.15.7:0/837517622 shutdown_connections 2024-01-11T18:05:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.446+0000 7f82fee1ab80 1 -- 172.21.15.7:0/837517622 wait complete. 2024-01-11T18:05:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.446+0000 7f82fee1ab80 1 -- start start 2024-01-11T18:05:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 Processor -- start 2024-01-11T18:05:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc26cbb20 con 0x562fc26bc120 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc26cbc90 con 0x562fc26bceb0 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc26cbe50 con 0x562fc26bd870 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.447+0000 7f82fee1ab80 10 client.0 _ll_register_callbacks cb 0x562fc26cdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.448+0000 7f82e8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.448+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.448+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35766/0 (socket says 172.21.15.7:35766) 2024-01-11T18:05:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.448+0000 7f82e3fff700 1 -- 172.21.15.7:0/1701792454 learned_addr learned my addr 172.21.15.7:0/1701792454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.448+0000 7f82e959c700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.449+0000 7f82e3fff700 1 -- 172.21.15.7:0/1701792454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 msgr2=0x562fc26cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.449+0000 7f82e3fff700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.449+0000 7f82e3fff700 1 -- 172.21.15.7:0/1701792454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 msgr2=0x562fc26ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.449+0000 7f82e3fff700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.449+0000 7f82e3fff700 1 -- 172.21.15.7:0/1701792454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc09eb50 con 0x562fc26bceb0 2024-01-11T18:05:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.449+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:05:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.450+0000 7f82e3fff700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26ca5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82d0005950 tx=0x7f82d00cf260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a88cc9bac61f49 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.450+0000 7f82c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:05:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.451+0000 7f82c3fff700 1 -- 172.21.15.7:0/1701792454 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0099250 con 0x562fc26bceb0 2024-01-11T18:05:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.451+0000 7f82c3fff700 1 -- 172.21.15.7:0/1701792454 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f82d009a260 con 0x562fc26bceb0 2024-01-11T18:05:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.451+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82e4082cc0 con 0x562fc26bceb0 2024-01-11T18:05:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82c3fff700 1 -- 172.21.15.7:0/1701792454 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82d0099250 con 0x562fc26bceb0 2024-01-11T18:05:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82fee1ab80 10 client.26153 fetch_fsmap learned FSMap version 37 2024-01-11T18:05:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562fc2430b00 con 0x562fc26bceb0 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82c3fff700 1 -- 172.21.15.7:0/1701792454 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e4082cc0 con 0x562fc26bceb0 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82c3fff700 1 -- 172.21.15.7:0/1701792454 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f82d0098ac0 con 0x562fc26bceb0 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82fee1ab80 10 client.26153 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82fee1ab80 10 client.26153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.452+0000 7f82fee1ab80 20 client.26153 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.453+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc2727ea0 0x562fc272a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.453+0000 7f82fee1ab80 4 client.26153 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:05:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.453+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562fc2608880 con 0x562fc2727ea0 2024-01-11T18:05:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.453+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc2727ea0 0x562fc272a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.454+0000 7f82e8d9b700 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc2727ea0 0x562fc272a290 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b3a43770f8364f server_cookie=27cca57c39d6f5f7 in_seq=0 out_seq=0 2024-01-11T18:05:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.454+0000 7f82c3fff700 10 client.26153 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.468+0000 7f82c3fff700 1 -- 172.21.15.7:0/1701792454 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562fc2608880 con 0x562fc2727ea0 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.468+0000 7f82c3fff700 10 client.26153 handle_command_reply: tid=0 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.468+0000 7f82fee1ab80 1 client.26153 shutdown 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 msgr2=0x562fc26ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26ca5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82d0005950 tx=0x7f82d00cf260 comp rx=0 tx=0).stop 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 shutdown_connections 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562fc2727ea0 0x562fc272a290 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fc26bd870 0x562fc26cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562fc26bc120 0x562fc26ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82c3fff700 0 client.26153 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 --2- 172.21.15.7:0/1701792454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fc26bceb0 0x562fc26ca5d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 >> 172.21.15.7:0/1701792454 conn(0x562fc2631740 msgr2=0x562fc26d86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 shutdown_connections 2024-01-11T18:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.469+0000 7f82fee1ab80 1 -- 172.21.15.7:0/1701792454 wait complete. 2024-01-11T18:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.470+0000 7f82fee1ab80 20 client.26153 trim_cache size 0 max 16384 2024-01-11T18:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.470+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82d40a1ec0 msgr2=0x7f82d40a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82d40a1ec0 0x7f82d40a4380 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f82d800b5c0 tx=0x7f82d800bbe0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc09baf0 msgr2=0x7f82dc12e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc09baf0 0x7f82dc12e700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82d000d950 tx=0x7f82d000dc60 comp rx=0 tx=0).stop 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 shutdown_connections 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82d40a1ec0 0x7f82d40a4380 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82dc0991c0 0x7f82dc12e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc133760 0x7f82dc135b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 --2- 172.21.15.7:0/4204264538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc09baf0 0x7f82dc12e700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 >> 172.21.15.7:0/4204264538 conn(0x7f82dc090260 msgr2=0x7f82dc096030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.471+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 shutdown_connections 2024-01-11T18:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:09.472+0000 7f82e9d9d700 1 -- 172.21.15.7:0/4204264538 wait complete. 2024-01-11T18:05:10.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:09 smithi016 ceph-mon[43195]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.8 MiB/s wr, 588 op/s 2024-01-11T18:05:10.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:09 smithi038 ceph-mon[43582]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.8 MiB/s wr, 588 op/s 2024-01-11T18:05:10.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:09 smithi007 ceph-mon[33361]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.8 MiB/s wr, 588 op/s 2024-01-11T18:05:12.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:11 smithi016 ceph-mon[43195]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.8 MiB/s wr, 761 op/s 2024-01-11T18:05:12.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:11 smithi038 ceph-mon[43582]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.8 MiB/s wr, 761 op/s 2024-01-11T18:05:12.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:11 smithi007 ceph-mon[33361]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.8 MiB/s wr, 761 op/s 2024-01-11T18:05:14.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 405.0 s, 518.0 tps, lat 194.008 ms stddev 255.507 2024-01-11T18:05:14.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:13 smithi016 ceph-mon[43195]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.9 MiB/s wr, 894 op/s 2024-01-11T18:05:14.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:13 smithi038 ceph-mon[43582]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.9 MiB/s wr, 894 op/s 2024-01-11T18:05:14.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:13 smithi007 ceph-mon[33361]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.9 MiB/s wr, 894 op/s 2024-01-11T18:05:15.991 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:15 smithi016 ceph-mon[43195]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 MiB/s wr, 923 op/s 2024-01-11T18:05:16.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:15 smithi038 ceph-mon[43582]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 MiB/s wr, 923 op/s 2024-01-11T18:05:16.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:15 smithi007 ceph-mon[33361]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 MiB/s wr, 923 op/s 2024-01-11T18:05:18.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:17 smithi016 ceph-mon[43195]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-01-11T18:05:18.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:17 smithi038 ceph-mon[43582]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-01-11T18:05:18.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:17 smithi007 ceph-mon[33361]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 MiB/s wr, 908 op/s 2024-01-11T18:05:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 410.0 s, 597.2 tps, lat 166.101 ms stddev 210.870 2024-01-11T18:05:19.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:18 smithi016 ceph-mon[43195]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 919 op/s 2024-01-11T18:05:19.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:18 smithi038 ceph-mon[43582]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 919 op/s 2024-01-11T18:05:19.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:18 smithi007 ceph-mon[33361]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 919 op/s 2024-01-11T18:05:22.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:21 smithi016 ceph-mon[43195]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 903 op/s 2024-01-11T18:05:22.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:21 smithi038 ceph-mon[43582]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 903 op/s 2024-01-11T18:05:22.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:21 smithi007 ceph-mon[33361]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 903 op/s 2024-01-11T18:05:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 -- 172.21.15.7:0/3904491622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 msgr2=0x7fa568106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 --2- 172.21.15.7:0/3904491622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa568106e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa558009f90 tx=0x7fa55800b330 comp rx=0 tx=0).stop 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 -- 172.21.15.7:0/3904491622 shutdown_connections 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 --2- 172.21.15.7:0/3904491622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa568107340 0x7fa568109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 --2- 172.21.15.7:0/3904491622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 0x7fa5681044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 --2- 172.21.15.7:0/3904491622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa568106e00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.771+0000 7fa56efff700 1 -- 172.21.15.7:0/3904491622 >> 172.21.15.7:0/3904491622 conn(0x7fa5680fba30 msgr2=0x7fa5680fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.772+0000 7fa56efff700 1 -- 172.21.15.7:0/3904491622 shutdown_connections 2024-01-11T18:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.772+0000 7fa56efff700 1 -- 172.21.15.7:0/3904491622 wait complete. 2024-01-11T18:05:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 Processor -- start 2024-01-11T18:05:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 -- start start 2024-01-11T18:05:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 0x7fa56810fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa5681101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa568107340 0x7fa56810c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56810cfe0 con 0x7fa568104a10 2024-01-11T18:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56810d150 con 0x7fa5681020e0 2024-01-11T18:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.774+0000 7fa56efff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56810d310 con 0x7fa568107340 2024-01-11T18:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.775+0000 7fa567fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa5681101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.775+0000 7fa567fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa5681101f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60486/0 (socket says 172.21.15.7:60486) 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.775+0000 7fa567fff700 1 -- 172.21.15.7:0/3731165529 learned_addr learned my addr 172.21.15.7:0/3731165529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.775+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 0x7fa56810fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.776+0000 7fa56cd9b700 1 -- 172.21.15.7:0/3731165529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa568107340 msgr2=0x7fa56810c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.776+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa568107340 0x7fa56810c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.776+0000 7fa56cd9b700 1 -- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 msgr2=0x7fa5681101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.776+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa5681101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.776+0000 7fa56cd9b700 1 -- 172.21.15.7:0/3731165529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558009c40 con 0x7fa5681020e0 2024-01-11T18:05:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.776+0000 7fa567fff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa5681101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.781+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 0x7fa56810fcb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d950 tx=0x7fa55000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.782+0000 7fa565ffb700 1 -- 172.21.15.7:0/3731165529 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550009930 con 0x7fa5681020e0 2024-01-11T18:05:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.782+0000 7fa565ffb700 1 -- 172.21.15.7:0/3731165529 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa550010460 con 0x7fa5681020e0 2024-01-11T18:05:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.782+0000 7fa565ffb700 1 -- 172.21.15.7:0/3731165529 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000b5d0 con 0x7fa5681020e0 2024-01-11T18:05:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.782+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5681a5d30 con 0x7fa5681020e0 2024-01-11T18:05:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.782+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5681a61b0 con 0x7fa5681020e0 2024-01-11T18:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.788+0000 7fa565ffb700 1 -- 172.21.15.7:0/3731165529 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa55000f620 con 0x7fa5681020e0 2024-01-11T18:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.790+0000 7fa565ffb700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5540a1f50 0x7fa5540a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.791+0000 7fa567fff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5540a1f50 0x7fa5540a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.791+0000 7fa582c8bb80 1 Processor -- start 2024-01-11T18:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.791+0000 7fa582c8bb80 1 -- start start 2024-01-11T18:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.791+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703cff20 0x55a2703d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.791+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d0c10 0x55a2703d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.791+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2702c7030 con 0x55a2703cff20 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a27032aac0 con 0x55a2703d0c10 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2703283e0 con 0x55a2703d15d0 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50302/0 (socket says 172.21.15.7:50302) 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 -- 172.21.15.7:0/3488612159 learned_addr learned my addr 172.21.15.7:0/3488612159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa567fff700 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d0c10 0x55a2703d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 -- 172.21.15.7:0/3488612159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d0c10 msgr2=0x55a2703d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d0c10 0x55a2703d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 -- 172.21.15.7:0/3488612159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703cff20 msgr2=0x55a2703d0340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703cff20 0x55a2703d0340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.792+0000 7fa56d59c700 1 -- 172.21.15.7:0/3488612159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a270198350 con 0x55a2703d15d0 2024-01-11T18:05:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.793+0000 7fa56d59c700 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d7e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa560009cd0 tx=0x7fa56000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=355352bb8d6c96b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.794+0000 7fa55effd700 1 -- 172.21.15.7:0/3488612159 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000ec00 con 0x55a2703d15d0 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.794+0000 7fa55effd700 1 -- 172.21.15.7:0/3488612159 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa56000ed60 con 0x55a2703d15d0 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.794+0000 7fa55effd700 1 -- 172.21.15.7:0/3488612159 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560010430 con 0x55a2703d15d0 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 -- 172.21.15.7:0/3488612159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 msgr2=0x55a2703d7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d7e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa560009cd0 tx=0x7fa56000e350 comp rx=0 tx=0).stop 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 -- 172.21.15.7:0/3488612159 shutdown_connections 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d7e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703cff20 0x55a2703d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/3488612159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d0c10 0x55a2703d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.795+0000 7fa582c8bb80 1 -- 172.21.15.7:0/3488612159 >> 172.21.15.7:0/3488612159 conn(0x55a270345740 msgr2=0x55a2703ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.798+0000 7fa567fff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5540a1f50 0x7fa5540a4410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa558007a30 tx=0x7fa5580096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.798+0000 7fa565ffb700 1 -- 172.21.15.7:0/3731165529 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa550097890 con 0x7fa5681020e0 2024-01-11T18:05:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.800+0000 7fa582c8bb80 1 -- 172.21.15.7:0/3488612159 shutdown_connections 2024-01-11T18:05:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.800+0000 7fa582c8bb80 1 -- 172.21.15.7:0/3488612159 wait complete. 2024-01-11T18:05:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.801+0000 7fa582c8bb80 1 -- start start 2024-01-11T18:05:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.801+0000 7fa582c8bb80 1 Processor -- start 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.802+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.802+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 0x55a2703df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.802+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a2703e16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.802+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a270323cd0 con 0x55a2703da570 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.802+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a270292340 con 0x55a2703da0f0 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.802+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a270291fe0 con 0x55a2703d15d0 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa582c8bb80 10 client.0 _ll_register_callbacks cb 0x55a2702c46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50306/0 (socket says 172.21.15.7:50306) 2024-01-11T18:05:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 -- 172.21.15.7:0/842423908 learned_addr learned my addr 172.21.15.7:0/842423908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 -- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 msgr2=0x55a2703df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 0x55a2703df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 -- 172.21.15.7:0/842423908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 msgr2=0x55a2703e16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a2703e16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.803+0000 7fa56cd9b700 1 -- 172.21.15.7:0/842423908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa560009980 con 0x55a2703d15d0 2024-01-11T18:05:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d9bb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa55009ae80 tx=0x7fa55009adf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7566d33d0dd3074d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa547fff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:05:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa547fff700 1 -- 172.21.15.7:0/842423908 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55009b880 con 0x55a2703d15d0 2024-01-11T18:05:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa547fff700 1 -- 172.21.15.7:0/842423908 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa55009b180 con 0x55a2703d15d0 2024-01-11T18:05:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2703c2d60 con 0x55a2703d15d0 2024-01-11T18:05:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa547fff700 1 -- 172.21.15.7:0/842423908 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa55009f2d0 con 0x55a2703d15d0 2024-01-11T18:05:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa582c8bb80 10 client.26194 fetch_fsmap learned FSMap version 37 2024-01-11T18:05:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.806+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a270144b00 con 0x55a2703d15d0 2024-01-11T18:05:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.807+0000 7fa547fff700 1 -- 172.21.15.7:0/842423908 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2703c2d60 con 0x55a2703d15d0 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.807+0000 7fa547fff700 1 -- 172.21.15.7:0/842423908 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa5500a49a0 con 0x55a2703d15d0 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.809+0000 7fa582c8bb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.809+0000 7fa582c8bb80 10 client.26194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.809+0000 7fa582c8bb80 20 client.26194 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.809+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a2703f7230 0x55a2703f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.809+0000 7fa582c8bb80 4 client.26194 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.809+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a27031cb60 con 0x55a2703f7230 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.810+0000 7fa567fff700 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a2703f7230 0x55a2703f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.812+0000 7fa567fff700 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a2703f7230 0x55a2703f9650 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a219df1f98aa5112 server_cookie=615db657196a3bc in_seq=0 out_seq=0 2024-01-11T18:05:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.812+0000 7fa547fff700 10 client.26194 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.812+0000 7fa547fff700 1 -- 172.21.15.7:0/842423908 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a27031cb60 con 0x55a2703f7230 2024-01-11T18:05:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa547fff700 10 client.26194 handle_command_reply: tid=0 2024-01-11T18:05:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 client.26194 shutdown 2024-01-11T18:05:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 msgr2=0x55a2703d9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d9bb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa55009ae80 tx=0x7fa55009adf0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 shutdown_connections 2024-01-11T18:05:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a2703f7230 0x55a2703f9650 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a2703d9bb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a2703e16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa547fff700 0 client.26194 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/842423908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 0x55a2703df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.813+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 >> 172.21.15.7:0/842423908 conn(0x55a270345740 msgr2=0x55a2703c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.814+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 shutdown_connections 2024-01-11T18:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.814+0000 7fa582c8bb80 1 -- 172.21.15.7:0/842423908 wait complete. 2024-01-11T18:05:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.814+0000 7fa582c8bb80 20 client.26194 trim_cache size 0 max 16384 2024-01-11T18:05:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.862+0000 7fa582c8bb80 1 Processor -- start 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.863+0000 7fa582c8bb80 1 -- start start 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.864+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 0x55a2703df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.864+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703da0f0 0x55a2703c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.864+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a2703d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.864+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a27019a640 con 0x55a2703da570 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.864+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2703cb790 con 0x55a2703d15d0 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.864+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2703e1cd0 con 0x55a2703da0f0 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.865+0000 7fa56cd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 0x55a2703df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.866+0000 7fa56cd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 0x55a2703df0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38270/0 (socket says 172.21.15.7:38270) 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.866+0000 7fa56cd9b700 1 -- 172.21.15.7:0/1862836898 learned_addr learned my addr 172.21.15.7:0/1862836898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.867+0000 7fa56cd9b700 1 -- 172.21.15.7:0/1862836898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703da0f0 msgr2=0x55a2703c69b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.867+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703da0f0 0x55a2703c69b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.867+0000 7fa56cd9b700 1 -- 172.21.15.7:0/1862836898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 msgr2=0x55a2703d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.867+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a2703d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.867+0000 7fa56cd9b700 1 -- 172.21.15.7:0/1862836898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2702e51d0 con 0x55a2703d15d0 2024-01-11T18:05:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.868+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 0x55a2703df0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa550005950 tx=0x7fa550097ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76ae44857ba31da server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.872+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1862836898 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55009b180 con 0x55a2703d15d0 2024-01-11T18:05:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.872+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1862836898 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55009f2d0 con 0x55a2703d15d0 2024-01-11T18:05:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.872+0000 7fa55e7fc700 1 -- 172.21.15.7:0/1862836898 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500222b0 con 0x55a2703d15d0 2024-01-11T18:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.876+0000 7fa582c8bb80 1 -- 172.21.15.7:0/1862836898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 msgr2=0x55a2703df0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.876+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 0x55a2703df0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa550005950 tx=0x7fa550097ad0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.877+0000 7fa582c8bb80 1 -- 172.21.15.7:0/1862836898 shutdown_connections 2024-01-11T18:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.877+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a2703d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.877+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703d15d0 0x55a2703df0e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.877+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/1862836898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703da0f0 0x55a2703c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.877+0000 7fa582c8bb80 1 -- 172.21.15.7:0/1862836898 >> 172.21.15.7:0/1862836898 conn(0x55a270345740 msgr2=0x55a2703ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.878+0000 7fa582c8bb80 1 -- 172.21.15.7:0/1862836898 shutdown_connections 2024-01-11T18:05:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.879+0000 7fa582c8bb80 1 -- 172.21.15.7:0/1862836898 wait complete. 2024-01-11T18:05:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.879+0000 7fa582c8bb80 1 -- start start 2024-01-11T18:05:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.880+0000 7fa582c8bb80 1 Processor -- start 2024-01-11T18:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a270449b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 0x55a270407530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a270407a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a27045d3d0 con 0x55a2703da570 2024-01-11T18:05:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a27045d540 con 0x55a2703da0f0 2024-01-11T18:05:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a27045d700 con 0x55a2703d15d0 2024-01-11T18:05:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.882+0000 7fa582c8bb80 10 client.0 _ll_register_callbacks cb 0x55a2702c46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.883+0000 7fa56d59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a270407a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.883+0000 7fa56d59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a270407a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60518/0 (socket says 172.21.15.7:60518) 2024-01-11T18:05:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.883+0000 7fa56d59c700 1 -- 172.21.15.7:0/4292552678 learned_addr learned my addr 172.21.15.7:0/4292552678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.888+0000 7fa56d59c700 1 -- 172.21.15.7:0/4292552678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 msgr2=0x55a270449b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.888+0000 7fa56d59c700 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a270449b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.888+0000 7fa56d59c700 1 -- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 msgr2=0x55a270407530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.888+0000 7fa56d59c700 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 0x55a270407530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.888+0000 7fa56d59c700 1 -- 172.21.15.7:0/4292552678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a270144b00 con 0x55a2703da570 2024-01-11T18:05:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.889+0000 7fa56d59c700 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a270407a90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e750 tx=0x7fa560004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a53d62a43137c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.889+0000 7fa547fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:05:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.890+0000 7fa547fff700 1 -- 172.21.15.7:0/4292552678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560018430 con 0x55a2703da570 2024-01-11T18:05:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.890+0000 7fa547fff700 1 -- 172.21.15.7:0/4292552678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa560014070 con 0x55a2703da570 2024-01-11T18:05:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.890+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa56000ed60 con 0x55a2703da570 2024-01-11T18:05:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.893+0000 7fa547fff700 1 -- 172.21.15.7:0/4292552678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa560018590 con 0x55a2703da570 2024-01-11T18:05:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.893+0000 7fa582c8bb80 10 client.16017 fetch_fsmap learned FSMap version 37 2024-01-11T18:05:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.893+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a27019cc70 con 0x55a2703da570 2024-01-11T18:05:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.893+0000 7fa547fff700 1 -- 172.21.15.7:0/4292552678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa56000ed60 con 0x55a2703da570 2024-01-11T18:05:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.894+0000 7fa547fff700 1 -- 172.21.15.7:0/4292552678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa560010960 con 0x55a2703da570 2024-01-11T18:05:22.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa582c8bb80 10 client.16017 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:05:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa582c8bb80 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa582c8bb80 20 client.16017 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a27045f040 0x55a270461460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa582c8bb80 4 client.16017 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:05:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a27031c880 con 0x55a27045f040 2024-01-11T18:05:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.895+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a27045f040 0x55a270461460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.902+0000 7fa56cd9b700 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a27045f040 0x55a270461460 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe76382c8cef1b29 server_cookie=10b34d7114758872 in_seq=0 out_seq=0 2024-01-11T18:05:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:22.902+0000 7fa547fff700 10 client.16017 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.209+0000 7fa547fff700 1 -- 172.21.15.7:0/4292552678 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a27031c880 con 0x55a27045f040 2024-01-11T18:05:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.209+0000 7fa547fff700 10 client.16017 handle_command_reply: tid=0 2024-01-11T18:05:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.209+0000 7fa582c8bb80 1 client.16017 shutdown 2024-01-11T18:05:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 msgr2=0x55a270407a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a270407a90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e750 tx=0x7fa560004740 comp rx=0 tx=0).stop 2024-01-11T18:05:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 shutdown_connections 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55a27045f040 0x55a270461460 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2703d15d0 0x55a270449b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a2703da570 0x55a270407a90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 --2- 172.21.15.7:0/4292552678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2703da0f0 0x55a270407530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 >> 172.21.15.7:0/4292552678 conn(0x55a270345740 msgr2=0x55a2703d9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.211+0000 7fa547fff700 0 client.16017 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.212+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 shutdown_connections 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.212+0000 7fa582c8bb80 1 -- 172.21.15.7:0/4292552678 wait complete. 2024-01-11T18:05:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.212+0000 7fa582c8bb80 20 client.16017 trim_cache size 0 max 16384 2024-01-11T18:05:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:05:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:05:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:05:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:05:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.213+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5540a1f50 msgr2=0x7fa5540a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.213+0000 7fa56efff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5540a1f50 0x7fa5540a4410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa558007a30 tx=0x7fa5580096c0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.213+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 msgr2=0x7fa56810fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.213+0000 7fa56efff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 0x7fa56810fcb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d950 tx=0x7fa55000dc60 comp rx=0 tx=0).stop 2024-01-11T18:05:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.218+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 shutdown_connections 2024-01-11T18:05:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.218+0000 7fa56efff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5540a1f50 0x7fa5540a4410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.218+0000 7fa56efff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa568107340 0x7fa56810c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.218+0000 7fa56efff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5681020e0 0x7fa56810fcb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.219+0000 7fa56efff700 1 --2- 172.21.15.7:0/3731165529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa568104a10 0x7fa5681101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.219+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 >> 172.21.15.7:0/3731165529 conn(0x7fa5680fba30 msgr2=0x7fa568107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.219+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 shutdown_connections 2024-01-11T18:05:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:23.220+0000 7fa56efff700 1 -- 172.21.15.7:0/3731165529 wait complete. 2024-01-11T18:05:23.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:22 smithi016 ceph-mon[43195]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 704 op/s 2024-01-11T18:05:23.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:22 smithi038 ceph-mon[43582]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 704 op/s 2024-01-11T18:05:23.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:22 smithi007 ceph-mon[33361]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 704 op/s 2024-01-11T18:05:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 415.0 s, 503.4 tps, lat 195.941 ms stddev 240.907 2024-01-11T18:05:25.989 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:25 smithi016 ceph-mon[43195]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 MiB/s wr, 577 op/s 2024-01-11T18:05:26.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:25 smithi038 ceph-mon[43582]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 MiB/s wr, 577 op/s 2024-01-11T18:05:26.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:25 smithi007 ceph-mon[33361]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 MiB/s wr, 577 op/s 2024-01-11T18:05:28.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:27 smithi016 ceph-mon[43195]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 540 op/s 2024-01-11T18:05:28.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:27 smithi038 ceph-mon[43582]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 540 op/s 2024-01-11T18:05:28.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:27 smithi007 ceph-mon[33361]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 540 op/s 2024-01-11T18:05:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 420.0 s, 441.8 tps, lat 222.170 ms stddev 304.583 2024-01-11T18:05:29.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:28 smithi016 ceph-mon[43195]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 578 op/s 2024-01-11T18:05:29.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:28 smithi038 ceph-mon[43582]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 578 op/s 2024-01-11T18:05:29.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:28 smithi007 ceph-mon[33361]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 578 op/s 2024-01-11T18:05:32.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:31 smithi016 ceph-mon[43195]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-01-11T18:05:32.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:31 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:05:32.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:31 smithi038 ceph-mon[43582]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-01-11T18:05:32.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:31 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:05:32.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:31 smithi007 ceph-mon[33361]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-01-11T18:05:32.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:31 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:05:33.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:32 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:32 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:33.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:32 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 425.0 s, 478.4 tps, lat 215.699 ms stddev 264.533 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi038 to 183.5M 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: Unable to set osd_memory_target on smithi038 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: Unable to set osd_memory_target on smithi016 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 607 op/s 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: Adjusting osd_memory_target on smithi007 to 223.0M 2024-01-11T18:05:34.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: Unable to set osd_memory_target on smithi007 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2024-01-11T18:05:34.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:05:34.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:05:34.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:05:34.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi038 to 183.5M 2024-01-11T18:05:34.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: Unable to set osd_memory_target on smithi038 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: Unable to set osd_memory_target on smithi016 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 607 op/s 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: Adjusting osd_memory_target on smithi007 to 223.0M 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: Unable to set osd_memory_target on smithi007 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:05:34.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi038 to 183.5M 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: Unable to set osd_memory_target on smithi038 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: Unable to set osd_memory_target on smithi016 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 607 op/s 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-11T18:05:34.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: Adjusting osd_memory_target on smithi007 to 223.0M 2024-01-11T18:05:34.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: Unable to set osd_memory_target on smithi007 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2024-01-11T18:05:34.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:05:34.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:05:34.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:05:35.989 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:35 smithi016 ceph-mon[43195]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.1 MiB/s wr, 615 op/s 2024-01-11T18:05:36.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:35 smithi038 ceph-mon[43582]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.1 MiB/s wr, 615 op/s 2024-01-11T18:05:36.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:35 smithi007 ceph-mon[33361]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.1 MiB/s wr, 615 op/s 2024-01-11T18:05:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 -- 172.21.15.7:0/3414539855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908108dd0 msgr2=0x7f8908109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 --2- 172.21.15.7:0/3414539855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908108dd0 0x7f8908109230 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009920 tx=0x7f88fc009c30 comp rx=0 tx=0).stop 2024-01-11T18:05:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 -- 172.21.15.7:0/3414539855 shutdown_connections 2024-01-11T18:05:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 --2- 172.21.15.7:0/3414539855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 0x7f8908108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 --2- 172.21.15.7:0/3414539855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f8908110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 --2- 172.21.15.7:0/3414539855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908108dd0 0x7f8908109230 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009920 tx=0x7f88fc009c30 comp rx=0 tx=0).stop 2024-01-11T18:05:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.529+0000 7f890edff700 1 -- 172.21.15.7:0/3414539855 >> 172.21.15.7:0/3414539855 conn(0x7f8908076b60 msgr2=0x7f8908078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.530+0000 7f890edff700 1 -- 172.21.15.7:0/3414539855 shutdown_connections 2024-01-11T18:05:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.530+0000 7f890edff700 1 -- 172.21.15.7:0/3414539855 wait complete. 2024-01-11T18:05:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.531+0000 7f890edff700 1 Processor -- start 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 -- start start 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 0x7f890819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f890819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081a3380 0x7f89081a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89081a5d40 con 0x7f89081a3380 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890806b030 con 0x7f8908109770 2024-01-11T18:05:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.532+0000 7f890edff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890806b1c0 con 0x7f8908107be0 2024-01-11T18:05:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.533+0000 7f8907fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f890819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.533+0000 7f8907fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f890819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35710/0 (socket says 172.21.15.7:35710) 2024-01-11T18:05:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.533+0000 7f890cb9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 0x7f890819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.533+0000 7f8907fff700 1 -- 172.21.15.7:0/564005980 learned_addr learned my addr 172.21.15.7:0/564005980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.533+0000 7f890d39c700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081a3380 0x7f89081a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f890cb9b700 1 -- 172.21.15.7:0/564005980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 msgr2=0x7f890819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f890cb9b700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f890819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f890cb9b700 1 -- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081a3380 msgr2=0x7f89081a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f890cb9b700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081a3380 0x7f89081a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f890cb9b700 1 -- 172.21.15.7:0/564005980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc009520 con 0x7f8908107be0 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f8907fff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f890819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:05:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f890cb9b700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 0x7f890819dde0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f890000bd40 tx=0x7f890000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.534+0000 7f8905ffb700 1 -- 172.21.15.7:0/564005980 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000cd10 con 0x7f8908107be0 2024-01-11T18:05:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.535+0000 7f8905ffb700 1 -- 172.21.15.7:0/564005980 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f890000ce70 con 0x7f8908107be0 2024-01-11T18:05:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.535+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890806b4a0 con 0x7f8908107be0 2024-01-11T18:05:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.535+0000 7f8905ffb700 1 -- 172.21.15.7:0/564005980 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900012840 con 0x7f8908107be0 2024-01-11T18:05:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.535+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890806ba80 con 0x7f8908107be0 2024-01-11T18:05:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.544+0000 7f8922bf1b80 1 Processor -- start 2024-01-11T18:05:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.544+0000 7f8922bf1b80 1 -- start start 2024-01-11T18:05:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b8f20 0x562ec77b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec76b0030 con 0x562ec77b8f20 2024-01-11T18:05:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec7713ac0 con 0x562ec77b9c10 2024-01-11T18:05:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec77113e0 con 0x562ec77ba5d0 2024-01-11T18:05:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8905ffb700 1 -- 172.21.15.7:0/564005980 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f8900014b00 con 0x7f8908107be0 2024-01-11T18:05:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.545+0000 7f8907fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f8907fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:60876/0 (socket says 172.21.15.7:60876) 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f8907fff700 1 -- 172.21.15.7:0/867494177 learned_addr learned my addr 172.21.15.7:0/867494177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f890d39c700 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b8f20 0x562ec77b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f890d39c700 1 -- 172.21.15.7:0/867494177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 msgr2=0x562ec77c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f890d39c700 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f890d39c700 1 -- 172.21.15.7:0/867494177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 msgr2=0x562ec77ba090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f890d39c700 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77ba090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.546+0000 7f890d39c700 1 -- 172.21.15.7:0/867494177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec7581350 con 0x562ec77b8f20 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.547+0000 7f890d39c700 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b8f20 0x562ec77b9340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f88f800bbf0 tx=0x7f88f800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaa3d6926a6924e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.547+0000 7f88eeffd700 1 -- 172.21.15.7:0/867494177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800fbc0 con 0x562ec77b8f20 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.547+0000 7f88eeffd700 1 -- 172.21.15.7:0/867494177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f8004cf0 con 0x562ec77b8f20 2024-01-11T18:05:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.547+0000 7f88eeffd700 1 -- 172.21.15.7:0/867494177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f80056f0 con 0x562ec77b8f20 2024-01-11T18:05:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.548+0000 7f8905ffb700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f88f00a1f50 0x7f88f00a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.548+0000 7f8905ffb700 1 -- 172.21.15.7:0/564005980 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8900006e40 con 0x7f8908107be0 2024-01-11T18:05:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 -- 172.21.15.7:0/867494177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b8f20 msgr2=0x562ec77b9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b8f20 0x562ec77b9340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f88f800bbf0 tx=0x7f88f800bf00 comp rx=0 tx=0).stop 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f890cb9b700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f88f00a1f50 0x7f88f00a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 -- 172.21.15.7:0/867494177 shutdown_connections 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b8f20 0x562ec77b9340 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/867494177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 -- 172.21.15.7:0/867494177 >> 172.21.15.7:0/867494177 conn(0x562ec772e740 msgr2=0x562ec77b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.549+0000 7f8922bf1b80 1 -- 172.21.15.7:0/867494177 shutdown_connections 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.550+0000 7f8922bf1b80 1 -- 172.21.15.7:0/867494177 wait complete. 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.550+0000 7f8922bf1b80 1 -- start start 2024-01-11T18:05:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 Processor -- start 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b8f20 0x562ec77d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec770ccd0 con 0x562ec77ba5d0 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec767b340 con 0x562ec77b9c10 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec767afe0 con 0x562ec77b8f20 2024-01-11T18:05:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.551+0000 7f8922bf1b80 10 client.0 _ll_register_callbacks cb 0x562ec76ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.552+0000 7f890d39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.552+0000 7f8907fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.552+0000 7f890d39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77daea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36488/0 (socket says 172.21.15.7:36488) 2024-01-11T18:05:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.552+0000 7f890d39c700 1 -- 172.21.15.7:0/3298375279 learned_addr learned my addr 172.21.15.7:0/3298375279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.553+0000 7f890cb9b700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f88f00a1f50 0x7f88f00a4410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8900097550 tx=0x7f890009ee80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.553+0000 7f890d39c700 1 -- 172.21.15.7:0/3298375279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b8f20 msgr2=0x562ec77d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.553+0000 7f890d39c700 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b8f20 0x562ec77d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.553+0000 7f890d39c700 1 -- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 msgr2=0x562ec77d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.554+0000 7f890d39c700 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77d3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.554+0000 7f890d39c700 1 -- 172.21.15.7:0/3298375279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f800b820 con 0x562ec77ba5d0 2024-01-11T18:05:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.554+0000 7f8907fff700 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.556+0000 7f890d39c700 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77daea0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000c00 tx=0x7f88f8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293d49ff48958e4f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.556+0000 7f88e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:05:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.557+0000 7f88e7fff700 1 -- 172.21.15.7:0/3298375279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f801d070 con 0x562ec77ba5d0 2024-01-11T18:05:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.557+0000 7f88e7fff700 1 -- 172.21.15.7:0/3298375279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f88f8004960 con 0x562ec77ba5d0 2024-01-11T18:05:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.557+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ec77abd60 con 0x562ec77ba5d0 2024-01-11T18:05:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.557+0000 7f88e7fff700 1 -- 172.21.15.7:0/3298375279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f8004b20 con 0x562ec77ba5d0 2024-01-11T18:05:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.557+0000 7f8922bf1b80 10 client.16029 fetch_fsmap learned FSMap version 37 2024-01-11T18:05:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.557+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562ec752db00 con 0x562ec77ba5d0 2024-01-11T18:05:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.561+0000 7f88e7fff700 1 -- 172.21.15.7:0/3298375279 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ec77abd60 con 0x562ec77ba5d0 2024-01-11T18:05:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.562+0000 7f88e7fff700 1 -- 172.21.15.7:0/3298375279 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f88f802d070 con 0x562ec77ba5d0 2024-01-11T18:05:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.563+0000 7f8922bf1b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:05:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.563+0000 7f8922bf1b80 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.563+0000 7f8922bf1b80 20 client.16029 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.563+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec77e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.563+0000 7f8922bf1b80 4 client.16029 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:05:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.563+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ec7705b60 con 0x562ec77df5b0 2024-01-11T18:05:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.564+0000 7f8907fff700 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec77e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.567+0000 7f88e7fff700 10 client.16029 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.567+0000 7f8907fff700 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec77e19d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419ffe48575fa2b9 server_cookie=d8c5d829e6d168fc in_seq=0 out_seq=0 2024-01-11T18:05:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.569+0000 7f88e7fff700 1 -- 172.21.15.7:0/3298375279 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ec7705b60 con 0x562ec77df5b0 2024-01-11T18:05:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.569+0000 7f88e7fff700 10 client.16029 handle_command_reply: tid=0 2024-01-11T18:05:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.569+0000 7f8922bf1b80 1 client.16029 shutdown 2024-01-11T18:05:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.569+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 msgr2=0x562ec77daea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.569+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77daea0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000c00 tx=0x7f88f8005050 comp rx=0 tx=0).stop 2024-01-11T18:05:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 shutdown_connections 2024-01-11T18:05:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec77e19d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b8f20 0x562ec77d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f88e7fff700 0 client.16029 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77daea0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/3298375279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b9c10 0x562ec77d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 >> 172.21.15.7:0/3298375279 conn(0x562ec772e740 msgr2=0x562ec77b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 shutdown_connections 2024-01-11T18:05:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 1 -- 172.21.15.7:0/3298375279 wait complete. 2024-01-11T18:05:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.570+0000 7f8922bf1b80 20 client.16029 trim_cache size 0 max 16384 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 Processor -- start 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 -- start start 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b9c10 0x562ec77dae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec7583640 con 0x562ec77ba5d0 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec77b8bb0 con 0x562ec77b8f20 2024-01-11T18:05:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.602+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec77dc020 con 0x562ec77b9c10 2024-01-11T18:05:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.603+0000 7f890d39c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.603+0000 7f890d39c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35734/0 (socket says 172.21.15.7:35734) 2024-01-11T18:05:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.603+0000 7f890d39c700 1 -- 172.21.15.7:0/160734386 learned_addr learned my addr 172.21.15.7:0/160734386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.604+0000 7f890d39c700 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.604+0000 7f890d39c700 1 -- 172.21.15.7:0/160734386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b9c10 msgr2=0x562ec77dae60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.604+0000 7f890d39c700 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b9c10 0x562ec77dae60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.604+0000 7f890d39c700 1 -- 172.21.15.7:0/160734386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 msgr2=0x562ec77c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.604+0000 7f890d39c700 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.604+0000 7f890d39c700 1 -- 172.21.15.7:0/160734386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec76ce1d0 con 0x562ec77b8f20 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.605+0000 7f890d39c700 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77b9ff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f88f800f770 tx=0x7f88f8004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a3e547bfb69443 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.606+0000 7f88ee7fc700 1 -- 172.21.15.7:0/160734386 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f801d040 con 0x562ec77b8f20 2024-01-11T18:05:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.611+0000 7f88ee7fc700 1 -- 172.21.15.7:0/160734386 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f8005170 con 0x562ec77b8f20 2024-01-11T18:05:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.611+0000 7f88ee7fc700 1 -- 172.21.15.7:0/160734386 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8011800 con 0x562ec77b8f20 2024-01-11T18:05:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.612+0000 7f8922bf1b80 1 -- 172.21.15.7:0/160734386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 msgr2=0x562ec77b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.612+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77b9ff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f88f800f770 tx=0x7f88f8004690 comp rx=0 tx=0).stop 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.613+0000 7f8922bf1b80 1 -- 172.21.15.7:0/160734386 shutdown_connections 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.613+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77ba5d0 0x562ec77c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.613+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77b9ff0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.614+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/160734386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77b9c10 0x562ec77dae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.614+0000 7f8922bf1b80 1 -- 172.21.15.7:0/160734386 >> 172.21.15.7:0/160734386 conn(0x562ec772e740 msgr2=0x562ec77af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.614+0000 7f8922bf1b80 1 -- 172.21.15.7:0/160734386 shutdown_connections 2024-01-11T18:05:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.614+0000 7f8922bf1b80 1 -- 172.21.15.7:0/160734386 wait complete. 2024-01-11T18:05:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.615+0000 7f8922bf1b80 1 -- start start 2024-01-11T18:05:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.615+0000 7f8922bf1b80 1 Processor -- start 2024-01-11T18:05:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.616+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.616+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 0x562ec77d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.617+0000 7f8922bf1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.617+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec77d9e90 con 0x562ec77b9c10 2024-01-11T18:05:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.617+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec77da000 con 0x562ec77b8f20 2024-01-11T18:05:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.617+0000 7f8922bf1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec77da1c0 con 0x562ec77ba5d0 2024-01-11T18:05:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.617+0000 7f8922bf1b80 10 client.0 _ll_register_callbacks cb 0x562ec76ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.618+0000 7f8907fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 0x562ec77d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.618+0000 7f8907fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 0x562ec77d8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36504/0 (socket says 172.21.15.7:36504) 2024-01-11T18:05:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.618+0000 7f8907fff700 1 -- 172.21.15.7:0/2169259554 learned_addr learned my addr 172.21.15.7:0/2169259554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.618+0000 7f890d39c700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f890d39c700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f8907fff700 1 -- 172.21.15.7:0/2169259554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 msgr2=0x562ec77d9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f8907fff700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77d9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f8907fff700 1 -- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 msgr2=0x562ec77d8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f8907fff700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f8907fff700 1 -- 172.21.15.7:0/2169259554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ec752db00 con 0x562ec77b9c10 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.619+0000 7f890d39c700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.620+0000 7f8907fff700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 0x562ec77d8bc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098f0 tx=0x7f88fc00f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa431142a972351 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.620+0000 7f88e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:05:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.620+0000 7f88e7fff700 1 -- 172.21.15.7:0/2169259554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc01c040 con 0x562ec77b9c10 2024-01-11T18:05:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.621+0000 7f88e7fff700 1 -- 172.21.15.7:0/2169259554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f88fc000e40 con 0x562ec77b9c10 2024-01-11T18:05:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.621+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88f8004cf0 con 0x562ec77b9c10 2024-01-11T18:05:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.621+0000 7f88e7fff700 1 -- 172.21.15.7:0/2169259554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88fc01c040 con 0x562ec77b9c10 2024-01-11T18:05:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.622+0000 7f8922bf1b80 10 client.16035 fetch_fsmap learned FSMap version 37 2024-01-11T18:05:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.622+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562ec7585c70 con 0x562ec77b9c10 2024-01-11T18:05:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f88e7fff700 1 -- 172.21.15.7:0/2169259554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f8004cf0 con 0x562ec77b9c10 2024-01-11T18:05:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f88e7fff700 1 -- 172.21.15.7:0/2169259554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f88fc015020 con 0x562ec77b9c10 2024-01-11T18:05:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f8922bf1b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 37 2024-01-11T18:05:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f8922bf1b80 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f8922bf1b80 20 client.16035 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec78383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f8922bf1b80 4 client.16035 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:05:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.623+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ec7705880 con 0x562ec77df5b0 2024-01-11T18:05:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.626+0000 7f890d39c700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec78383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.626+0000 7f890d39c700 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec78383b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5726ede81ed5a9 server_cookie=fa584588891cc5e2 in_seq=0 out_seq=0 2024-01-11T18:05:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.626+0000 7f88e7fff700 10 client.16035 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.670+0000 7f88e7fff700 1 -- 172.21.15.7:0/2169259554 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ec7705880 con 0x562ec77df5b0 2024-01-11T18:05:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f88e7fff700 10 client.16035 handle_command_reply: tid=0 2024-01-11T18:05:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 client.16035 shutdown 2024-01-11T18:05:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 msgr2=0x562ec77d8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 0x562ec77d8bc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098f0 tx=0x7f88fc00f800 comp rx=0 tx=0).stop 2024-01-11T18:05:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 shutdown_connections 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562ec77df5b0 0x562ec78383b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562ec77ba5d0 0x562ec77d9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec77b8f20 0x562ec77d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f88e7fff700 0 client.16035 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 --2- 172.21.15.7:0/2169259554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec77b9c10 0x562ec77d8bc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.671+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 >> 172.21.15.7:0/2169259554 conn(0x562ec772e740 msgr2=0x562ec77d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.672+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 shutdown_connections 2024-01-11T18:05:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.672+0000 7f8922bf1b80 1 -- 172.21.15.7:0/2169259554 wait complete. 2024-01-11T18:05:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.672+0000 7f8922bf1b80 20 client.16035 trim_cache size 0 max 16384 2024-01-11T18:05:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:05:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:05:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:05:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:05:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f88f00a1f50 msgr2=0x7f88f00a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f88f00a1f50 0x7f88f00a4410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8900097550 tx=0x7f890009ee80 comp rx=0 tx=0).stop 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 msgr2=0x7f890819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 0x7f890819dde0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f890000bd40 tx=0x7f890000be20 comp rx=0 tx=0).stop 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 shutdown_connections 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f88f00a1f50 0x7f88f00a4410 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8908107be0 0x7f890819dde0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89081a3380 0x7f89081a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 --2- 172.21.15.7:0/564005980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8908109770 0x7f890819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 >> 172.21.15.7:0/564005980 conn(0x7f8908076b60 msgr2=0x7f890810c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.674+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 shutdown_connections 2024-01-11T18:05:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:36.675+0000 7f890edff700 1 -- 172.21.15.7:0/564005980 wait complete. 2024-01-11T18:05:37.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:36 smithi016 ceph-mon[43195]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 565 op/s 2024-01-11T18:05:37.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:36 smithi038 ceph-mon[43582]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 565 op/s 2024-01-11T18:05:37.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:36 smithi007 ceph-mon[33361]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 565 op/s 2024-01-11T18:05:39.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 430.0 s, 495.6 tps, lat 198.576 ms stddev 226.760 2024-01-11T18:05:40.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:39 smithi016 ceph-mon[43195]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2024-01-11T18:05:40.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:39 smithi038 ceph-mon[43582]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2024-01-11T18:05:40.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:39 smithi007 ceph-mon[33361]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2024-01-11T18:05:41.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:40 smithi038 ceph-mon[43582]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-01-11T18:05:41.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:40 smithi007 ceph-mon[33361]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-01-11T18:05:41.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:40 smithi016 ceph-mon[43195]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-01-11T18:05:44.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 435.0 s, 689.6 tps, lat 147.383 ms stddev 179.588 2024-01-11T18:05:44.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:43 smithi016 ceph-mon[43195]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 518 op/s 2024-01-11T18:05:44.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:43 smithi038 ceph-mon[43582]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 518 op/s 2024-01-11T18:05:44.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:43 smithi007 ceph-mon[33361]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 518 op/s 2024-01-11T18:05:45.989 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:45 smithi016 ceph-mon[43195]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-01-11T18:05:46.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:45 smithi038 ceph-mon[43582]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-01-11T18:05:46.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:45 smithi007 ceph-mon[33361]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2024-01-11T18:05:48.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:47 smithi016 ceph-mon[43195]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.2 MiB/s wr, 502 op/s 2024-01-11T18:05:48.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:47 smithi038 ceph-mon[43582]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.2 MiB/s wr, 502 op/s 2024-01-11T18:05:48.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:47 smithi007 ceph-mon[33361]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.2 MiB/s wr, 502 op/s 2024-01-11T18:05:49.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 440.0 s, 546.0 tps, lat 182.350 ms stddev 208.329 2024-01-11T18:05:50.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:49 smithi016 ceph-mon[43195]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 506 op/s 2024-01-11T18:05:50.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:49 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:05:50.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:49 smithi038 ceph-mon[43582]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 506 op/s 2024-01-11T18:05:50.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:49 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:05:50.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:49 smithi007 ceph-mon[33361]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 506 op/s 2024-01-11T18:05:50.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:49 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:05:51.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:50 smithi016 ceph-mon[43195]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-01-11T18:05:51.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:50 smithi038 ceph-mon[43582]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-01-11T18:05:51.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:50 smithi007 ceph-mon[33361]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-01-11T18:05:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.236+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/2184285316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58102020 msgr2=0x7fdb58104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.236+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/2184285316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58102020 0x7fdb58104410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009990 tx=0x7fdb54009ca0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.238+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/2184285316 shutdown_connections 2024-01-11T18:05:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.238+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/2184285316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58102020 0x7fdb58104410 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.238+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/2184285316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 0x7fdb581096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.238+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/2184285316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb58104950 0x7fdb58106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.238+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/2184285316 >> 172.21.15.7:0/2184285316 conn(0x7fdb580fba30 msgr2=0x7fdb580fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.239+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/2184285316 shutdown_connections 2024-01-11T18:05:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.239+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/2184285316 wait complete. 2024-01-11T18:05:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.241+0000 7fdb5f3ff700 1 Processor -- start 2024-01-11T18:05:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.241+0000 7fdb5f3ff700 1 -- start start 2024-01-11T18:05:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.242+0000 7fdb5f3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb58102020 0x7fdb5819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.242+0000 7fdb5f3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 0x7fdb5819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.242+0000 7fdb5f3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 0x7fdb581a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.242+0000 7fdb5f3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581a5600 con 0x7fdb58102020 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.242+0000 7fdb5f3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581a5740 con 0x7fdb58107280 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.242+0000 7fdb5f3ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581a58d0 con 0x7fdb58104950 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 0x7fdb5819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 0x7fdb5819dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:32850/0 (socket says 172.21.15.7:32850) 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 -- 172.21.15.7:0/968191547 learned_addr learned my addr 172.21.15.7:0/968191547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5d99c700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 0x7fdb581a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb58102020 0x7fdb5819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 -- 172.21.15.7:0/968191547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 msgr2=0x7fdb581a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 0x7fdb581a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 -- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb58102020 msgr2=0x7fdb5819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb58102020 0x7fdb5819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.243+0000 7fdb5c99a700 1 -- 172.21.15.7:0/968191547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb540095f0 con 0x7fdb58104950 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.244+0000 7fdb5d99c700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 0x7fdb581a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.244+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 0x7fdb5819dfc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800e9a0 tx=0x7fdb4800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.245+0000 7fdb4e7fc700 1 -- 172.21.15.7:0/968191547 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48009790 con 0x7fdb58104950 2024-01-11T18:05:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.245+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb581a5bb0 con 0x7fdb58104950 2024-01-11T18:05:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.245+0000 7fdb4e7fc700 1 -- 172.21.15.7:0/968191547 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb48010430 con 0x7fdb58104950 2024-01-11T18:05:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.245+0000 7fdb4e7fc700 1 -- 172.21.15.7:0/968191547 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48018680 con 0x7fdb58104950 2024-01-11T18:05:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.245+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb581a6110 con 0x7fdb58104950 2024-01-11T18:05:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.251+0000 7fdb4e7fc700 1 -- 172.21.15.7:0/968191547 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fdb480187e0 con 0x7fdb58104950 2024-01-11T18:05:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.253+0000 7fdb4e7fc700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb440a1f10 0x7fdb440a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.254+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb440a1f10 0x7fdb440a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.254+0000 7fdb4e7fc700 1 -- 172.21.15.7:0/968191547 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdb48014070 con 0x7fdb58104950 2024-01-11T18:05:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.863+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb440a1f10 0x7fdb440a43d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b2e0 tx=0x7fdb54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.940+0000 7fdb73036b80 1 Processor -- start 2024-01-11T18:05:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.940+0000 7fdb73036b80 1 -- start start 2024-01-11T18:05:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e951340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e952090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 0x55b15e958e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb73036b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e848030 con 0x55b15e950f20 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb73036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e8abac0 con 0x55b15e9525d0 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb73036b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e8a93e0 con 0x55b15e951c10 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb5d99c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 0x55b15e958e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb5d99c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 0x55b15e958e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33248/0 (socket says 172.21.15.7:33248) 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb5d99c700 1 -- 172.21.15.7:0/334291392 learned_addr learned my addr 172.21.15.7:0/334291392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.941+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e952090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.942+0000 7fdb5d99c700 1 -- 172.21.15.7:0/334291392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 msgr2=0x55b15e952090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.942+0000 7fdb5d99c700 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e952090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.942+0000 7fdb5d99c700 1 -- 172.21.15.7:0/334291392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 msgr2=0x55b15e951340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.942+0000 7fdb5d99c700 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e951340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.942+0000 7fdb5d99c700 1 -- 172.21.15.7:0/334291392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b15e719350 con 0x55b15e9525d0 2024-01-11T18:05:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.942+0000 7fdb5d99c700 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 0x55b15e958e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb50003eb0 tx=0x7fdb50009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c7420cba66230f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.958+0000 7fdb437fe700 1 -- 172.21.15.7:0/334291392 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500092d0 con 0x55b15e9525d0 2024-01-11T18:05:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.958+0000 7fdb437fe700 1 -- 172.21.15.7:0/334291392 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5000f040 con 0x55b15e9525d0 2024-01-11T18:05:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.958+0000 7fdb437fe700 1 -- 172.21.15.7:0/334291392 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000e8b0 con 0x55b15e9525d0 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.959+0000 7fdb73036b80 1 -- 172.21.15.7:0/334291392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 msgr2=0x55b15e958e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.959+0000 7fdb73036b80 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 0x55b15e958e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb50003eb0 tx=0x7fdb50009920 comp rx=0 tx=0).stop 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 -- 172.21.15.7:0/334291392 shutdown_connections 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e9525d0 0x55b15e958e60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e951340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 --2- 172.21.15.7:0/334291392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e952090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 -- 172.21.15.7:0/334291392 >> 172.21.15.7:0/334291392 conn(0x55b15e8c6740 msgr2=0x55b15e94b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 -- 172.21.15.7:0/334291392 shutdown_connections 2024-01-11T18:05:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.960+0000 7fdb73036b80 1 -- 172.21.15.7:0/334291392 wait complete. 2024-01-11T18:05:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.961+0000 7fdb73036b80 1 -- start start 2024-01-11T18:05:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.961+0000 7fdb73036b80 1 Processor -- start 2024-01-11T18:05:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e950f20 0x55b15e95aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e95af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 0x55b15e961f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb5d19b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e950f20 0x55b15e95aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e95af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e95af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33262/0 (socket says 172.21.15.7:33262) 2024-01-11T18:05:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.963+0000 7fdb5d19b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e950f20 0x55b15e95aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:32874/0 (socket says 172.21.15.7:32874) 2024-01-11T18:05:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb73036b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e8a4cd0 con 0x55b15e9525d0 2024-01-11T18:05:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb73036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e813340 con 0x55b15e951c10 2024-01-11T18:05:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb73036b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e812fe0 con 0x55b15e950f20 2024-01-11T18:05:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb73036b80 10 client.0 _ll_register_callbacks cb 0x55b15e8456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb5c99a700 1 -- 172.21.15.7:0/913870735 learned_addr learned my addr 172.21.15.7:0/913870735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb5c99a700 1 -- 172.21.15.7:0/913870735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e950f20 msgr2=0x55b15e95aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e950f20 0x55b15e95aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb5c99a700 1 -- 172.21.15.7:0/913870735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 msgr2=0x55b15e961f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.964+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 0x55b15e961f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.965+0000 7fdb5c99a700 1 -- 172.21.15.7:0/913870735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb50003b90 con 0x55b15e951c10 2024-01-11T18:05:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.965+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e95af50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb48098250 tx=0x7fdb4809b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6960a2a8e957843 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.965+0000 7fdb40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:05:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.969+0000 7fdb40ff9700 1 -- 172.21.15.7:0/913870735 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4809f930 con 0x55b15e951c10 2024-01-11T18:05:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.969+0000 7fdb40ff9700 1 -- 172.21.15.7:0/913870735 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdb480a3b70 con 0x55b15e951c10 2024-01-11T18:05:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.969+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b15e943d60 con 0x55b15e951c10 2024-01-11T18:05:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.970+0000 7fdb40ff9700 1 -- 172.21.15.7:0/913870735 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb480105a0 con 0x55b15e951c10 2024-01-11T18:05:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.970+0000 7fdb73036b80 10 client.26192 fetch_fsmap learned FSMap version 38 2024-01-11T18:05:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.970+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b15e6c5b00 con 0x55b15e951c10 2024-01-11T18:05:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.970+0000 7fdb40ff9700 1 -- 172.21.15.7:0/913870735 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b15e943d60 con 0x55b15e951c10 2024-01-11T18:05:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.970+0000 7fdb40ff9700 1 -- 172.21.15.7:0/913870735 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fdb480a36e0 con 0x55b15e951c10 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.970+0000 7fdb73036b80 10 client.26192 fetch_fsmap finished waiting for FSMap version 38 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.971+0000 7fdb73036b80 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.971+0000 7fdb73036b80 20 client.26192 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.971+0000 7fdb73036b80 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e966880 0x55b15e968ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.971+0000 7fdb73036b80 4 client.26192 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.971+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b15e89db60 con 0x55b15e966880 2024-01-11T18:05:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.971+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e966880 0x55b15e968ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.972+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e966880 0x55b15e968ca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a61471a07818a49 server_cookie=7939cdc89f051bb2 in_seq=0 out_seq=0 2024-01-11T18:05:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.972+0000 7fdb40ff9700 10 client.26192 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.973+0000 7fdb40ff9700 1 -- 172.21.15.7:0/913870735 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b15e89db60 con 0x55b15e966880 2024-01-11T18:05:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.973+0000 7fdb40ff9700 10 client.26192 handle_command_reply: tid=0 2024-01-11T18:05:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 client.26192 shutdown 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 msgr2=0x55b15e95af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e95af50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb48098250 tx=0x7fdb4809b960 comp rx=0 tx=0).stop 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 shutdown_connections 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e966880 0x55b15e968ca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e950f20 0x55b15e95aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb40ff9700 0 client.26192 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 0x55b15e961f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 --2- 172.21.15.7:0/913870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e95af50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.974+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 >> 172.21.15.7:0/913870735 conn(0x55b15e8c6740 msgr2=0x55b15e94af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.975+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 shutdown_connections 2024-01-11T18:05:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.975+0000 7fdb73036b80 1 -- 172.21.15.7:0/913870735 wait complete. 2024-01-11T18:05:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:53.975+0000 7fdb73036b80 20 client.26192 trim_cache size 0 max 16384 2024-01-11T18:05:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.009+0000 7fdb73036b80 1 Processor -- start 2024-01-11T18:05:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.010+0000 7fdb73036b80 1 -- start start 2024-01-11T18:05:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.010+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e954e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e961f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 0x55b15e96d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb73036b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e71b640 con 0x55b15e950f20 2024-01-11T18:05:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb73036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e9517a0 con 0x55b15e951c10 2024-01-11T18:05:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e961f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb5d19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e954e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb5d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 0x55b15e96d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb5d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 0x55b15e96d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:32876/0 (socket says 172.21.15.7:32876) 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb5d99c700 1 -- 172.21.15.7:0/1879154651 learned_addr learned my addr 172.21.15.7:0/1879154651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb73036b80 1 -- 172.21.15.7:0/1879154651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e89db60 con 0x55b15e9525d0 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.011+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e961f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33278/0 (socket says 172.21.15.7:33278) 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.012+0000 7fdb5c99a700 1 -- 172.21.15.7:0/1879154651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 msgr2=0x55b15e96d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.012+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 0x55b15e96d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.012+0000 7fdb5c99a700 1 -- 172.21.15.7:0/1879154651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 msgr2=0x55b15e954e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.012+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e954e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.012+0000 7fdb5c99a700 1 -- 172.21.15.7:0/1879154651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b15e8661d0 con 0x55b15e951c10 2024-01-11T18:05:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.012+0000 7fdb5d99c700 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 0x55b15e96d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:05:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.013+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e961f50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800e970 tx=0x7fdb48010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f526a4a1a7160a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.013+0000 7fdb42ffd700 1 -- 172.21.15.7:0/1879154651 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48097620 con 0x55b15e951c10 2024-01-11T18:05:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.013+0000 7fdb42ffd700 1 -- 172.21.15.7:0/1879154651 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4809b5b0 con 0x55b15e951c10 2024-01-11T18:05:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.013+0000 7fdb42ffd700 1 -- 172.21.15.7:0/1879154651 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a36e0 con 0x55b15e951c10 2024-01-11T18:05:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.014+0000 7fdb73036b80 1 -- 172.21.15.7:0/1879154651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 msgr2=0x55b15e961f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.014+0000 7fdb73036b80 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e961f50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800e970 tx=0x7fdb48010c80 comp rx=0 tx=0).stop 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.016+0000 7fdb73036b80 1 -- 172.21.15.7:0/1879154651 shutdown_connections 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.016+0000 7fdb73036b80 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e9525d0 0x55b15e96d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.016+0000 7fdb73036b80 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e950f20 0x55b15e954e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.016+0000 7fdb73036b80 1 --2- 172.21.15.7:0/1879154651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e951c10 0x55b15e961f50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.016+0000 7fdb73036b80 1 -- 172.21.15.7:0/1879154651 >> 172.21.15.7:0/1879154651 conn(0x55b15e8c6740 msgr2=0x55b15e959f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.016+0000 7fdb73036b80 1 -- 172.21.15.7:0/1879154651 shutdown_connections 2024-01-11T18:05:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.017+0000 7fdb73036b80 1 -- 172.21.15.7:0/1879154651 wait complete. 2024-01-11T18:05:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.018+0000 7fdb73036b80 1 -- start start 2024-01-11T18:05:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.018+0000 7fdb73036b80 1 Processor -- start 2024-01-11T18:05:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.019+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e950f20 0x55b15e95f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.019+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e95f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.019+0000 7fdb73036b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 0x55b15e960820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb73036b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e960d60 con 0x55b15e9525d0 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb73036b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e960ed0 con 0x55b15e950f20 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb73036b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15e961090 con 0x55b15e951c10 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb73036b80 10 client.0 _ll_register_callbacks cb 0x55b15e962b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e95f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb5c99a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e95f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:32888/0 (socket says 172.21.15.7:32888) 2024-01-11T18:05:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.020+0000 7fdb5c99a700 1 -- 172.21.15.7:0/38578527 learned_addr learned my addr 172.21.15.7:0/38578527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e950f20 0x55b15e95f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5c99a700 1 -- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e950f20 msgr2=0x55b15e95f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e950f20 0x55b15e95f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5c99a700 1 -- 172.21.15.7:0/38578527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 msgr2=0x55b15e960820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 0x55b15e960820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5c99a700 1 -- 172.21.15.7:0/38578527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b15e719350 con 0x55b15e951c10 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e950f20 0x55b15e95f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb5c99a700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e95f810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb4809b770 tx=0x7fdb48097c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1490dc1a12814443 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.021+0000 7fdb40ff9700 1 -- 172.21.15.7:0/38578527 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4809f930 con 0x55b15e951c10 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.022+0000 7fdb40ff9700 1 -- 172.21.15.7:0/38578527 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdb48084bd0 con 0x55b15e951c10 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.022+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb5000f040 con 0x55b15e951c10 2024-01-11T18:05:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.022+0000 7fdb40ff9700 1 -- 172.21.15.7:0/38578527 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb48084d90 con 0x55b15e951c10 2024-01-11T18:05:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.022+0000 7fdb73036b80 10 client.26218 fetch_fsmap learned FSMap version 38 2024-01-11T18:05:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.022+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b15e6c5b00 con 0x55b15e951c10 2024-01-11T18:05:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb40ff9700 1 -- 172.21.15.7:0/38578527 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5000f040 con 0x55b15e951c10 2024-01-11T18:05:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb40ff9700 1 -- 172.21.15.7:0/38578527 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fdb48018bb0 con 0x55b15e951c10 2024-01-11T18:05:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb73036b80 10 client.26218 fetch_fsmap finished waiting for FSMap version 38 2024-01-11T18:05:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb73036b80 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:05:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb73036b80 20 client.26218 populate_metadata read hostname 'smithi007' 2024-01-11T18:05:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb73036b80 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e9bce30 0x55b15e9bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:05:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb73036b80 4 client.26218 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:05:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.023+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b15e89d880 con 0x55b15e9bce30 2024-01-11T18:05:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.024+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e9bce30 0x55b15e9bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:05:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.025+0000 7fdb5d19b700 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e9bce30 0x55b15e9bf250 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8e2f83036b2a67 server_cookie=2055fa2f57e1030d in_seq=0 out_seq=0 2024-01-11T18:05:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.025+0000 7fdb40ff9700 10 client.26218 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.072+0000 7fdb40ff9700 1 -- 172.21.15.7:0/38578527 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b15e89d880 con 0x55b15e9bce30 2024-01-11T18:05:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.072+0000 7fdb40ff9700 10 client.26218 handle_command_reply: tid=0 2024-01-11T18:05:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.072+0000 7fdb73036b80 1 client.26218 shutdown 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 msgr2=0x55b15e95f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e95f810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb4809b770 tx=0x7fdb48097c50 comp rx=0 tx=0).stop 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 shutdown_connections 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55b15e9bce30 0x55b15e9bf250 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b15e9525d0 0x55b15e960820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b15e950f20 0x55b15e95f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb40ff9700 0 client.26218 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 --2- 172.21.15.7:0/38578527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b15e951c10 0x55b15e95f810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 >> 172.21.15.7:0/38578527 conn(0x55b15e8c6740 msgr2=0x55b15e96d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 shutdown_connections 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.073+0000 7fdb73036b80 1 -- 172.21.15.7:0/38578527 wait complete. 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.074+0000 7fdb73036b80 20 client.26218 trim_cache size 0 max 16384 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:05:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb440a1f10 msgr2=0x7fdb440a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb440a1f10 0x7fdb440a43d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b2e0 tx=0x7fdb54005fd0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 msgr2=0x7fdb5819dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 0x7fdb5819dfc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800e9a0 tx=0x7fdb4800ecb0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 shutdown_connections 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fdb440a1f10 0x7fdb440a43d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb58107280 0x7fdb581a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb58102020 0x7fdb5819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 --2- 172.21.15.7:0/968191547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb58104950 0x7fdb5819dfc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:05:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.075+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 >> 172.21.15.7:0/968191547 conn(0x7fdb580fba30 msgr2=0x7fdb58102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:05:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.076+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 shutdown_connections 2024-01-11T18:05:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:05:54.076+0000 7fdb5f3ff700 1 -- 172.21.15.7:0/968191547 wait complete. 2024-01-11T18:05:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 445.0 s, 462.8 tps, lat 213.927 ms stddev 245.814 2024-01-11T18:05:54.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:53 smithi016 ceph-mon[43195]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.2 MiB/s wr, 539 op/s 2024-01-11T18:05:54.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:53 smithi038 ceph-mon[43582]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.2 MiB/s wr, 539 op/s 2024-01-11T18:05:54.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:53 smithi007 ceph-mon[33361]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.2 MiB/s wr, 539 op/s 2024-01-11T18:05:55.989 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:55 smithi016 ceph-mon[43195]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.0 MiB/s wr, 538 op/s 2024-01-11T18:05:56.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:55 smithi038 ceph-mon[43582]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.0 MiB/s wr, 538 op/s 2024-01-11T18:05:56.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:55 smithi007 ceph-mon[33361]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.0 MiB/s wr, 538 op/s 2024-01-11T18:05:58.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:57 smithi016 ceph-mon[43195]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 MiB/s wr, 517 op/s 2024-01-11T18:05:58.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:57 smithi038 ceph-mon[43582]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 MiB/s wr, 517 op/s 2024-01-11T18:05:58.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:57 smithi007 ceph-mon[33361]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 MiB/s wr, 517 op/s 2024-01-11T18:05:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 450.0 s, 426.4 tps, lat 231.968 ms stddev 291.941 2024-01-11T18:05:59.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:05:59 smithi007 ceph-mon[33361]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.0 MiB/s wr, 541 op/s 2024-01-11T18:05:59.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:05:59 smithi016 ceph-mon[43195]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.0 MiB/s wr, 541 op/s 2024-01-11T18:05:59.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:05:59 smithi038 ceph-mon[43582]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.0 MiB/s wr, 541 op/s 2024-01-11T18:06:01.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:00 smithi016 ceph-mon[43195]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 MiB/s wr, 577 op/s 2024-01-11T18:06:01.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:00 smithi038 ceph-mon[43582]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 MiB/s wr, 577 op/s 2024-01-11T18:06:01.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:00 smithi007 ceph-mon[33361]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 MiB/s wr, 577 op/s 2024-01-11T18:06:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 455.0 s, 497.6 tps, lat 197.468 ms stddev 239.671 2024-01-11T18:06:04.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:03 smithi016 ceph-mon[43195]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 600 op/s 2024-01-11T18:06:04.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:03 smithi038 ceph-mon[43582]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 600 op/s 2024-01-11T18:06:04.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:03 smithi007 ceph-mon[33361]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 600 op/s 2024-01-11T18:06:05.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:05 smithi016 ceph-mon[43195]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2024-01-11T18:06:06.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:05 smithi038 ceph-mon[43582]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2024-01-11T18:06:06.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:05 smithi007 ceph-mon[33361]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2024-01-11T18:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 -- 172.21.15.7:0/3899210751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 msgr2=0x7f4028102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 --2- 172.21.15.7:0/3899210751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 0x7f4028102090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f401c009990 tx=0x7f401c009ca0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 -- 172.21.15.7:0/3899210751 shutdown_connections 2024-01-11T18:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 --2- 172.21.15.7:0/3899210751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f4028105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 --2- 172.21.15.7:0/3899210751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 0x7f4028102090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 --2- 172.21.15.7:0/3899210751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40281025d0 0x7f4028104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.442+0000 7f402d99d700 1 -- 172.21.15.7:0/3899210751 >> 172.21.15.7:0/3899210751 conn(0x7f40280fb820 msgr2=0x7f40280fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.443+0000 7f402d99d700 1 -- 172.21.15.7:0/3899210751 shutdown_connections 2024-01-11T18:06:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.445+0000 7f402d99d700 1 -- 172.21.15.7:0/3899210751 wait complete. 2024-01-11T18:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.446+0000 7f402d99d700 1 Processor -- start 2024-01-11T18:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 -- start start 2024-01-11T18:06:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 0x7f402819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40281025d0 0x7f402819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f40281a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40281a5490 con 0x7f4028104fa0 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40281a55d0 con 0x7f40280ffc60 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.447+0000 7f402d99d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40281a5760 con 0x7f40281025d0 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.448+0000 7f402d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f40281a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.448+0000 7f402d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f40281a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35050/0 (socket says 172.21.15.7:35050) 2024-01-11T18:06:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.448+0000 7f402d19c700 1 -- 172.21.15.7:0/3851020569 learned_addr learned my addr 172.21.15.7:0/3851020569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.449+0000 7f402d19c700 1 -- 172.21.15.7:0/3851020569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40281025d0 msgr2=0x7f402819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.449+0000 7f402d19c700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40281025d0 0x7f402819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.449+0000 7f402d19c700 1 -- 172.21.15.7:0/3851020569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 msgr2=0x7f402819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.449+0000 7f402d19c700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 0x7f402819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.449+0000 7f402d19c700 1 -- 172.21.15.7:0/3851020569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0095f0 con 0x7f4028104fa0 2024-01-11T18:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.450+0000 7f402d19c700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f40281a4ec0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4020007d70 tx=0x7f402000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.450+0000 7f4025ffb700 1 -- 172.21.15.7:0/3851020569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000ad40 con 0x7f4028104fa0 2024-01-11T18:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.450+0000 7f4025ffb700 1 -- 172.21.15.7:0/3851020569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40200128e0 con 0x7f4028104fa0 2024-01-11T18:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.450+0000 7f4025ffb700 1 -- 172.21.15.7:0/3851020569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020011780 con 0x7f4028104fa0 2024-01-11T18:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.450+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40281a5a40 con 0x7f4028104fa0 2024-01-11T18:06:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.450+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40281a5f00 con 0x7f4028104fa0 2024-01-11T18:06:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.458+0000 7f4025ffb700 1 -- 172.21.15.7:0/3851020569 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4020012a50 con 0x7f4028104fa0 2024-01-11T18:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.462+0000 7f40428e6b80 1 Processor -- start 2024-01-11T18:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.462+0000 7f40428e6b80 1 -- start start 2024-01-11T18:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 0x5573f3eb2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2eb0 0x5573f3eb3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3eba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f40428e6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3da9030 con 0x5573f3eb3870 2024-01-11T18:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f40428e6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3e0cac0 con 0x5573f3eb2eb0 2024-01-11T18:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f40428e6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3e0a3e0 con 0x5573f3eb2120 2024-01-11T18:06:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f4027fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 0x5573f3eb2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f4025ffb700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40180a2010 0x7f40180a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f4027fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 0x5573f3eb2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37168/0 (socket says 172.21.15.7:37168) 2024-01-11T18:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.463+0000 7f4027fff700 1 -- 172.21.15.7:0/3884984801 learned_addr learned my addr 172.21.15.7:0/3884984801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.464+0000 7f4025ffb700 1 -- 172.21.15.7:0/3851020569 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40200cdf20 con 0x7f4028104fa0 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.464+0000 7f402c99b700 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2eb0 0x5573f3eb3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f402d19c700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40180a2010 0x7f40180a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f4027fff700 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3eba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f4027fff700 1 -- 172.21.15.7:0/3884984801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2eb0 msgr2=0x5573f3eb3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f4027fff700 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2eb0 0x5573f3eb3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f4027fff700 1 -- 172.21.15.7:0/3884984801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 msgr2=0x5573f3eba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f4027fff700 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3eba100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.465+0000 7f4027fff700 1 -- 172.21.15.7:0/3884984801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f3c7a350 con 0x5573f3eb2120 2024-01-11T18:06:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.466+0000 7f4027fff700 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 0x5573f3eb2540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4014009bb0 tx=0x7f40140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce8ffc3563dc7fa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.466+0000 7f402d19c700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40180a2010 0x7f40180a44d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f40200973c0 tx=0x7f4020007f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.468+0000 7f4012ffd700 1 -- 172.21.15.7:0/3884984801 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400f910 con 0x5573f3eb2120 2024-01-11T18:06:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.468+0000 7f4012ffd700 1 -- 172.21.15.7:0/3884984801 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401400c070 con 0x5573f3eb2120 2024-01-11T18:06:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.468+0000 7f4012ffd700 1 -- 172.21.15.7:0/3884984801 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014014e20 con 0x5573f3eb2120 2024-01-11T18:06:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 -- 172.21.15.7:0/3884984801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 msgr2=0x5573f3eb2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 0x5573f3eb2540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4014009bb0 tx=0x7f40140091a0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 -- 172.21.15.7:0/3884984801 shutdown_connections 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2120 0x5573f3eb2540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3eba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 --2- 172.21.15.7:0/3884984801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2eb0 0x5573f3eb3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.469+0000 7f40428e6b80 1 -- 172.21.15.7:0/3884984801 >> 172.21.15.7:0/3884984801 conn(0x5573f3e27740 msgr2=0x5573f3eacb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.470+0000 7f40428e6b80 1 -- 172.21.15.7:0/3884984801 shutdown_connections 2024-01-11T18:06:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.470+0000 7f40428e6b80 1 -- 172.21.15.7:0/3884984801 wait complete. 2024-01-11T18:06:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.470+0000 7f40428e6b80 1 -- start start 2024-01-11T18:06:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.471+0000 7f40428e6b80 1 Processor -- start 2024-01-11T18:06:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.473+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2120 0x5573f3ebbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.474+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 0x5573f3ebc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.474+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3ec30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.474+0000 7f40428e6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3e05cd0 con 0x5573f3eb3870 2024-01-11T18:06:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.474+0000 7f40428e6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3d74340 con 0x5573f3eb2120 2024-01-11T18:06:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.474+0000 7f40428e6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3d73fe0 con 0x5573f3eb2eb0 2024-01-11T18:06:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.474+0000 7f40428e6b80 10 client.0 _ll_register_callbacks cb 0x5573f3da66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.475+0000 7f4027fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2120 0x5573f3ebbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.475+0000 7f402c99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 0x5573f3ebc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.476+0000 7f402c99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 0x5573f3ebc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37170/0 (socket says 172.21.15.7:37170) 2024-01-11T18:06:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.476+0000 7f402c99b700 1 -- 172.21.15.7:0/2132826627 learned_addr learned my addr 172.21.15.7:0/2132826627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f4027fff700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3ec30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f402c99b700 1 -- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2120 msgr2=0x5573f3ebbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f402c99b700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2120 0x5573f3ebbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f402c99b700 1 -- 172.21.15.7:0/2132826627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 msgr2=0x5573f3ec30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f402c99b700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3ec30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f402c99b700 1 -- 172.21.15.7:0/2132826627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4014009860 con 0x5573f3eb2eb0 2024-01-11T18:06:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.478+0000 7f4027fff700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3ec30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:06:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.479+0000 7f402c99b700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 0x5573f3ebc100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f401c009f50 tx=0x7f401c0050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e957293d0d53bce server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.479+0000 7f4007fff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:06:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.481+0000 7f4007fff700 1 -- 172.21.15.7:0/2132826627 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c005650 con 0x5573f3eb2eb0 2024-01-11T18:06:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.481+0000 7f4007fff700 1 -- 172.21.15.7:0/2132826627 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f401c004710 con 0x5573f3eb2eb0 2024-01-11T18:06:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.481+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573f3ea4fb0 con 0x5573f3eb2eb0 2024-01-11T18:06:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.482+0000 7f4007fff700 1 -- 172.21.15.7:0/2132826627 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c0048d0 con 0x5573f3eb2eb0 2024-01-11T18:06:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.482+0000 7f40428e6b80 10 client.26224 fetch_fsmap learned FSMap version 38 2024-01-11T18:06:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.482+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5573f3c26b00 con 0x5573f3eb2eb0 2024-01-11T18:06:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.482+0000 7f4007fff700 1 -- 172.21.15.7:0/2132826627 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573f3ea4fb0 con 0x5573f3eb2eb0 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.482+0000 7f4007fff700 1 -- 172.21.15.7:0/2132826627 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f401c00fc60 con 0x5573f3eb2eb0 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f40428e6b80 10 client.26224 fetch_fsmap finished waiting for FSMap version 38 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f40428e6b80 10 client.26224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f40428e6b80 20 client.26224 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3ec7a30 0x5573f3ec9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f40428e6b80 4 client.26224 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573f3dfeb60 con 0x5573f3ec7a30 2024-01-11T18:06:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.483+0000 7f4027fff700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3ec7a30 0x5573f3ec9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.484+0000 7f4027fff700 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3ec7a30 0x5573f3ec9e50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b52e2808fe777d server_cookie=5db834e75a532826 in_seq=0 out_seq=0 2024-01-11T18:06:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.484+0000 7f4007fff700 10 client.26224 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.487+0000 7f4007fff700 1 -- 172.21.15.7:0/2132826627 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573f3dfeb60 con 0x5573f3ec7a30 2024-01-11T18:06:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.487+0000 7f4007fff700 10 client.26224 handle_command_reply: tid=0 2024-01-11T18:06:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.487+0000 7f40428e6b80 1 client.26224 shutdown 2024-01-11T18:06:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 msgr2=0x5573f3ebc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 0x5573f3ebc100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f401c009f50 tx=0x7f401c0050f0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 shutdown_connections 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3ec7a30 0x5573f3ec9e50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb3870 0x5573f3ec30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3eb2120 0x5573f3ebbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2132826627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb2eb0 0x5573f3ebc100 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f401c009f50 tx=0x7f401c0050f0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f4007fff700 0 client.26224 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 >> 172.21.15.7:0/2132826627 conn(0x5573f3e27740 msgr2=0x5573f3eac1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.488+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 shutdown_connections 2024-01-11T18:06:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.491+0000 7f40428e6b80 1 -- 172.21.15.7:0/2132826627 wait complete. 2024-01-11T18:06:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.491+0000 7f40428e6b80 20 client.26224 trim_cache size 0 max 16384 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 Processor -- start 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 -- start start 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ea4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ece7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3c7c640 con 0x5573f3eb2120 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3ebc820 con 0x5573f3ebb140 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.531+0000 7f40428e6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3eadd90 con 0x5573f3eb3870 2024-01-11T18:06:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ece7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ece7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54660/0 (socket says 172.21.15.7:54660) 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 -- 172.21.15.7:0/2987673100 learned_addr learned my addr 172.21.15.7:0/2987673100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f402c99b700 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ea4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 -- 172.21.15.7:0/2987673100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 msgr2=0x5573f3ec3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 -- 172.21.15.7:0/2987673100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 msgr2=0x5573f3ea4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.532+0000 7f4027fff700 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ea4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.533+0000 7f4027fff700 1 -- 172.21.15.7:0/2987673100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f3dc71d0 con 0x5573f3ebb140 2024-01-11T18:06:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.534+0000 7f4027fff700 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ece7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4014014790 tx=0x7f40140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a285a15f5f60cd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.535+0000 7f40127fc700 1 -- 172.21.15.7:0/2987673100 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014009270 con 0x5573f3ebb140 2024-01-11T18:06:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.535+0000 7f40127fc700 1 -- 172.21.15.7:0/2987673100 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4014027070 con 0x5573f3ebb140 2024-01-11T18:06:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.535+0000 7f40127fc700 1 -- 172.21.15.7:0/2987673100 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401401f450 con 0x5573f3ebb140 2024-01-11T18:06:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 -- 172.21.15.7:0/2987673100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 msgr2=0x5573f3ece7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ece7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4014014790 tx=0x7f40140091a0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 -- 172.21.15.7:0/2987673100 shutdown_connections 2024-01-11T18:06:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ece7b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ea4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2987673100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.536+0000 7f40428e6b80 1 -- 172.21.15.7:0/2987673100 >> 172.21.15.7:0/2987673100 conn(0x5573f3e27740 msgr2=0x5573f3ea8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.537+0000 7f40428e6b80 1 -- 172.21.15.7:0/2987673100 shutdown_connections 2024-01-11T18:06:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.537+0000 7f40428e6b80 1 -- 172.21.15.7:0/2987673100 wait complete. 2024-01-11T18:06:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.537+0000 7f40428e6b80 1 -- start start 2024-01-11T18:06:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.540+0000 7f40428e6b80 1 Processor -- start 2024-01-11T18:06:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.541+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ec0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.541+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f40428e6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ec22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f40428e6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3ec2800 con 0x5573f3eb2120 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f40428e6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3ec2970 con 0x5573f3ebb140 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f40428e6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f3ec2b30 con 0x5573f3eb3870 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f40428e6b80 10 client.0 _ll_register_callbacks cb 0x5573f3da66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f402c99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f402c99b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37192/0 (socket says 172.21.15.7:37192) 2024-01-11T18:06:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f402c99b700 1 -- 172.21.15.7:0/2779039595 learned_addr learned my addr 172.21.15.7:0/2779039595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.542+0000 7f4027fff700 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ec22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.543+0000 7f402c99b700 1 -- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 msgr2=0x5573f3ec22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.543+0000 7f402c99b700 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ec22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.543+0000 7f402c99b700 1 -- 172.21.15.7:0/2779039595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 msgr2=0x5573f3ec0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.543+0000 7f402c99b700 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ec0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.543+0000 7f402c99b700 1 -- 172.21.15.7:0/2779039595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573f3c7a350 con 0x5573f3eb3870 2024-01-11T18:06:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.544+0000 7f402c99b700 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec12b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f401c017630 tx=0x7f401c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32259a1b3cbb5530 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.544+0000 7f4007fff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:06:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.544+0000 7f4007fff700 1 -- 172.21.15.7:0/2779039595 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c016cd0 con 0x5573f3eb3870 2024-01-11T18:06:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.544+0000 7f4007fff700 1 -- 172.21.15.7:0/2779039595 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f401c016e30 con 0x5573f3eb3870 2024-01-11T18:06:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.544+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f401400c070 con 0x5573f3eb3870 2024-01-11T18:06:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.545+0000 7f4007fff700 1 -- 172.21.15.7:0/2779039595 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c0056e0 con 0x5573f3eb3870 2024-01-11T18:06:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.545+0000 7f40428e6b80 10 client.26230 fetch_fsmap learned FSMap version 38 2024-01-11T18:06:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.545+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5573f3c7ec70 con 0x5573f3eb3870 2024-01-11T18:06:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f4007fff700 1 -- 172.21.15.7:0/2779039595 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401400c070 con 0x5573f3eb3870 2024-01-11T18:06:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f4007fff700 1 -- 172.21.15.7:0/2779039595 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f401c00b260 con 0x5573f3eb3870 2024-01-11T18:06:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f40428e6b80 10 client.26230 fetch_fsmap finished waiting for FSMap version 38 2024-01-11T18:06:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f40428e6b80 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f40428e6b80 20 client.26230 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3f1eaa0 0x5573f3f20ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f40428e6b80 4 client.26230 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:06:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.546+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573f3dfe880 con 0x5573f3f1eaa0 2024-01-11T18:06:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.547+0000 7f4027fff700 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3f1eaa0 0x5573f3f20ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.548+0000 7f4027fff700 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3f1eaa0 0x5573f3f20ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653369b60ff660a9 server_cookie=5e6cdf8481dad141 in_seq=0 out_seq=0 2024-01-11T18:06:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.548+0000 7f4007fff700 10 client.26230 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.673+0000 7f4007fff700 1 -- 172.21.15.7:0/2779039595 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573f3dfe880 con 0x5573f3f1eaa0 2024-01-11T18:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.673+0000 7f4007fff700 10 client.26230 handle_command_reply: tid=0 2024-01-11T18:06:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.674+0000 7f40428e6b80 1 client.26230 shutdown 2024-01-11T18:06:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.674+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 msgr2=0x5573f3ec12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.674+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec12b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f401c017630 tx=0x7f401c005c70 comp rx=0 tx=0).stop 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.674+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 shutdown_connections 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.674+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5573f3f1eaa0 0x5573f3f20ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.674+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573f3ebb140 0x5573f3ec22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f4007fff700 0 client.26230 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573f3eb2120 0x5573f3ec0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f40428e6b80 1 --2- 172.21.15.7:0/2779039595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573f3eb3870 0x5573f3ec12b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 >> 172.21.15.7:0/2779039595 conn(0x5573f3e27740 msgr2=0x5573f3ece7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 shutdown_connections 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f40428e6b80 1 -- 172.21.15.7:0/2779039595 wait complete. 2024-01-11T18:06:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.675+0000 7f40428e6b80 20 client.26230 trim_cache size 0 max 16384 2024-01-11T18:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:06:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40180a2010 msgr2=0x7f40180a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40180a2010 0x7f40180a44d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f40200973c0 tx=0x7f4020007f70 comp rx=0 tx=0).stop 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 msgr2=0x7f40281a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f40281a4ec0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4020007d70 tx=0x7f402000a340 comp rx=0 tx=0).stop 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 shutdown_connections 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40180a2010 0x7f40180a44d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4028104fa0 0x7f40281a4ec0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40280ffc60 0x7f402819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 --2- 172.21.15.7:0/3851020569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40281025d0 0x7f402819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.676+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 >> 172.21.15.7:0/3851020569 conn(0x7f40280fb820 msgr2=0x7f4028102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.677+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 shutdown_connections 2024-01-11T18:06:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:07.677+0000 7f402d99d700 1 -- 172.21.15.7:0/3851020569 wait complete. 2024-01-11T18:06:08.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:07 smithi016 ceph-mon[43195]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.2 MiB/s wr, 569 op/s 2024-01-11T18:06:08.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:07 smithi038 ceph-mon[43582]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.2 MiB/s wr, 569 op/s 2024-01-11T18:06:08.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:07 smithi007 ceph-mon[33361]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.2 MiB/s wr, 569 op/s 2024-01-11T18:06:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 460.0 s, 612.2 tps, lat 170.838 ms stddev 215.638 2024-01-11T18:06:09.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:08 smithi016 ceph-mon[43195]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.1 MiB/s wr, 562 op/s 2024-01-11T18:06:09.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:08 smithi038 ceph-mon[43582]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.1 MiB/s wr, 562 op/s 2024-01-11T18:06:09.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:08 smithi007 ceph-mon[33361]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.1 MiB/s wr, 562 op/s 2024-01-11T18:06:10.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:09 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:10.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:09 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:10.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:09 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:11.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:10 smithi016 ceph-mon[43195]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 573 op/s 2024-01-11T18:06:11.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:10 smithi038 ceph-mon[43582]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 573 op/s 2024-01-11T18:06:11.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:10 smithi007 ceph-mon[33361]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 573 op/s 2024-01-11T18:06:14.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 465.0 s, 590.8 tps, lat 166.485 ms stddev 216.873 2024-01-11T18:06:14.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:13 smithi016 ceph-mon[43195]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 545 op/s 2024-01-11T18:06:14.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:13 smithi038 ceph-mon[43582]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 545 op/s 2024-01-11T18:06:14.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:13 smithi007 ceph-mon[33361]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 545 op/s 2024-01-11T18:06:15.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:14 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:15.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:14 smithi016 ceph-mon[43195]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 498 op/s 2024-01-11T18:06:15.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:14 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:15.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:14 smithi038 ceph-mon[43582]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 498 op/s 2024-01-11T18:06:15.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:14 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:15.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:14 smithi007 ceph-mon[33361]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 498 op/s 2024-01-11T18:06:18.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:17 smithi016 ceph-mon[43195]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 MiB/s wr, 495 op/s 2024-01-11T18:06:18.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:17 smithi038 ceph-mon[43582]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 MiB/s wr, 495 op/s 2024-01-11T18:06:18.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:17 smithi007 ceph-mon[33361]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 MiB/s wr, 495 op/s 2024-01-11T18:06:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 470.0 s, 547.8 tps, lat 184.242 ms stddev 219.383 2024-01-11T18:06:20.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:19 smithi016 ceph-mon[43195]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 508 op/s 2024-01-11T18:06:20.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:19 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:20.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:19 smithi038 ceph-mon[43582]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 508 op/s 2024-01-11T18:06:20.274 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:19 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:20.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:19 smithi007 ceph-mon[33361]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 508 op/s 2024-01-11T18:06:20.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:19 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:21.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:20 smithi038 ceph-mon[43582]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.5 MiB/s wr, 528 op/s 2024-01-11T18:06:21.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:20 smithi007 ceph-mon[33361]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.5 MiB/s wr, 528 op/s 2024-01-11T18:06:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.487+0000 7febccdff700 1 -- 172.21.15.7:0/478540129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8107280 msgr2=0x7febc81096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.487+0000 7febccdff700 1 --2- 172.21.15.7:0/478540129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8107280 0x7febc81096f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7febc000b0a0 tx=0x7febc000b3b0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.487+0000 7febccdff700 1 -- 172.21.15.7:0/478540129 shutdown_connections 2024-01-11T18:06:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.487+0000 7febccdff700 1 --2- 172.21.15.7:0/478540129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8107280 0x7febc81096f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.487+0000 7febccdff700 1 --2- 172.21.15.7:0/478540129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102020 0x7febc8104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.488+0000 7febccdff700 1 --2- 172.21.15.7:0/478540129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.488+0000 7febccdff700 1 -- 172.21.15.7:0/478540129 >> 172.21.15.7:0/478540129 conn(0x7febc80fba30 msgr2=0x7febc80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.488+0000 7febccdff700 1 -- 172.21.15.7:0/478540129 shutdown_connections 2024-01-11T18:06:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.489+0000 7febccdff700 1 -- 172.21.15.7:0/478540129 wait complete. 2024-01-11T18:06:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.490+0000 7febccdff700 1 Processor -- start 2024-01-11T18:06:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.490+0000 7febccdff700 1 -- start start 2024-01-11T18:06:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.494+0000 7febccdff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8102020 0x7febc8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.494+0000 7febccdff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:20 smithi016 ceph-mon[43195]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.5 MiB/s wr, 528 op/s 2024-01-11T18:06:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.494+0000 7febccdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8107280 0x7febc81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.494+0000 7febccdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81a1190 con 0x7febc8107280 2024-01-11T18:06:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.494+0000 7febccdff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81a12d0 con 0x7febc8104950 2024-01-11T18:06:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.494+0000 7febccdff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81a1460 con 0x7febc8102020 2024-01-11T18:06:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34620/0 (socket says 172.21.15.7:34620) 2024-01-11T18:06:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 -- 172.21.15.7:0/3419264265 learned_addr learned my addr 172.21.15.7:0/3419264265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc6d9d700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8107280 0x7febc81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 -- 172.21.15.7:0/3419264265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8102020 msgr2=0x7febc8199610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8102020 0x7febc8199610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 -- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8107280 msgr2=0x7febc81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8107280 0x7febc81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc5d9b700 1 -- 172.21.15.7:0/3419264265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009d00 con 0x7febc8104950 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.495+0000 7febc6d9d700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8107280 0x7febc81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.496+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8199b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c9b0 tx=0x7febbc00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.496+0000 7febb77fe700 1 -- 172.21.15.7:0/3419264265 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007b00 con 0x7febc8104950 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.496+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc81a1740 con 0x7febc8104950 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.496+0000 7febb77fe700 1 -- 172.21.15.7:0/3419264265 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febbc007c60 con 0x7febc8104950 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.496+0000 7febb77fe700 1 -- 172.21.15.7:0/3419264265 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00e610 con 0x7febc8104950 2024-01-11T18:06:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.496+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81a1d20 con 0x7febc8104950 2024-01-11T18:06:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.502+0000 7febb77fe700 1 -- 172.21.15.7:0/3419264265 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7febbc00e830 con 0x7febc8104950 2024-01-11T18:06:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.504+0000 7febb77fe700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febb00a1fd0 0x7febb00a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.505+0000 7febb77fe700 1 -- 172.21.15.7:0/3419264265 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7febbc0cf0e0 con 0x7febc8104950 2024-01-11T18:06:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.506+0000 7febc659c700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febb00a1fd0 0x7febb00a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.518+0000 7febe0bb3b80 1 Processor -- start 2024-01-11T18:06:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.518+0000 7febe0bb3b80 1 -- start start 2024-01-11T18:06:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.518+0000 7febc659c700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febb00a1fd0 0x7febb00a4490 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7febb8009730 tx=0x7febb8006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e749340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e74a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e750e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e640030 con 0x55d07e748f20 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e6a3ac0 con 0x55d07e749c10 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febc6d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e750e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febc6d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e750e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44818/0 (socket says 172.21.15.7:44818) 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febc6d9d700 1 -- 172.21.15.7:0/2482588752 learned_addr learned my addr 172.21.15.7:0/2482588752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febc659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e749340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febc5d9b700 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e74a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.519+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e6a13e0 con 0x55d07e74a5d0 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.520+0000 7febc6d9d700 1 -- 172.21.15.7:0/2482588752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 msgr2=0x55d07e74a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.520+0000 7febc6d9d700 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e74a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.520+0000 7febc6d9d700 1 -- 172.21.15.7:0/2482588752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 msgr2=0x55d07e749340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.520+0000 7febc6d9d700 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e749340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.520+0000 7febc6d9d700 1 -- 172.21.15.7:0/2482588752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d07e511350 con 0x55d07e74a5d0 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.520+0000 7febc6d9d700 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e750e60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7febc0009820 tx=0x7febc0008db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0acb78b54029a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.521+0000 7febb4ff9700 1 -- 172.21.15.7:0/2482588752 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc000e050 con 0x55d07e74a5d0 2024-01-11T18:06:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.521+0000 7febb4ff9700 1 -- 172.21.15.7:0/2482588752 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc00092d0 con 0x55d07e74a5d0 2024-01-11T18:06:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.521+0000 7febb4ff9700 1 -- 172.21.15.7:0/2482588752 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00052a0 con 0x55d07e74a5d0 2024-01-11T18:06:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.522+0000 7febe0bb3b80 1 -- 172.21.15.7:0/2482588752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 msgr2=0x55d07e750e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.522+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e750e60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7febc0009820 tx=0x7febc0008db0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.523+0000 7febe0bb3b80 1 -- 172.21.15.7:0/2482588752 shutdown_connections 2024-01-11T18:06:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.523+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e750e60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.523+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e749340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.523+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/2482588752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e74a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.523+0000 7febe0bb3b80 1 -- 172.21.15.7:0/2482588752 >> 172.21.15.7:0/2482588752 conn(0x55d07e6be740 msgr2=0x55d07e743950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.523+0000 7febe0bb3b80 1 -- 172.21.15.7:0/2482588752 shutdown_connections 2024-01-11T18:06:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.525+0000 7febe0bb3b80 1 -- 172.21.15.7:0/2482588752 wait complete. 2024-01-11T18:06:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.526+0000 7febe0bb3b80 1 -- start start 2024-01-11T18:06:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.527+0000 7febe0bb3b80 1 Processor -- start 2024-01-11T18:06:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.532+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 0x55d07e752920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.533+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e752e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.533+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e74a5d0 0x55d07e759e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.533+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e69ccd0 con 0x55d07e74a5d0 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.533+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e60b340 con 0x55d07e749c10 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.533+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e60afe0 con 0x55d07e748f20 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.533+0000 7febc659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 0x55d07e752920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.534+0000 7febc659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 0x55d07e752920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44820/0 (socket says 172.21.15.7:44820) 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.534+0000 7febc659c700 1 -- 172.21.15.7:0/642298420 learned_addr learned my addr 172.21.15.7:0/642298420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.534+0000 7febe0bb3b80 10 client.0 _ll_register_callbacks cb 0x55d07e63d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.534+0000 7febc5d9b700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e752e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.534+0000 7febc6d9d700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e74a5d0 0x55d07e759e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.535+0000 7febc6d9d700 1 -- 172.21.15.7:0/642298420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 msgr2=0x55d07e752920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.535+0000 7febc6d9d700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 0x55d07e752920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febc6d9d700 1 -- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 msgr2=0x55d07e752e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febc6d9d700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e752e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febc6d9d700 1 -- 172.21.15.7:0/642298420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc8100300 con 0x55d07e74a5d0 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d07e73bd60 con 0x55d07e74a5d0 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febc5d9b700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e752e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:06:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febc659c700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 0x55d07e752920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:06:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febc6d9d700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e74a5d0 0x55d07e759e30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febc000d010 tx=0x7febc001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3cd166c6bdf71f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.536+0000 7febadffb700 10 client.16053 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:06:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.537+0000 7febadffb700 1 -- 172.21.15.7:0/642298420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0006000 con 0x55d07e74a5d0 2024-01-11T18:06:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.537+0000 7febadffb700 1 -- 172.21.15.7:0/642298420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7febc0009440 con 0x55d07e74a5d0 2024-01-11T18:06:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.538+0000 7febadffb700 1 -- 172.21.15.7:0/642298420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc00079f0 con 0x55d07e74a5d0 2024-01-11T18:06:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.538+0000 7febe0bb3b80 10 client.16053 fetch_fsmap learned FSMap version 41 2024-01-11T18:06:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.538+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=41}) v3 -- 0x55d07e4bdb00 con 0x55d07e74a5d0 2024-01-11T18:06:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febadffb700 1 -- 172.21.15.7:0/642298420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7febc000b0d0 con 0x55d07e74a5d0 2024-01-11T18:06:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febe0bb3b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T18:06:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febe0bb3b80 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febe0bb3b80 20 client.16053 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e75e0d0 0x55d07e7604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febe0bb3b80 4 client.16053 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:06:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d07e695b60 con 0x55d07e75e0d0 2024-01-11T18:06:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.539+0000 7febc659c700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e75e0d0 0x55d07e7604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.541+0000 7febc659c700 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e75e0d0 0x55d07e7604c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64965b02040a0400 server_cookie=b0f67dcc49a2c553 in_seq=0 out_seq=0 2024-01-11T18:06:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.541+0000 7febadffb700 10 client.16053 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.542+0000 7febadffb700 1 -- 172.21.15.7:0/642298420 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d07e695b60 con 0x55d07e75e0d0 2024-01-11T18:06:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.542+0000 7febadffb700 10 client.16053 handle_command_reply: tid=0 2024-01-11T18:06:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.542+0000 7febe0bb3b80 1 client.16053 shutdown 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e74a5d0 msgr2=0x55d07e759e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e74a5d0 0x55d07e759e30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febc000d010 tx=0x7febc001c670 comp rx=0 tx=0).stop 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 shutdown_connections 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e75e0d0 0x55d07e7604c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e748f20 0x55d07e752920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e74a5d0 0x55d07e759e30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/642298420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e749c10 0x55d07e752e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febadffb700 0 client.16053 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.543+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 >> 172.21.15.7:0/642298420 conn(0x55d07e6be740 msgr2=0x55d07e742f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.544+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 shutdown_connections 2024-01-11T18:06:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.544+0000 7febe0bb3b80 1 -- 172.21.15.7:0/642298420 wait complete. 2024-01-11T18:06:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.544+0000 7febe0bb3b80 20 client.16053 trim_cache size 0 max 16384 2024-01-11T18:06:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.585+0000 7febe0bb3b80 1 Processor -- start 2024-01-11T18:06:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.585+0000 7febe0bb3b80 1 -- start start 2024-01-11T18:06:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e748f20 0x55d07e7604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e749c10 0x55d07e759df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e749ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e513640 con 0x55d07e749c10 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e740560 con 0x55d07e748f20 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e745020 con 0x55d07e74a5d0 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febc6d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e749ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febc6d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e749ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44828/0 (socket says 172.21.15.7:44828) 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.586+0000 7febc6d9d700 1 -- 172.21.15.7:0/3890025257 learned_addr learned my addr 172.21.15.7:0/3890025257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.587+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e749c10 0x55d07e759df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.587+0000 7febc5d9b700 1 -- 172.21.15.7:0/3890025257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 msgr2=0x55d07e749ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.587+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e749ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.587+0000 7febc5d9b700 1 -- 172.21.15.7:0/3890025257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e748f20 msgr2=0x55d07e7604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.587+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e748f20 0x55d07e7604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.587+0000 7febc5d9b700 1 -- 172.21.15.7:0/3890025257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d07e65e1d0 con 0x55d07e749c10 2024-01-11T18:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.588+0000 7febc5d9b700 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e749c10 0x55d07e759df0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7febbc0cc850 tx=0x7febbc0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4cbd2f6901d223 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.589+0000 7febaffff700 1 -- 172.21.15.7:0/3890025257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0b3070 con 0x55d07e749c10 2024-01-11T18:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.589+0000 7febaffff700 1 -- 172.21.15.7:0/3890025257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febbc09fc00 con 0x55d07e749c10 2024-01-11T18:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.589+0000 7febaffff700 1 -- 172.21.15.7:0/3890025257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc084ea0 con 0x55d07e749c10 2024-01-11T18:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 -- 172.21.15.7:0/3890025257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e749c10 msgr2=0x55d07e759df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e749c10 0x55d07e759df0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7febbc0cc850 tx=0x7febbc0bb040 comp rx=0 tx=0).stop 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 -- 172.21.15.7:0/3890025257 shutdown_connections 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e74a5d0 0x55d07e749ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e748f20 0x55d07e7604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/3890025257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e749c10 0x55d07e759df0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 -- 172.21.15.7:0/3890025257 >> 172.21.15.7:0/3890025257 conn(0x55d07e6be740 msgr2=0x55d07e73f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.590+0000 7febe0bb3b80 1 -- 172.21.15.7:0/3890025257 shutdown_connections 2024-01-11T18:06:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.591+0000 7febe0bb3b80 1 -- 172.21.15.7:0/3890025257 wait complete. 2024-01-11T18:06:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.591+0000 7febe0bb3b80 1 -- start start 2024-01-11T18:06:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.592+0000 7febe0bb3b80 1 Processor -- start 2024-01-11T18:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e757690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e749c10 0x55d07e757bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e74a5d0 0x55d07e758be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e759120 con 0x55d07e748f20 2024-01-11T18:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e759290 con 0x55d07e74a5d0 2024-01-11T18:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e759450 con 0x55d07e749c10 2024-01-11T18:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.593+0000 7febe0bb3b80 10 client.0 _ll_register_callbacks cb 0x55d07e63d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.594+0000 7febc5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e749c10 0x55d07e757bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.594+0000 7febc659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e757690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.595+0000 7febc659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e757690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57764/0 (socket says 172.21.15.7:57764) 2024-01-11T18:06:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.595+0000 7febc659c700 1 -- 172.21.15.7:0/704851600 learned_addr learned my addr 172.21.15.7:0/704851600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febc659c700 1 -- 172.21.15.7:0/704851600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e749c10 msgr2=0x55d07e757bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febc659c700 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e749c10 0x55d07e757bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febc659c700 1 -- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e74a5d0 msgr2=0x55d07e758be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febc659c700 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e74a5d0 0x55d07e758be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febc659c700 1 -- 172.21.15.7:0/704851600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81a1d20 con 0x55d07e748f20 2024-01-11T18:06:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febc659c700 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e757690 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d07e760e10 tx=0x7febb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951a376203ed8cd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.596+0000 7febadffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:06:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.597+0000 7febadffb700 1 -- 172.21.15.7:0/704851600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb80244f0 con 0x55d07e748f20 2024-01-11T18:06:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.597+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc00092d0 con 0x55d07e748f20 2024-01-11T18:06:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.597+0000 7febadffb700 1 -- 172.21.15.7:0/704851600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7febb8006e80 con 0x55d07e748f20 2024-01-11T18:06:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.598+0000 7febadffb700 1 -- 172.21.15.7:0/704851600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb8024650 con 0x55d07e748f20 2024-01-11T18:06:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.598+0000 7febe0bb3b80 10 client.16059 fetch_fsmap learned FSMap version 41 2024-01-11T18:06:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.598+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d07e515c70 con 0x55d07e748f20 2024-01-11T18:06:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.598+0000 7febadffb700 1 -- 172.21.15.7:0/704851600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc00092d0 con 0x55d07e748f20 2024-01-11T18:06:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febadffb700 1 -- 172.21.15.7:0/704851600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7febb8021400 con 0x55d07e748f20 2024-01-11T18:06:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febe0bb3b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T18:06:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febe0bb3b80 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febe0bb3b80 20 client.16059 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e7b51e0 0x55d07e7b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febe0bb3b80 4 client.16059 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:06:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.599+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d07e695880 con 0x55d07e7b51e0 2024-01-11T18:06:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.610+0000 7febc5d9b700 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e7b51e0 0x55d07e7b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.611+0000 7febc5d9b700 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e7b51e0 0x55d07e7b7600 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f126e2569ce218 server_cookie=664c1e45858001c4 in_seq=0 out_seq=0 2024-01-11T18:06:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.611+0000 7febadffb700 10 client.16059 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.725+0000 7febadffb700 1 -- 172.21.15.7:0/704851600 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d07e695880 con 0x55d07e7b51e0 2024-01-11T18:06:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.725+0000 7febadffb700 10 client.16059 handle_command_reply: tid=0 2024-01-11T18:06:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.725+0000 7febe0bb3b80 1 client.16059 shutdown 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 msgr2=0x55d07e757690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e757690 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d07e760e10 tx=0x7febb8004750 comp rx=0 tx=0).stop 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 shutdown_connections 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d07e7b51e0 0x55d07e7b7600 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d07e74a5d0 0x55d07e758be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d07e748f20 0x55d07e757690 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 --2- 172.21.15.7:0/704851600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d07e749c10 0x55d07e757bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 >> 172.21.15.7:0/704851600 conn(0x55d07e6be740 msgr2=0x55d07e74d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febadffb700 0 client.16059 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.726+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 shutdown_connections 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.727+0000 7febe0bb3b80 1 -- 172.21.15.7:0/704851600 wait complete. 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.727+0000 7febe0bb3b80 20 client.16059 trim_cache size 0 max 16384 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:06:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:06:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febb00a1fd0 msgr2=0x7febb00a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febb00a1fd0 0x7febb00a4490 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7febb8009730 tx=0x7febb8006cb0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 msgr2=0x7febc8199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8199b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c9b0 tx=0x7febbc00cd70 comp rx=0 tx=0).stop 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 shutdown_connections 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febb00a1fd0 0x7febb00a4490 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc8102020 0x7febc8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8107280 0x7febc81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 --2- 172.21.15.7:0/3419264265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8104950 0x7febc8199b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.729+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 >> 172.21.15.7:0/3419264265 conn(0x7febc80fba30 msgr2=0x7febc8102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.730+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 shutdown_connections 2024-01-11T18:06:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:21.730+0000 7febccdff700 1 -- 172.21.15.7:0/3419264265 wait complete. 2024-01-11T18:06:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 475.0 s, 689.4 tps, lat 148.230 ms stddev 197.940 2024-01-11T18:06:24.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:23 smithi016 ceph-mon[43195]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.9 MiB/s wr, 513 op/s 2024-01-11T18:06:24.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:23 smithi038 ceph-mon[43582]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.9 MiB/s wr, 513 op/s 2024-01-11T18:06:24.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:23 smithi007 ceph-mon[33361]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.9 MiB/s wr, 513 op/s 2024-01-11T18:06:25.989 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:25 smithi016 ceph-mon[43195]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 510 op/s 2024-01-11T18:06:26.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:25 smithi038 ceph-mon[43582]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 510 op/s 2024-01-11T18:06:26.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:25 smithi007 ceph-mon[33361]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 510 op/s 2024-01-11T18:06:28.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:27 smithi016 ceph-mon[43195]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.2 MiB/s wr, 540 op/s 2024-01-11T18:06:28.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:27 smithi038 ceph-mon[43582]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.2 MiB/s wr, 540 op/s 2024-01-11T18:06:28.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:27 smithi007 ceph-mon[33361]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.2 MiB/s wr, 540 op/s 2024-01-11T18:06:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 480.0 s, 477.4 tps, lat 200.895 ms stddev 240.262 2024-01-11T18:06:29.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:29 smithi038 ceph-mon[43582]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 538 op/s 2024-01-11T18:06:29.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:29 smithi007 ceph-mon[33361]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 538 op/s 2024-01-11T18:06:29.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:29 smithi016 ceph-mon[43195]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 538 op/s 2024-01-11T18:06:32.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:31 smithi016 ceph-mon[43195]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.6 MiB/s wr, 588 op/s 2024-01-11T18:06:32.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:31 smithi038 ceph-mon[43582]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.6 MiB/s wr, 588 op/s 2024-01-11T18:06:32.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:31 smithi007 ceph-mon[33361]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.6 MiB/s wr, 588 op/s 2024-01-11T18:06:34.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 485.0 s, 521.6 tps, lat 189.257 ms stddev 187.241 2024-01-11T18:06:34.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:33 smithi016 ceph-mon[43195]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 592 op/s 2024-01-11T18:06:34.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:33 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:06:34.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:33 smithi038 ceph-mon[43582]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 592 op/s 2024-01-11T18:06:34.276 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:33 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:06:34.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:33 smithi007 ceph-mon[33361]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 592 op/s 2024-01-11T18:06:34.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:33 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:06:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.002+0000 7f261cfff700 1 -- 172.21.15.7:0/2850822967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618101690 msgr2=0x7f2618101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.002+0000 7f261cfff700 1 --2- 172.21.15.7:0/2850822967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618101690 0x7f2618101af0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f260c0099c0 tx=0x7f260c009cd0 comp rx=0 tx=0).stop 2024-01-11T18:06:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.004+0000 7f261cfff700 1 -- 172.21.15.7:0/2850822967 shutdown_connections 2024-01-11T18:06:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.004+0000 7f261cfff700 1 --2- 172.21.15.7:0/2850822967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26181004a0 0x7f26181008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.004+0000 7f261cfff700 1 --2- 172.21.15.7:0/2850822967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618102030 0x7f26181088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.005+0000 7f261cfff700 1 --2- 172.21.15.7:0/2850822967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618101690 0x7f2618101af0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.005+0000 7f261cfff700 1 -- 172.21.15.7:0/2850822967 >> 172.21.15.7:0/2850822967 conn(0x7f26180fba30 msgr2=0x7f26180fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.005+0000 7f261cfff700 1 -- 172.21.15.7:0/2850822967 shutdown_connections 2024-01-11T18:06:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.005+0000 7f261cfff700 1 -- 172.21.15.7:0/2850822967 wait complete. 2024-01-11T18:06:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.007+0000 7f261cfff700 1 Processor -- start 2024-01-11T18:06:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.007+0000 7f261cfff700 1 -- start start 2024-01-11T18:06:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.009+0000 7f261cfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26181004a0 0x7f261806b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.009+0000 7f261cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 0x7f261806b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f261cfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618102030 0x7f26180726e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f261cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618072d20 con 0x7f2618101690 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f261cfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618072e90 con 0x7f2618102030 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f261cfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618073050 con 0x7f26181004a0 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 0x7f261806b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 0x7f261806b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40712/0 (socket says 172.21.15.7:40712) 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.010+0000 7f2615d9b700 1 -- 172.21.15.7:0/4224967169 learned_addr learned my addr 172.21.15.7:0/4224967169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.011+0000 7f2615d9b700 1 -- 172.21.15.7:0/4224967169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26181004a0 msgr2=0x7f261806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.011+0000 7f2615d9b700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26181004a0 0x7f261806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.011+0000 7f2615d9b700 1 -- 172.21.15.7:0/4224967169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618102030 msgr2=0x7f26180726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.011+0000 7f2615d9b700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618102030 0x7f26180726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.011+0000 7f2615d9b700 1 -- 172.21.15.7:0/4224967169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0095f0 con 0x7f2618101690 2024-01-11T18:06:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.012+0000 7f2615d9b700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 0x7f261806b790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f260c003730 tx=0x7f260c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.013+0000 7f26077fe700 1 -- 172.21.15.7:0/4224967169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c01d070 con 0x7f2618101690 2024-01-11T18:06:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.013+0000 7f26077fe700 1 -- 172.21.15.7:0/4224967169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f260c00bda0 con 0x7f2618101690 2024-01-11T18:06:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.013+0000 7f26077fe700 1 -- 172.21.15.7:0/4224967169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c017650 con 0x7f2618101690 2024-01-11T18:06:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.014+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26180732d0 con 0x7f2618101690 2024-01-11T18:06:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.014+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26181ae970 con 0x7f2618101690 2024-01-11T18:06:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.022+0000 7f26077fe700 1 -- 172.21.15.7:0/4224967169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f260c022470 con 0x7f2618101690 2024-01-11T18:06:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.022+0000 7f2630cbfb80 1 Processor -- start 2024-01-11T18:06:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.022+0000 7f2630cbfb80 1 -- start start 2024-01-11T18:06:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.023+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b12581340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.024+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b12581340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.024+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b12581340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40728/0 (socket says 172.21.15.7:40728) 2024-01-11T18:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.024+0000 7f2616d9d700 1 -- 172.21.15.7:0/4192153558 learned_addr learned my addr 172.21.15.7:0/4192153558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.024+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12582090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.024+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b125825d0 0x563b12588e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.025+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12478030 con 0x563b12580f20 2024-01-11T18:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.025+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b124dbac0 con 0x563b12581c10 2024-01-11T18:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.025+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b124d93e0 con 0x563b125825d0 2024-01-11T18:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.025+0000 7f261659c700 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12582090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f26077fe700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f26000a1f50 0x7f26000a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f2615d9b700 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b125825d0 0x563b12588e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f261659c700 1 -- 172.21.15.7:0/4192153558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b125825d0 msgr2=0x563b12588e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f261659c700 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b125825d0 0x563b12588e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f261659c700 1 -- 172.21.15.7:0/4192153558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 msgr2=0x563b12581340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f261659c700 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b12581340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f261659c700 1 -- 172.21.15.7:0/4192153558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b12349350 con 0x563b12581c10 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.026+0000 7f261659c700 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12582090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2608009bc0 tx=0x7f2608009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93dc78340f3fbe3e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.027+0000 7f26077fe700 1 -- 172.21.15.7:0/4224967169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f260c0d2170 con 0x7f2618101690 2024-01-11T18:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.027+0000 7f2616d9d700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f26000a1f50 0x7f26000a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.028+0000 7f2604ff9700 1 -- 172.21.15.7:0/4192153558 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800eb60 con 0x563b12581c10 2024-01-11T18:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.028+0000 7f2604ff9700 1 -- 172.21.15.7:0/4192153558 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2608004cf0 con 0x563b12581c10 2024-01-11T18:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.029+0000 7f2604ff9700 1 -- 172.21.15.7:0/4192153558 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608010430 con 0x563b12581c10 2024-01-11T18:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.029+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 msgr2=0x563b12582090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.029+0000 7f2616d9d700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f26000a1f50 0x7f26000a4410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x563b12580cc0 tx=0x7f261000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.030+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12582090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2608009bc0 tx=0x7f2608009ed0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.030+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 shutdown_connections 2024-01-11T18:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.030+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b125825d0 0x563b12588e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.030+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b12581340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.030+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/4192153558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12582090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.030+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 >> 172.21.15.7:0/4192153558 conn(0x563b124f6740 msgr2=0x563b1257b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.031+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 shutdown_connections 2024-01-11T18:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.031+0000 7f2630cbfb80 1 -- 172.21.15.7:0/4192153558 wait complete. 2024-01-11T18:06:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.032+0000 7f2630cbfb80 1 -- start start 2024-01-11T18:06:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.032+0000 7f2630cbfb80 1 Processor -- start 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b1258aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b125825d0 0x563b12591fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b124d4cd0 con 0x563b12580f20 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12443340 con 0x563b125825d0 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12442fe0 con 0x563b12581c10 2024-01-11T18:06:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.033+0000 7f2630cbfb80 10 client.0 _ll_register_callbacks cb 0x563b124756c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.035+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.035+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33490/0 (socket says 172.21.15.7:33490) 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.035+0000 7f2615d9b700 1 -- 172.21.15.7:0/3327665686 learned_addr learned my addr 172.21.15.7:0/3327665686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.035+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b125825d0 0x563b12591fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.036+0000 7f2616d9d700 1 -- 172.21.15.7:0/3327665686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 msgr2=0x563b1258af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.036+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.036+0000 7f2616d9d700 1 -- 172.21.15.7:0/3327665686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 msgr2=0x563b1258aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.036+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b1258aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.036+0000 7f2616d9d700 1 -- 172.21.15.7:0/3327665686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26080098a0 con 0x563b125825d0 2024-01-11T18:06:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.037+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b125825d0 0x563b12591fa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2610014dc0 tx=0x7f26100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42dfe1abe565ce1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.037+0000 7f25fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:06:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.037+0000 7f25fdffb700 1 -- 172.21.15.7:0/3327665686 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100139e0 con 0x563b125825d0 2024-01-11T18:06:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.037+0000 7f25fdffb700 1 -- 172.21.15.7:0/3327665686 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2610013b40 con 0x563b125825d0 2024-01-11T18:06:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.037+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b12573d60 con 0x563b125825d0 2024-01-11T18:06:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.038+0000 7f25fdffb700 1 -- 172.21.15.7:0/3327665686 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2610013d00 con 0x563b125825d0 2024-01-11T18:06:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.038+0000 7f2630cbfb80 10 client.26225 fetch_fsmap learned FSMap version 41 2024-01-11T18:06:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.038+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563b122f5b00 con 0x563b125825d0 2024-01-11T18:06:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.039+0000 7f25fdffb700 1 -- 172.21.15.7:0/3327665686 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b12573d60 con 0x563b125825d0 2024-01-11T18:06:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.039+0000 7f25fdffb700 1 -- 172.21.15.7:0/3327665686 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f261001f440 con 0x563b125825d0 2024-01-11T18:06:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.040+0000 7f2630cbfb80 10 client.26225 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T18:06:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.040+0000 7f2630cbfb80 10 client.26225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.040+0000 7f2630cbfb80 20 client.26225 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.040+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b12596990 0x563b12598db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.041+0000 7f2630cbfb80 4 client.26225 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:06:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.041+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b124cdb60 con 0x563b12596990 2024-01-11T18:06:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.041+0000 7f261659c700 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b12596990 0x563b12598db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.042+0000 7f261659c700 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b12596990 0x563b12598db0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60d380dc384da83 server_cookie=cda1ecaeccef5b01 in_seq=0 out_seq=0 2024-01-11T18:06:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.043+0000 7f25fdffb700 10 client.26225 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.043+0000 7f25fdffb700 1 -- 172.21.15.7:0/3327665686 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b124cdb60 con 0x563b12596990 2024-01-11T18:06:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.043+0000 7f25fdffb700 10 client.26225 handle_command_reply: tid=0 2024-01-11T18:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.043+0000 7f2630cbfb80 1 client.26225 shutdown 2024-01-11T18:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.047+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b125825d0 msgr2=0x563b12591fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.048+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b125825d0 0x563b12591fa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2610014dc0 tx=0x7f26100091a0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 shutdown_connections 2024-01-11T18:06:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b12596990 0x563b12598db0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b125825d0 0x563b12591fa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b12580f20 0x563b1258aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3327665686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 >> 172.21.15.7:0/3327665686 conn(0x563b124f6740 msgr2=0x563b1257af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.051+0000 7f25fdffb700 0 client.26225 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.052+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 shutdown_connections 2024-01-11T18:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.052+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3327665686 wait complete. 2024-01-11T18:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.052+0000 7f2630cbfb80 20 client.26225 trim_cache size 0 max 16384 2024-01-11T18:06:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.081+0000 7f2630cbfb80 1 Processor -- start 2024-01-11T18:06:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.082+0000 7f2630cbfb80 1 -- start start 2024-01-11T18:06:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.083+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 0x563b12584e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.083+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12591f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.083+0000 7f261659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 0x563b12584e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.083+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12589fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.083+0000 7f261659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 0x563b12584e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33504/0 (socket says 172.21.15.7:33504) 2024-01-11T18:06:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.084+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1234b640 con 0x563b125825d0 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.084+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12581ad0 con 0x563b12581c10 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.084+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1257ccd0 con 0x563b12580f20 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.084+0000 7f261659c700 1 -- 172.21.15.7:0/3731684739 learned_addr learned my addr 172.21.15.7:0/3731684739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.084+0000 7f2615d9b700 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12591f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.084+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12589fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f261659c700 1 -- 172.21.15.7:0/3731684739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 msgr2=0x563b12591f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f261659c700 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12591f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f261659c700 1 -- 172.21.15.7:0/3731684739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 msgr2=0x563b12589fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f261659c700 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12589fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f261659c700 1 -- 172.21.15.7:0/3731684739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b124961d0 con 0x563b12580f20 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12589fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.085+0000 7f261659c700 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 0x563b12584e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2608009bc0 tx=0x7f2608004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0d3c7f7fcb8c84 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.086+0000 7f25fffff700 1 -- 172.21.15.7:0/3731684739 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608004a30 con 0x563b12580f20 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.086+0000 7f25fffff700 1 -- 172.21.15.7:0/3731684739 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f260800ee20 con 0x563b12580f20 2024-01-11T18:06:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.086+0000 7f25fffff700 1 -- 172.21.15.7:0/3731684739 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608027460 con 0x563b12580f20 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3731684739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 msgr2=0x563b12584e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 0x563b12584e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2608009bc0 tx=0x7f2608004750 comp rx=0 tx=0).stop 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3731684739 shutdown_connections 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12580f20 0x563b12584e40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12589fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3731684739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12581c10 0x563b12591f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3731684739 >> 172.21.15.7:0/3731684739 conn(0x563b124f6740 msgr2=0x563b125779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3731684739 shutdown_connections 2024-01-11T18:06:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.087+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3731684739 wait complete. 2024-01-11T18:06:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.088+0000 7f2630cbfb80 1 -- start start 2024-01-11T18:06:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.088+0000 7f2630cbfb80 1 Processor -- start 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12580f20 0x563b1258f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12590680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12590bc0 con 0x563b125825d0 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12590d30 con 0x563b12580f20 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b12590ef0 con 0x563b12581c10 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.089+0000 7f2630cbfb80 10 client.0 _ll_register_callbacks cb 0x563b12592b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.090+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12590680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.090+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12590680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40760/0 (socket says 172.21.15.7:40760) 2024-01-11T18:06:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.090+0000 7f2616d9d700 1 -- 172.21.15.7:0/3167795240 learned_addr learned my addr 172.21.15.7:0/3167795240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.090+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.090+0000 7f261659c700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12580f20 0x563b1258f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.091+0000 7f2615d9b700 1 -- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12580f20 msgr2=0x563b1258f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.091+0000 7f2615d9b700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12580f20 0x563b1258f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.091+0000 7f2615d9b700 1 -- 172.21.15.7:0/3167795240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 msgr2=0x563b12590680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.091+0000 7f2615d9b700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12590680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.091+0000 7f2615d9b700 1 -- 172.21.15.7:0/3167795240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b122f5b00 con 0x563b12581c10 2024-01-11T18:06:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.091+0000 7f2616d9d700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12590680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.092+0000 7f2615d9b700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258f670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f260c098150 tx=0x7f260c005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6abde020062ad1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.092+0000 7f25fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.092+0000 7f25fdffb700 1 -- 172.21.15.7:0/3167795240 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c09b220 con 0x563b12581c10 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.092+0000 7f25fdffb700 1 -- 172.21.15.7:0/3167795240 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f260c09bde0 con 0x563b12581c10 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.093+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2608004cf0 con 0x563b12581c10 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.093+0000 7f25fdffb700 1 -- 172.21.15.7:0/3167795240 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f260c09b040 con 0x563b12581c10 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.093+0000 7f2630cbfb80 10 client.26257 fetch_fsmap learned FSMap version 41 2024-01-11T18:06:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.093+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f2608027e70 con 0x563b12581c10 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.093+0000 7f25fdffb700 1 -- 172.21.15.7:0/3167795240 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2608004cf0 con 0x563b12581c10 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f25fdffb700 1 -- 172.21.15.7:0/3167795240 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f260c09acf0 con 0x563b12581c10 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f2630cbfb80 10 client.26257 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f2630cbfb80 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f2630cbfb80 20 client.26257 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b125fe030 0x563b12600450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f2630cbfb80 4 client.26257 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:06:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b124cd880 con 0x563b125fe030 2024-01-11T18:06:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.094+0000 7f261659c700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b125fe030 0x563b12600450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.095+0000 7f261659c700 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b125fe030 0x563b12600450 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec9621f336f4c7f server_cookie=7870c3f004de3729 in_seq=0 out_seq=0 2024-01-11T18:06:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.095+0000 7f25fdffb700 10 client.26257 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.114+0000 7f25fdffb700 1 -- 172.21.15.7:0/3167795240 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b124cd880 con 0x563b125fe030 2024-01-11T18:06:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.114+0000 7f25fdffb700 10 client.26257 handle_command_reply: tid=0 2024-01-11T18:06:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.114+0000 7f2630cbfb80 1 client.26257 shutdown 2024-01-11T18:06:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 msgr2=0x563b1258f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258f670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f260c098150 tx=0x7f260c005c60 comp rx=0 tx=0).stop 2024-01-11T18:06:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 shutdown_connections 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x563b125fe030 0x563b12600450 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b125825d0 0x563b12590680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b12580f20 0x563b1258f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f25fdffb700 0 client.26257 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 --2- 172.21.15.7:0/3167795240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563b12581c10 0x563b1258f670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 >> 172.21.15.7:0/3167795240 conn(0x563b124f6740 msgr2=0x563b12589ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 shutdown_connections 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.115+0000 7f2630cbfb80 1 -- 172.21.15.7:0/3167795240 wait complete. 2024-01-11T18:06:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f2630cbfb80 20 client.26257 trim_cache size 0 max 16384 2024-01-11T18:06:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:06:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:06:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:06:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f26000a1f50 msgr2=0x7f26000a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f26000a1f50 0x7f26000a4410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x563b12580cc0 tx=0x7f261000b040 comp rx=0 tx=0).stop 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 msgr2=0x7f261806b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 0x7f261806b790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f260c003730 tx=0x7f260c00fab0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 shutdown_connections 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f26000a1f50 0x7f26000a4410 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26181004a0 0x7f261806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2618102030 0x7f26180726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 --2- 172.21.15.7:0/4224967169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2618101690 0x7f261806b790 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.116+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 >> 172.21.15.7:0/4224967169 conn(0x7f26180fba30 msgr2=0x7f261810d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.117+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 shutdown_connections 2024-01-11T18:06:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:35.118+0000 7f261cfff700 1 -- 172.21.15.7:0/4224967169 wait complete. 2024-01-11T18:06:36.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:35 smithi038 ceph-mon[43582]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-01-11T18:06:36.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:35 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:06:36.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:35 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:06:36.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:35 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:06:36.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:35 smithi007 ceph-mon[33361]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-01-11T18:06:36.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:06:36.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:06:36.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:06:36.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:35 smithi016 ceph-mon[43195]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-01-11T18:06:36.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:35 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:06:36.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:35 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:06:36.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:35 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:06:37.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:37 smithi038 ceph-mon[43582]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 7.8 MiB/s wr, 596 op/s 2024-01-11T18:06:37.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:37 smithi007 ceph-mon[33361]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 7.8 MiB/s wr, 596 op/s 2024-01-11T18:06:37.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:37 smithi016 ceph-mon[43195]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 7.8 MiB/s wr, 596 op/s 2024-01-11T18:06:39.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 490.0 s, 462.0 tps, lat 219.069 ms stddev 241.542 2024-01-11T18:06:40.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:39 smithi016 ceph-mon[43195]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 6.5 MiB/s wr, 561 op/s 2024-01-11T18:06:40.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:39 smithi038 ceph-mon[43582]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 6.5 MiB/s wr, 561 op/s 2024-01-11T18:06:40.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:39 smithi007 ceph-mon[33361]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 6.5 MiB/s wr, 561 op/s 2024-01-11T18:06:41.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:41 smithi038 ceph-mon[43582]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 7.1 MiB/s wr, 559 op/s 2024-01-11T18:06:41.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:41 smithi007 ceph-mon[33361]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 7.1 MiB/s wr, 559 op/s 2024-01-11T18:06:41.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:41 smithi016 ceph-mon[43195]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 7.1 MiB/s wr, 559 op/s 2024-01-11T18:06:44.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 495.0 s, 569.8 tps, lat 181.541 ms stddev 208.179 2024-01-11T18:06:44.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:43 smithi016 ceph-mon[43195]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 6.3 MiB/s wr, 517 op/s 2024-01-11T18:06:44.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:43 smithi038 ceph-mon[43582]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 6.3 MiB/s wr, 517 op/s 2024-01-11T18:06:44.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:43 smithi007 ceph-mon[33361]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 6.3 MiB/s wr, 517 op/s 2024-01-11T18:06:45.988 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:45 smithi016 ceph-mon[43195]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 6.3 MiB/s wr, 492 op/s 2024-01-11T18:06:46.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:45 smithi038 ceph-mon[43582]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 6.3 MiB/s wr, 492 op/s 2024-01-11T18:06:46.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:45 smithi007 ceph-mon[33361]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 6.3 MiB/s wr, 492 op/s 2024-01-11T18:06:48.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:47 smithi016 ceph-mon[43195]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 6.5 MiB/s wr, 530 op/s 2024-01-11T18:06:48.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:47 smithi038 ceph-mon[43582]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 6.5 MiB/s wr, 530 op/s 2024-01-11T18:06:48.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:47 smithi007 ceph-mon[33361]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 6.5 MiB/s wr, 530 op/s 2024-01-11T18:06:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 -- 172.21.15.7:0/3181097587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06680755f0 msgr2=0x7f0668073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 --2- 172.21.15.7:0/3181097587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06680755f0 0x7f0668073a40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0658009990 tx=0x7f0658009ca0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 -- 172.21.15.7:0/3181097587 shutdown_connections 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 --2- 172.21.15.7:0/3181097587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06680755f0 0x7f0668073a40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 --2- 172.21.15.7:0/3181097587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06681094c0 0x7f066810b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 --2- 172.21.15.7:0/3181097587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668074010 0x7f0668074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 -- 172.21.15.7:0/3181097587 >> 172.21.15.7:0/3181097587 conn(0x7f06680fcb60 msgr2=0x7f06680fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.487+0000 7f066cfff700 1 -- 172.21.15.7:0/3181097587 shutdown_connections 2024-01-11T18:06:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.488+0000 7f066cfff700 1 -- 172.21.15.7:0/3181097587 wait complete. 2024-01-11T18:06:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.489+0000 7f066cfff700 1 Processor -- start 2024-01-11T18:06:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.489+0000 7f066cfff700 1 -- start start 2024-01-11T18:06:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066cfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 0x7f0668199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06680755f0 0x7f0668199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066cfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06681094c0 0x7f06681a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681a1210 con 0x7f06680755f0 2024-01-11T18:06:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066cfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681a1350 con 0x7f0668074010 2024-01-11T18:06:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066cfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681a14e0 con 0x7f06681094c0 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 0x7f0668199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 0x7f0668199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59788/0 (socket says 172.21.15.7:59788) 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f066659c700 1 -- 172.21.15.7:0/4063386486 learned_addr learned my addr 172.21.15.7:0/4063386486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f0666d9d700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06681094c0 0x7f06681a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.490+0000 7f0665d9b700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06680755f0 0x7f0668199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f066659c700 1 -- 172.21.15.7:0/4063386486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06681094c0 msgr2=0x7f06681a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f066659c700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06681094c0 0x7f06681a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f066659c700 1 -- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06680755f0 msgr2=0x7f0668199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f066659c700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06680755f0 0x7f0668199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f066659c700 1 -- 172.21.15.7:0/4063386486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580095f0 con 0x7f0668074010 2024-01-11T18:06:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f0665d9b700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06680755f0 0x7f0668199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:06:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.491+0000 7f066659c700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 0x7f0668199690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0658006010 tx=0x7f065800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.492+0000 7f06577fe700 1 -- 172.21.15.7:0/4063386486 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065801d070 con 0x7f0668074010 2024-01-11T18:06:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.492+0000 7f06577fe700 1 -- 172.21.15.7:0/4063386486 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0658003d50 con 0x7f0668074010 2024-01-11T18:06:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.492+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06681a1760 con 0x7f0668074010 2024-01-11T18:06:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.492+0000 7f06577fe700 1 -- 172.21.15.7:0/4063386486 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065800f460 con 0x7f0668074010 2024-01-11T18:06:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.492+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06681a1be0 con 0x7f0668074010 2024-01-11T18:06:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.497+0000 7f0680c90b80 1 Processor -- start 2024-01-11T18:06:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.497+0000 7f06577fe700 1 -- 172.21.15.7:0/4063386486 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f065800f5c0 con 0x7f0668074010 2024-01-11T18:06:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 -- start start 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65690 0x55ad31e65ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e66f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31d54030 con 0x55ad31e65690 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31db7ac0 con 0x55ad31e66ad0 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.499+0000 7f0680c90b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31db53e0 con 0x55ad31e65ff0 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.500+0000 7f066659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.500+0000 7f066659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45918/0 (socket says 172.21.15.7:45918) 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.500+0000 7f066659c700 1 -- 172.21.15.7:0/3289722021 learned_addr learned my addr 172.21.15.7:0/3289722021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.500+0000 7f0666d9d700 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65690 0x55ad31e65ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.500+0000 7f0665d9b700 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e66f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f06577fe700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06500a1fd0 0x7f06500a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f0666d9d700 1 -- 172.21.15.7:0/3289722021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 msgr2=0x55ad31e664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f0666d9d700 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f0666d9d700 1 -- 172.21.15.7:0/3289722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 msgr2=0x55ad31e66f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f0666d9d700 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e66f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f0666d9d700 1 -- 172.21.15.7:0/3289722021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad31c25350 con 0x55ad31e65690 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f06577fe700 1 -- 172.21.15.7:0/4063386486 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f06580cf270 con 0x7f0668074010 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.501+0000 7f066659c700 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.502+0000 7f0666d9d700 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65690 0x55ad31e65ab0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0660011580 tx=0x7f0660009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe3c895879454f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.502+0000 7f0666d9d700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06500a1fd0 0x7f06500a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.502+0000 7f0654ff9700 1 -- 172.21.15.7:0/3289722021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000e040 con 0x55ad31e65690 2024-01-11T18:06:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.502+0000 7f0654ff9700 1 -- 172.21.15.7:0/3289722021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066000f040 con 0x55ad31e65690 2024-01-11T18:06:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.502+0000 7f0654ff9700 1 -- 172.21.15.7:0/3289722021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660007870 con 0x55ad31e65690 2024-01-11T18:06:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 -- 172.21.15.7:0/3289722021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65690 msgr2=0x55ad31e65ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65690 0x55ad31e65ab0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0660011580 tx=0x7f0660009920 comp rx=0 tx=0).stop 2024-01-11T18:06:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0666d9d700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06500a1fd0 0x7f06500a4490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0660009550 tx=0x7f0660007d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 -- 172.21.15.7:0/3289722021 shutdown_connections 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e66f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65690 0x55ad31e65ab0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 --2- 172.21.15.7:0/3289722021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.503+0000 7f0680c90b80 1 -- 172.21.15.7:0/3289722021 >> 172.21.15.7:0/3289722021 conn(0x55ad31dd2740 msgr2=0x55ad31e57950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.504+0000 7f0680c90b80 1 -- 172.21.15.7:0/3289722021 shutdown_connections 2024-01-11T18:06:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.504+0000 7f0680c90b80 1 -- 172.21.15.7:0/3289722021 wait complete. 2024-01-11T18:06:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.504+0000 7f0680c90b80 1 -- start start 2024-01-11T18:06:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.504+0000 7f0680c90b80 1 Processor -- start 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.505+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65690 0x55ad31e6f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e76720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0680c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31db0cd0 con 0x55ad31e65ff0 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0680c90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31d1f340 con 0x55ad31e66ad0 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f066659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65690 0x55ad31e6f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0680c90b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31d1efe0 con 0x55ad31e65690 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f066659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65690 0x55ad31e6f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45930/0 (socket says 172.21.15.7:45930) 2024-01-11T18:06:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f066659c700 1 -- 172.21.15.7:0/67272846 learned_addr learned my addr 172.21.15.7:0/67272846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0665d9b700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0680c90b80 10 client.0 _ll_register_callbacks cb 0x55ad31d516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.506+0000 7f0666d9d700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e76720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.507+0000 7f0665d9b700 1 -- 172.21.15.7:0/67272846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65690 msgr2=0x55ad31e6f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.507+0000 7f0665d9b700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65690 0x55ad31e6f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.507+0000 7f0665d9b700 1 -- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 msgr2=0x55ad31e76720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.507+0000 7f0665d9b700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e76720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.507+0000 7f0665d9b700 1 -- 172.21.15.7:0/67272846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06681a1be0 con 0x55ad31e65ff0 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.508+0000 7f0665d9b700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6f750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f065c00baa0 tx=0x7f065c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3abbb68d0b6315 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.508+0000 7f064dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:06:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.509+0000 7f064dffb700 1 -- 172.21.15.7:0/67272846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c01b440 con 0x55ad31e65ff0 2024-01-11T18:06:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.509+0000 7f064dffb700 1 -- 172.21.15.7:0/67272846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f065c011440 con 0x55ad31e65ff0 2024-01-11T18:06:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.511+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad31e4fd60 con 0x55ad31e65ff0 2024-01-11T18:06:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.511+0000 7f064dffb700 1 -- 172.21.15.7:0/67272846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f065c011600 con 0x55ad31e65ff0 2024-01-11T18:06:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.511+0000 7f0680c90b80 10 client.16086 fetch_fsmap learned FSMap version 41 2024-01-11T18:06:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.511+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f0660011230 con 0x55ad31e65ff0 2024-01-11T18:06:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.512+0000 7f064dffb700 1 -- 172.21.15.7:0/67272846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad31e4fd60 con 0x55ad31e65ff0 2024-01-11T18:06:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.512+0000 7f064dffb700 1 -- 172.21.15.7:0/67272846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f065c01b940 con 0x55ad31e65ff0 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.512+0000 7f0680c90b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.512+0000 7f0680c90b80 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.512+0000 7f0680c90b80 20 client.16086 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.512+0000 7f0680c90b80 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31e7b080 0x55ad31e7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.513+0000 7f0680c90b80 4 client.16086 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.513+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad31da9b60 con 0x55ad31e7b080 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.513+0000 7f066659c700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31e7b080 0x55ad31e7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.513+0000 7f066659c700 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31e7b080 0x55ad31e7d4c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d83cf73fb42b375 server_cookie=f897ed599a633980 in_seq=0 out_seq=0 2024-01-11T18:06:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.513+0000 7f064dffb700 10 client.16086 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.514+0000 7f064dffb700 1 -- 172.21.15.7:0/67272846 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad31da9b60 con 0x55ad31e7b080 2024-01-11T18:06:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.514+0000 7f064dffb700 10 client.16086 handle_command_reply: tid=0 2024-01-11T18:06:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.514+0000 7f0680c90b80 1 client.16086 shutdown 2024-01-11T18:06:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 msgr2=0x55ad31e6f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6f750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f065c00baa0 tx=0x7f065c00be60 comp rx=0 tx=0).stop 2024-01-11T18:06:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 shutdown_connections 2024-01-11T18:06:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31e7b080 0x55ad31e7d4c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65690 0x55ad31e6f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e66ad0 0x55ad31e76720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f064dffb700 0 client.16086 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 --2- 172.21.15.7:0/67272846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6f750 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 >> 172.21.15.7:0/67272846 conn(0x55ad31dd2740 msgr2=0x55ad31e56f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 shutdown_connections 2024-01-11T18:06:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 1 -- 172.21.15.7:0/67272846 wait complete. 2024-01-11T18:06:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.515+0000 7f0680c90b80 20 client.16086 trim_cache size 0 max 16384 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 Processor -- start 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 -- start start 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e65a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e66ad0 0x55ad31e766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31c27640 con 0x55ad31e65ff0 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31e778a0 con 0x55ad31e65690 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.548+0000 7f0680c90b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31e6ed80 con 0x55ad31e66ad0 2024-01-11T18:06:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.549+0000 7f066659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e65a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.549+0000 7f066659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e65a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59822/0 (socket says 172.21.15.7:59822) 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.549+0000 7f066659c700 1 -- 172.21.15.7:0/1209251495 learned_addr learned my addr 172.21.15.7:0/1209251495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.549+0000 7f066659c700 1 -- 172.21.15.7:0/1209251495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e66ad0 msgr2=0x55ad31e766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.549+0000 7f066659c700 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e66ad0 0x55ad31e766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.550+0000 7f066659c700 1 -- 172.21.15.7:0/1209251495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 msgr2=0x55ad31e6ddd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.550+0000 7f066659c700 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.550+0000 7f066659c700 1 -- 172.21.15.7:0/1209251495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad31bd1b00 con 0x55ad31e65690 2024-01-11T18:06:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.562+0000 7f066659c700 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e65a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0658098740 tx=0x7f06580d1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed4b1fe6094762f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.562+0000 7f064ffff700 1 -- 172.21.15.7:0/1209251495 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06580ccea0 con 0x55ad31e65690 2024-01-11T18:06:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.562+0000 7f064ffff700 1 -- 172.21.15.7:0/1209251495 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06580bb040 con 0x55ad31e65690 2024-01-11T18:06:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.562+0000 7f064ffff700 1 -- 172.21.15.7:0/1209251495 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065809bcd0 con 0x55ad31e65690 2024-01-11T18:06:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 -- 172.21.15.7:0/1209251495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 msgr2=0x55ad31e65a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e65a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0658098740 tx=0x7f06580d1040 comp rx=0 tx=0).stop 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 -- 172.21.15.7:0/1209251495 shutdown_connections 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e66ad0 0x55ad31e766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e65a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 --2- 172.21.15.7:0/1209251495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e65ff0 0x55ad31e6ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 -- 172.21.15.7:0/1209251495 >> 172.21.15.7:0/1209251495 conn(0x55ad31dd2740 msgr2=0x55ad31e539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.563+0000 7f0680c90b80 1 -- 172.21.15.7:0/1209251495 shutdown_connections 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.564+0000 7f0680c90b80 1 -- 172.21.15.7:0/1209251495 wait complete. 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.564+0000 7f0680c90b80 1 -- start start 2024-01-11T18:06:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.565+0000 7f0680c90b80 1 Processor -- start 2024-01-11T18:06:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.567+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.567+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e71e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.567+0000 7f0680c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 0x55ad31e74e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.568+0000 7f0680c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31e75370 con 0x55ad31e66ad0 2024-01-11T18:06:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.568+0000 7f0680c90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31e754e0 con 0x55ad31e65690 2024-01-11T18:06:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.568+0000 7f0666d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 0x55ad31e74e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.569+0000 7f0680c90b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad31e756a0 con 0x55ad31e65ff0 2024-01-11T18:06:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.569+0000 7f0666d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 0x55ad31e74e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52206/0 (socket says 172.21.15.7:52206) 2024-01-11T18:06:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.569+0000 7f0666d9d700 1 -- 172.21.15.7:0/2491555156 learned_addr learned my addr 172.21.15.7:0/2491555156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:06:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.569+0000 7f0680c90b80 10 client.0 _ll_register_callbacks cb 0x55ad31d516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:06:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.571+0000 7f0666d9d700 1 -- 172.21.15.7:0/2491555156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 msgr2=0x55ad31e71e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.571+0000 7f0666d9d700 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e71e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.571+0000 7f0666d9d700 1 -- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 msgr2=0x55ad31e718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:06:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.571+0000 7f0666d9d700 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.571+0000 7f0666d9d700 1 -- 172.21.15.7:0/2491555156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06681a1760 con 0x55ad31e66ad0 2024-01-11T18:06:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.572+0000 7f0666d9d700 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 0x55ad31e74e30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0660011520 tx=0x7f0660016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4077d93676ccb455 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:06:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.572+0000 7f064dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:06:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.573+0000 7f064dffb700 1 -- 172.21.15.7:0/2491555156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660025070 con 0x55ad31e66ad0 2024-01-11T18:06:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.573+0000 7f064dffb700 1 -- 172.21.15.7:0/2491555156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f06600073e0 con 0x55ad31e66ad0 2024-01-11T18:06:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.573+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f066000f040 con 0x55ad31e66ad0 2024-01-11T18:06:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.574+0000 7f064dffb700 1 -- 172.21.15.7:0/2491555156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06600075a0 con 0x55ad31e66ad0 2024-01-11T18:06:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.574+0000 7f0680c90b80 10 client.16089 fetch_fsmap learned FSMap version 41 2024-01-11T18:06:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.574+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ad31d721d0 con 0x55ad31e66ad0 2024-01-11T18:06:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.574+0000 7f064dffb700 1 -- 172.21.15.7:0/2491555156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066000f040 con 0x55ad31e66ad0 2024-01-11T18:06:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f064dffb700 1 -- 172.21.15.7:0/2491555156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f06600056c0 con 0x55ad31e66ad0 2024-01-11T18:06:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f0680c90b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T18:06:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f0680c90b80 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:06:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f0680c90b80 20 client.16089 populate_metadata read hostname 'smithi007' 2024-01-11T18:06:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f0680c90b80 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31ee2ad0 0x55ad31ee4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:06:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f0680c90b80 4 client.16089 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:06:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.575+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ad31da9880 con 0x55ad31ee2ad0 2024-01-11T18:06:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.585+0000 7f066659c700 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31ee2ad0 0x55ad31ee4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:06:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.590+0000 7f064dffb700 10 client.16089 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.590+0000 7f066659c700 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31ee2ad0 0x55ad31ee4ef0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5450468373786d6 server_cookie=d703b4700c6ae56a in_seq=0 out_seq=0 2024-01-11T18:06:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.701+0000 7f064dffb700 1 -- 172.21.15.7:0/2491555156 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ad31da9880 con 0x55ad31ee2ad0 2024-01-11T18:06:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.701+0000 7f064dffb700 10 client.16089 handle_command_reply: tid=0 2024-01-11T18:06:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.701+0000 7f0680c90b80 1 client.16089 shutdown 2024-01-11T18:06:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.701+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 msgr2=0x55ad31e74e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.701+0000 7f0680c90b80 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 0x55ad31e74e30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0660011520 tx=0x7f0660016000 comp rx=0 tx=0).stop 2024-01-11T18:06:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 shutdown_connections 2024-01-11T18:06:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ad31ee2ad0 0x55ad31ee4ef0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad31e66ad0 0x55ad31e74e30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad31e65690 0x55ad31e718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 --2- 172.21.15.7:0/2491555156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ad31e65ff0 0x55ad31e71e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f064dffb700 0 client.16089 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 >> 172.21.15.7:0/2491555156 conn(0x55ad31dd2740 msgr2=0x55ad31e69b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 shutdown_connections 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 1 -- 172.21.15.7:0/2491555156 wait complete. 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.702+0000 7f0680c90b80 20 client.16089 trim_cache size 0 max 16384 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:06:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.703+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06500a1fd0 msgr2=0x7f06500a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.703+0000 7f066cfff700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06500a1fd0 0x7f06500a4490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0660009550 tx=0x7f0660007d70 comp rx=0 tx=0).stop 2024-01-11T18:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.703+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 msgr2=0x7f0668199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.703+0000 7f066cfff700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 0x7f0668199690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0658006010 tx=0x7f065800b9e0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 shutdown_connections 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f06500a1fd0 0x7f06500a4490 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06681094c0 0x7f06681a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0668074010 0x7f0668199690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 --2- 172.21.15.7:0/4063386486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06680755f0 0x7f0668199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 >> 172.21.15.7:0/4063386486 conn(0x7f06680fcb60 msgr2=0x7f0668105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 shutdown_connections 2024-01-11T18:06:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:06:48.704+0000 7f066cfff700 1 -- 172.21.15.7:0/4063386486 wait complete. 2024-01-11T18:06:49.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 500.0 s, 566.8 tps, lat 168.319 ms stddev 221.474 2024-01-11T18:06:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:49 smithi016 ceph-mon[43195]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 522 op/s 2024-01-11T18:06:50.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:49 smithi038 ceph-mon[43582]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 522 op/s 2024-01-11T18:06:50.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:49 smithi007 ceph-mon[33361]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 522 op/s 2024-01-11T18:06:52.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:51 smithi016 ceph-mon[43195]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 553 op/s 2024-01-11T18:06:52.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:51 smithi038 ceph-mon[43582]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 553 op/s 2024-01-11T18:06:52.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:51 smithi007 ceph-mon[33361]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 553 op/s 2024-01-11T18:06:53.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:52 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:53.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:52 smithi016 ceph-mon[43195]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.6 MiB/s wr, 589 op/s 2024-01-11T18:06:53.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:52 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:53.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:52 smithi038 ceph-mon[43582]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.6 MiB/s wr, 589 op/s 2024-01-11T18:06:53.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:52 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:06:53.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:52 smithi007 ceph-mon[33361]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.6 MiB/s wr, 589 op/s 2024-01-11T18:06:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 505.0 s, 486.4 tps, lat 200.881 ms stddev 266.638 2024-01-11T18:06:55.990 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:55 smithi016 ceph-mon[43195]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.4 MiB/s wr, 567 op/s 2024-01-11T18:06:56.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:55 smithi038 ceph-mon[43582]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.4 MiB/s wr, 567 op/s 2024-01-11T18:06:56.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:55 smithi007 ceph-mon[33361]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.4 MiB/s wr, 567 op/s 2024-01-11T18:06:58.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:57 smithi016 ceph-mon[43195]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.7 MiB/s wr, 595 op/s 2024-01-11T18:06:58.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:57 smithi038 ceph-mon[43582]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.7 MiB/s wr, 595 op/s 2024-01-11T18:06:58.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:57 smithi007 ceph-mon[33361]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.7 MiB/s wr, 595 op/s 2024-01-11T18:06:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 510.0 s, 639.4 tps, lat 166.380 ms stddev 277.195 2024-01-11T18:06:59.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:06:58 smithi016 ceph-mon[43195]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 528 op/s 2024-01-11T18:06:59.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:06:58 smithi038 ceph-mon[43582]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 528 op/s 2024-01-11T18:06:59.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:06:58 smithi007 ceph-mon[33361]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 528 op/s 2024-01-11T18:07:01.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:01 smithi007 ceph-mon[33361]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 545 op/s 2024-01-11T18:07:01.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:01 smithi016 ceph-mon[43195]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 545 op/s 2024-01-11T18:07:01.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:01 smithi038 ceph-mon[43582]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 545 op/s 2024-01-11T18:07:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.983+0000 7f919e5ff700 1 -- 172.21.15.7:0/1547327807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 msgr2=0x7f9198104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.983+0000 7f919e5ff700 1 --2- 172.21.15.7:0/1547327807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198104410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9190009f90 tx=0x7f919000b330 comp rx=0 tx=0).stop 2024-01-11T18:07:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 -- 172.21.15.7:0/1547327807 shutdown_connections 2024-01-11T18:07:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 --2- 172.21.15.7:0/1547327807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198107280 0x7f91981096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 --2- 172.21.15.7:0/1547327807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198104410 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 --2- 172.21.15.7:0/1547327807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198104950 0x7f9198106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 -- 172.21.15.7:0/1547327807 >> 172.21.15.7:0/1547327807 conn(0x7f91980fba30 msgr2=0x7f91980fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 -- 172.21.15.7:0/1547327807 shutdown_connections 2024-01-11T18:07:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.986+0000 7f919e5ff700 1 -- 172.21.15.7:0/1547327807 wait complete. 2024-01-11T18:07:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.988+0000 7f919e5ff700 1 Processor -- start 2024-01-11T18:07:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.989+0000 7f919e5ff700 1 -- start start 2024-01-11T18:07:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f919e5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f919e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198104950 0x7f9198199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f919e5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198107280 0x7f91981a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f919e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981a1080 con 0x7f9198104950 2024-01-11T18:07:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f919e5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981a11c0 con 0x7f9198102020 2024-01-11T18:07:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f919e5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981a1350 con 0x7f9198107280 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f9197fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f9197fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59458/0 (socket says 172.21.15.7:59458) 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.990+0000 7f9197fff700 1 -- 172.21.15.7:0/3898733708 learned_addr learned my addr 172.21.15.7:0/3898733708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.991+0000 7f9197fff700 1 -- 172.21.15.7:0/3898733708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198107280 msgr2=0x7f91981a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.991+0000 7f9197fff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198107280 0x7f91981a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.991+0000 7f9197fff700 1 -- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198104950 msgr2=0x7f9198199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.991+0000 7f9197fff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198104950 0x7f9198199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:01.991+0000 7f9197fff700 1 -- 172.21.15.7:0/3898733708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9190009c40 con 0x7f9198102020 2024-01-11T18:07:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.000+0000 7f9197fff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198199550 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9190003fd0 tx=0x7f9190009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.001+0000 7f91957fa700 1 -- 172.21.15.7:0/3898733708 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000e050 con 0x7f9198102020 2024-01-11T18:07:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.001+0000 7f91957fa700 1 -- 172.21.15.7:0/3898733708 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9190005570 con 0x7f9198102020 2024-01-11T18:07:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.001+0000 7f91957fa700 1 -- 172.21.15.7:0/3898733708 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919001b980 con 0x7f9198102020 2024-01-11T18:07:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.001+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91981a15d0 con 0x7f9198102020 2024-01-11T18:07:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.001+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91981a1ad0 con 0x7f9198102020 2024-01-11T18:07:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.010+0000 7f91b23f2b80 1 Processor -- start 2024-01-11T18:07:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.011+0000 7f91b23f2b80 1 -- start start 2024-01-11T18:07:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.011+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555706324f20 0x555706325340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706326090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557063265d0 0x55570632ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55570621c030 con 0x5557063265d0 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55570627fac0 con 0x555706324f20 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55570627d3e0 con 0x555706325c10 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f9197fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706326090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f9197fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706326090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:47854/0 (socket says 172.21.15.7:47854) 2024-01-11T18:07:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.014+0000 7f9197fff700 1 -- 172.21.15.7:0/2452292387 learned_addr learned my addr 172.21.15.7:0/2452292387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.016+0000 7f91977fe700 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557063265d0 0x55570632ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.016+0000 7f91977fe700 1 -- 172.21.15.7:0/2452292387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 msgr2=0x555706326090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.016+0000 7f91977fe700 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706326090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.016+0000 7f91977fe700 1 -- 172.21.15.7:0/2452292387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555706324f20 msgr2=0x555706325340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.016+0000 7f91977fe700 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555706324f20 0x555706325340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.016+0000 7f91977fe700 1 -- 172.21.15.7:0/2452292387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557060ed350 con 0x5557063265d0 2024-01-11T18:07:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.017+0000 7f91977fe700 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557063265d0 0x55570632ce60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f918c009b80 tx=0x7f918c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b095d65db80491d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.018+0000 7f917e7fc700 1 -- 172.21.15.7:0/2452292387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c004230 con 0x5557063265d0 2024-01-11T18:07:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.018+0000 7f917e7fc700 1 -- 172.21.15.7:0/2452292387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f918c00eda0 con 0x5557063265d0 2024-01-11T18:07:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.018+0000 7f917e7fc700 1 -- 172.21.15.7:0/2452292387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c005020 con 0x5557063265d0 2024-01-11T18:07:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 -- 172.21.15.7:0/2452292387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557063265d0 msgr2=0x55570632ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557063265d0 0x55570632ce60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f918c009b80 tx=0x7f918c009e90 comp rx=0 tx=0).stop 2024-01-11T18:07:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 -- 172.21.15.7:0/2452292387 shutdown_connections 2024-01-11T18:07:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557063265d0 0x55570632ce60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555706324f20 0x555706325340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/2452292387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706326090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 -- 172.21.15.7:0/2452292387 >> 172.21.15.7:0/2452292387 conn(0x55570629a740 msgr2=0x55570631f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 -- 172.21.15.7:0/2452292387 shutdown_connections 2024-01-11T18:07:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.019+0000 7f91b23f2b80 1 -- 172.21.15.7:0/2452292387 wait complete. 2024-01-11T18:07:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.020+0000 7f91b23f2b80 1 -- start start 2024-01-11T18:07:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.021+0000 7f91b23f2b80 1 Processor -- start 2024-01-11T18:07:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.022+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x55570632eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706335eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555706278cd0 con 0x555706324f20 2024-01-11T18:07:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557061e7340 con 0x5557063265d0 2024-01-11T18:07:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557061e6fe0 con 0x555706325c10 2024-01-11T18:07:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91b23f2b80 10 client.0 _ll_register_callbacks cb 0x5557062196c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f919cb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f919cb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48702/0 (socket says 172.21.15.7:48702) 2024-01-11T18:07:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f919cb9c700 1 -- 172.21.15.7:0/1373334886 learned_addr learned my addr 172.21.15.7:0/1373334886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.028+0000 7f91977fe700 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706335eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.029+0000 7f91977fe700 1 -- 172.21.15.7:0/1373334886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 msgr2=0x55570632eec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:07:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.029+0000 7f91977fe700 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x55570632eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.029+0000 7f91977fe700 1 -- 172.21.15.7:0/1373334886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 msgr2=0x55570632e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.029+0000 7f91977fe700 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.029+0000 7f91977fe700 1 -- 172.21.15.7:0/1373334886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c0097f0 con 0x5557063265d0 2024-01-11T18:07:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.030+0000 7f91957fa700 1 -- 172.21.15.7:0/3898733708 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f91900056e0 con 0x7f9198102020 2024-01-11T18:07:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.032+0000 7f91957fa700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a1f30 0x7f91800a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.033+0000 7f91957fa700 1 -- 172.21.15.7:0/3898733708 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f91900cfc20 con 0x7f9198102020 2024-01-11T18:07:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.033+0000 7f919cb9c700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a1f30 0x7f91800a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.033+0000 7f91977fe700 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706335eb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f918c00a010 tx=0x7f918c00ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6491bee7a6d354d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.034+0000 7f91777fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:07:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.034+0000 7f91777fe700 1 -- 172.21.15.7:0/1373334886 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c005230 con 0x5557063265d0 2024-01-11T18:07:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.034+0000 7f91777fe700 1 -- 172.21.15.7:0/1373334886 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f918c014070 con 0x5557063265d0 2024-01-11T18:07:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.034+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555706317d60 con 0x5557063265d0 2024-01-11T18:07:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.034+0000 7f919cb9c700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a1f30 0x7f91800a43f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9188006fd0 tx=0x7f9188008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.043+0000 7f91777fe700 1 -- 172.21.15.7:0/1373334886 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918c005390 con 0x5557063265d0 2024-01-11T18:07:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.043+0000 7f91b23f2b80 10 client.26255 fetch_fsmap learned FSMap version 42 2024-01-11T18:07:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.044+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x555706099b00 con 0x5557063265d0 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.052+0000 7f91777fe700 1 -- 172.21.15.7:0/1373334886 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555706317d60 con 0x5557063265d0 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91777fe700 1 -- 172.21.15.7:0/1373334886 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 42) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f918c02a070 con 0x5557063265d0 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91b23f2b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 42 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91b23f2b80 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91b23f2b80 20 client.26255 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x5557063451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91b23f2b80 4 client.26255 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:07:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555706271b60 con 0x555706342dd0 2024-01-11T18:07:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.053+0000 7f9197fff700 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x5557063451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.061+0000 7f9197fff700 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x5557063451f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad87f642bc53f5e9 server_cookie=17ce42b56dc389b in_seq=0 out_seq=0 2024-01-11T18:07:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.061+0000 7f91777fe700 10 client.26255 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.062+0000 7f91777fe700 1 -- 172.21.15.7:0/1373334886 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555706271b60 con 0x555706342dd0 2024-01-11T18:07:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.062+0000 7f91777fe700 10 client.26255 handle_command_reply: tid=0 2024-01-11T18:07:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.062+0000 7f91b23f2b80 1 client.26255 shutdown 2024-01-11T18:07:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 msgr2=0x555706335eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706335eb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f918c00a010 tx=0x7f918c00ebc0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 shutdown_connections 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x5557063451f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91777fe700 0 client.26255 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706335eb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1373334886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x55570632eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.063+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 >> 172.21.15.7:0/1373334886 conn(0x55570629a740 msgr2=0x55570631ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.064+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 shutdown_connections 2024-01-11T18:07:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.064+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1373334886 wait complete. 2024-01-11T18:07:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.064+0000 7f91b23f2b80 20 client.26255 trim_cache size 0 max 16384 2024-01-11T18:07:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.107+0000 7f91b23f2b80 1 Processor -- start 2024-01-11T18:07:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.107+0000 7f91b23f2b80 1 -- start start 2024-01-11T18:07:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706335e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706329630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f9197fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557060ef640 con 0x555706324f20 2024-01-11T18:07:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55570632f9a0 con 0x5557063265d0 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557063257a0 con 0x555706325c10 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f919cb9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706329630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f919cb9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706329630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59488/0 (socket says 172.21.15.7:59488) 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f9197fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48706/0 (socket says 172.21.15.7:48706) 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.108+0000 7f919cb9c700 1 -- 172.21.15.7:0/1087697434 learned_addr learned my addr 172.21.15.7:0/1087697434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.109+0000 7f9197fff700 1 -- 172.21.15.7:0/1087697434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 msgr2=0x555706335e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.109+0000 7f9197fff700 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706335e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.109+0000 7f9197fff700 1 -- 172.21.15.7:0/1087697434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 msgr2=0x555706329630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.109+0000 7f9197fff700 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706329630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.109+0000 7f9197fff700 1 -- 172.21.15.7:0/1087697434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55570623a1d0 con 0x555706324f20 2024-01-11T18:07:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.110+0000 7f9197fff700 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632d610 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9190003fa0 tx=0x7f919000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9678ec3960ba49 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.110+0000 7f917dffb700 1 -- 172.21.15.7:0/1087697434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919009c680 con 0x555706324f20 2024-01-11T18:07:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.110+0000 7f917dffb700 1 -- 172.21.15.7:0/1087697434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91900a07d0 con 0x555706324f20 2024-01-11T18:07:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.110+0000 7f917dffb700 1 -- 172.21.15.7:0/1087697434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919009c8e0 con 0x555706324f20 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.111+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1087697434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 msgr2=0x55570632d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.111+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632d610 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9190003fa0 tx=0x7f919000b680 comp rx=0 tx=0).stop 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1087697434 shutdown_connections 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x555706329630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706324f20 0x55570632d610 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/1087697434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706325c10 0x555706335e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1087697434 >> 172.21.15.7:0/1087697434 conn(0x55570629a740 msgr2=0x55570632cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1087697434 shutdown_connections 2024-01-11T18:07:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.112+0000 7f91b23f2b80 1 -- 172.21.15.7:0/1087697434 wait complete. 2024-01-11T18:07:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.113+0000 7f91b23f2b80 1 -- start start 2024-01-11T18:07:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.116+0000 7f91b23f2b80 1 Processor -- start 2024-01-11T18:07:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.117+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 0x5557063334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.117+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706325c10 0x555706333a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 0x5557063334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f91b23f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x5557063349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555706334f00 con 0x555706325c10 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555706335070 con 0x5557063265d0 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f91b23f2b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555706335230 con 0x555706324f20 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f91b23f2b80 10 client.0 _ll_register_callbacks cb 0x5557062196c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 0x5557063334c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:47862/0 (socket says 172.21.15.7:47862) 2024-01-11T18:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 -- 172.21.15.7:0/3772317767 learned_addr learned my addr 172.21.15.7:0/3772317767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f919cb9c700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x5557063349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f91977fe700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706325c10 0x555706333a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 -- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 msgr2=0x5557063349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x5557063349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 -- 172.21.15.7:0/3772317767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706325c10 msgr2=0x555706333a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706325c10 0x555706333a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.118+0000 7f9197fff700 1 -- 172.21.15.7:0/3772317767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91981a1ad0 con 0x555706324f20 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.119+0000 7f91977fe700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706325c10 0x555706333a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.119+0000 7f9197fff700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 0x5557063334c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f919000b370 tx=0x7f91900cd5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd25d3e0a85e8350 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91777fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91777fe700 1 -- 172.21.15.7:0/3772317767 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919009bd70 con 0x555706324f20 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91777fe700 1 -- 172.21.15.7:0/3772317767 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f919009c680 con 0x555706324f20 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f918c00eda0 con 0x555706324f20 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91777fe700 1 -- 172.21.15.7:0/3772317767 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919009c8e0 con 0x555706324f20 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91b23f2b80 10 client.26278 fetch_fsmap learned FSMap version 42 2024-01-11T18:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.120+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5557060f1c70 con 0x555706324f20 2024-01-11T18:07:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.122+0000 7f91777fe700 1 -- 172.21.15.7:0/3772317767 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91900ac080 con 0x555706324f20 2024-01-11T18:07:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.122+0000 7f91777fe700 1 -- 172.21.15.7:0/3772317767 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 42) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f91900a44f0 con 0x555706324f20 2024-01-11T18:07:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.122+0000 7f91b23f2b80 10 client.26278 fetch_fsmap finished waiting for FSMap version 42 2024-01-11T18:07:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.122+0000 7f91b23f2b80 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.122+0000 7f91b23f2b80 20 client.26278 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.122+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x55570639b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.124+0000 7f91b23f2b80 4 client.26278 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:07:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.124+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555706271880 con 0x555706342dd0 2024-01-11T18:07:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.124+0000 7f91977fe700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x55570639b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.125+0000 7f91977fe700 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x55570639b7a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29dab639047885b5 server_cookie=d862f089adb39457 in_seq=0 out_seq=0 2024-01-11T18:07:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.125+0000 7f91777fe700 10 client.26278 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91777fe700 1 -- 172.21.15.7:0/3772317767 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555706271880 con 0x555706342dd0 2024-01-11T18:07:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91777fe700 10 client.26278 handle_command_reply: tid=0 2024-01-11T18:07:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 client.26278 shutdown 2024-01-11T18:07:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 msgr2=0x5557063334c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 0x5557063334c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f919000b370 tx=0x7f91900cd5f0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 shutdown_connections 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555706342dd0 0x55570639b7a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91777fe700 0 client.26278 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706324f20 0x5557063334c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557063265d0 0x5557063349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 --2- 172.21.15.7:0/3772317767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555706325c10 0x555706333a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 >> 172.21.15.7:0/3772317767 conn(0x55570629a740 msgr2=0x555706329630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.163+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 shutdown_connections 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.164+0000 7f91b23f2b80 1 -- 172.21.15.7:0/3772317767 wait complete. 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.164+0000 7f91b23f2b80 20 client.26278 trim_cache size 0 max 16384 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:07:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:07:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a1f30 msgr2=0x7f91800a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a1f30 0x7f91800a43f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9188006fd0 tx=0x7f9188008040 comp rx=0 tx=0).stop 2024-01-11T18:07:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 msgr2=0x7f9198199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198199550 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9190003fd0 tx=0x7f9190009bb0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 shutdown_connections 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f91800a1f30 0x7f91800a43f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9198107280 0x7f91981a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9198102020 0x7f9198199550 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 --2- 172.21.15.7:0/3898733708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9198104950 0x7f9198199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 >> 172.21.15.7:0/3898733708 conn(0x7f91980fba30 msgr2=0x7f9198100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 shutdown_connections 2024-01-11T18:07:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:02.165+0000 7f919e5ff700 1 -- 172.21.15.7:0/3898733708 wait complete. 2024-01-11T18:07:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 515.0 s, 542.8 tps, lat 180.916 ms stddev 214.827 2024-01-11T18:07:04.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:03 smithi016 ceph-mon[43195]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 555 op/s 2024-01-11T18:07:04.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:03 smithi038 ceph-mon[43582]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 555 op/s 2024-01-11T18:07:04.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:03 smithi007 ceph-mon[33361]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 555 op/s 2024-01-11T18:07:05.988 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:05 smithi016 ceph-mon[43195]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2024-01-11T18:07:06.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:05 smithi038 ceph-mon[43582]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2024-01-11T18:07:06.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:05 smithi007 ceph-mon[33361]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2024-01-11T18:07:08.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:07 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:07:08.242 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:07 smithi038 ceph-mon[43582]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 MiB/s wr, 496 op/s 2024-01-11T18:07:08.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:07 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:07:08.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:07 smithi016 ceph-mon[43195]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 MiB/s wr, 496 op/s 2024-01-11T18:07:08.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:07 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:07:08.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:07 smithi007 ceph-mon[33361]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 MiB/s wr, 496 op/s 2024-01-11T18:07:09.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 520.0 s, 495.6 tps, lat 204.246 ms stddev 213.928 2024-01-11T18:07:09.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:09 smithi038 ceph-mon[43582]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.0 MiB/s wr, 471 op/s 2024-01-11T18:07:09.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:09 smithi007 ceph-mon[33361]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.0 MiB/s wr, 471 op/s 2024-01-11T18:07:09.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:09 smithi016 ceph-mon[43195]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.0 MiB/s wr, 471 op/s 2024-01-11T18:07:12.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:11 smithi016 ceph-mon[43195]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.3 MiB/s wr, 520 op/s 2024-01-11T18:07:12.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:11 smithi038 ceph-mon[43582]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.3 MiB/s wr, 520 op/s 2024-01-11T18:07:12.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:11 smithi007 ceph-mon[33361]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.3 MiB/s wr, 520 op/s 2024-01-11T18:07:13.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:13 smithi038 ceph-mon[43582]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.3 MiB/s wr, 519 op/s 2024-01-11T18:07:13.317 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:13 smithi007 ceph-mon[33361]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.3 MiB/s wr, 519 op/s 2024-01-11T18:07:13.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:13 smithi016 ceph-mon[43195]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.3 MiB/s wr, 519 op/s 2024-01-11T18:07:14.116 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 525.0 s, 498.2 tps, lat 195.064 ms stddev 224.842 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.470+0000 7fafd7fff700 1 -- 172.21.15.7:0/1788559683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 msgr2=0x7fafd8102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.470+0000 7fafd7fff700 1 --2- 172.21.15.7:0/1788559683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 0x7fafd8102a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009920 tx=0x7fafc0009c30 comp rx=0 tx=0).stop 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 -- 172.21.15.7:0/1788559683 shutdown_connections 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 --2- 172.21.15.7:0/1788559683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd81013e0 0x7fafd8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 --2- 172.21.15.7:0/1788559683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd8102f70 0x7fafd8109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 --2- 172.21.15.7:0/1788559683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 0x7fafd8102a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 -- 172.21.15.7:0/1788559683 >> 172.21.15.7:0/1788559683 conn(0x7fafd80fc930 msgr2=0x7fafd80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 -- 172.21.15.7:0/1788559683 shutdown_connections 2024-01-11T18:07:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.471+0000 7fafd7fff700 1 -- 172.21.15.7:0/1788559683 wait complete. 2024-01-11T18:07:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 Processor -- start 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 -- start start 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd81013e0 0x7fafd8197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 0x7fafd81978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd8102f70 0x7fafd819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819eef0 con 0x7fafd81013e0 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819f030 con 0x7fafd81025d0 2024-01-11T18:07:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.473+0000 7fafd7fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819f1c0 con 0x7fafd8102f70 2024-01-11T18:07:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.474+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd8102f70 0x7fafd819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.474+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd8102f70 0x7fafd819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59272/0 (socket says 172.21.15.7:59272) 2024-01-11T18:07:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.474+0000 7fafd77fe700 1 -- 172.21.15.7:0/2157425565 learned_addr learned my addr 172.21.15.7:0/2157425565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.474+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd81013e0 0x7fafd8197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.475+0000 7fafd6ffd700 1 -- 172.21.15.7:0/2157425565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd8102f70 msgr2=0x7fafd819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.475+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd8102f70 0x7fafd819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.475+0000 7fafd6ffd700 1 -- 172.21.15.7:0/2157425565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 msgr2=0x7fafd81978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.475+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 0x7fafd81978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.475+0000 7fafd6ffd700 1 -- 172.21.15.7:0/2157425565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009520 con 0x7fafd81013e0 2024-01-11T18:07:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.476+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd81013e0 0x7fafd8197370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fafc8007f30 tx=0x7fafc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.479+0000 7fafd4ff9700 1 -- 172.21.15.7:0/2157425565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc800dc80 con 0x7fafd81013e0 2024-01-11T18:07:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.479+0000 7fafd4ff9700 1 -- 172.21.15.7:0/2157425565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc800f040 con 0x7fafd81013e0 2024-01-11T18:07:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.479+0000 7fafd4ff9700 1 -- 172.21.15.7:0/2157425565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8014600 con 0x7fafd81013e0 2024-01-11T18:07:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.480+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd80738a0 con 0x7fafd81013e0 2024-01-11T18:07:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.480+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd8073db0 con 0x7fafd81013e0 2024-01-11T18:07:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.489+0000 7faff1066b80 1 Processor -- start 2024-01-11T18:07:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.491+0000 7faff1066b80 1 -- start start 2024-01-11T18:07:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.494+0000 7fafd4ff9700 1 -- 172.21.15.7:0/2157425565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fafc8015400 con 0x7fafd81013e0 2024-01-11T18:07:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7fafd4ff9700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fafc40a63d0 0x7fafc40a8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7faff1066b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7faff1066b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706a3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7faff1066b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7faff1066b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f270599030 con 0x55f2706a2eb0 2024-01-11T18:07:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7faff1066b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2705fcac0 con 0x55f2706a2120 2024-01-11T18:07:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.497+0000 7faff1066b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2705fa3e0 con 0x55f2706a3870 2024-01-11T18:07:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.498+0000 7fafd6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.499+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.499+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706aa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59284/0 (socket says 172.21.15.7:59284) 2024-01-11T18:07:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.499+0000 7fafd77fe700 1 -- 172.21.15.7:0/1015581238 learned_addr learned my addr 172.21.15.7:0/1015581238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.499+0000 7fafd4ff9700 1 -- 172.21.15.7:0/2157425565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fafc80cd630 con 0x7fafd81013e0 2024-01-11T18:07:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.499+0000 7fafd77fe700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fafc40a63d0 0x7fafc40a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafce7fc700 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1015581238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 msgr2=0x55f2706aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1015581238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 msgr2=0x55f2706a3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706a3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1015581238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f27046a350 con 0x55f2706a2120 2024-01-11T18:07:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.500+0000 7fafce7fc700 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:07:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.501+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706a2540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafc8096500 tx=0x7fafc806cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db57131af5f5c848 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.501+0000 7fafd77fe700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fafc40a63d0 0x7fafc40a8890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fafd0006f60 tx=0x7fafd0009a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.502+0000 7fafcd7fa700 1 -- 172.21.15.7:0/1015581238 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8099960 con 0x55f2706a2120 2024-01-11T18:07:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.502+0000 7fafcd7fa700 1 -- 172.21.15.7:0/1015581238 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc809a700 con 0x55f2706a2120 2024-01-11T18:07:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 -- 172.21.15.7:0/1015581238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 msgr2=0x55f2706a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706a2540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafc8096500 tx=0x7fafc806cd80 comp rx=0 tx=0).stop 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 -- 172.21.15.7:0/1015581238 shutdown_connections 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706a2540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 --2- 172.21.15.7:0/1015581238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.503+0000 7faff1066b80 1 -- 172.21.15.7:0/1015581238 >> 172.21.15.7:0/1015581238 conn(0x55f270617740 msgr2=0x55f27069cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.504+0000 7faff1066b80 1 -- 172.21.15.7:0/1015581238 shutdown_connections 2024-01-11T18:07:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.504+0000 7faff1066b80 1 -- 172.21.15.7:0/1015581238 wait complete. 2024-01-11T18:07:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.505+0000 7faff1066b80 1 -- start start 2024-01-11T18:07:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.505+0000 7faff1066b80 1 Processor -- start 2024-01-11T18:07:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.506+0000 7faff1066b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.506+0000 7faff1066b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a2eb0 0x55f2706b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.506+0000 7faff1066b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a3870 0x55f2706bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7faff1066b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2705f5cd0 con 0x55f2706a3870 2024-01-11T18:07:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7faff1066b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f270564340 con 0x55f2706a2120 2024-01-11T18:07:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7faff1066b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f270563fe0 con 0x55f2706a2eb0 2024-01-11T18:07:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7faff1066b80 10 client.0 _ll_register_callbacks cb 0x55f2705966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7fafd6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7fafd6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58976/0 (socket says 172.21.15.7:58976) 2024-01-11T18:07:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.507+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1570565473 learned_addr learned my addr 172.21.15.7:0/1570565473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1570565473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a2eb0 msgr2=0x55f2706b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a2eb0 0x55f2706b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1570565473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a3870 msgr2=0x55f2706bb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a3870 0x55f2706bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafd6ffd700 1 -- 172.21.15.7:0/1570565473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd8073db0 con 0x55f2706a2120 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.508+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b4340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafc80780b0 tx=0x7fafc8015c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209815813ef7df05 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.509+0000 7fafbe7fc700 1 -- 172.21.15.7:0/1570565473 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80994f0 con 0x55f2706a2120 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.509+0000 7fafbe7fc700 1 -- 172.21.15.7:0/1570565473 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fafc8099650 con 0x55f2706a2120 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.509+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f270694fb0 con 0x55f2706a2120 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.509+0000 7fafbe7fc700 1 -- 172.21.15.7:0/1570565473 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc80994f0 con 0x55f2706a2120 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.509+0000 7faff1066b80 10 client.26270 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.509+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f270416b00 con 0x55f2706a2120 2024-01-11T18:07:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.510+0000 7fafbe7fc700 1 -- 172.21.15.7:0/1570565473 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc80b8240 con 0x55f2706a2120 2024-01-11T18:07:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.510+0000 7fafbe7fc700 1 -- 172.21.15.7:0/1570565473 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fafc8095f00 con 0x55f2706a2120 2024-01-11T18:07:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.511+0000 7faff1066b80 10 client.26270 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.511+0000 7faff1066b80 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.511+0000 7faff1066b80 20 client.26270 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.511+0000 7faff1066b80 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706bfe90 0x55f2706c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.511+0000 7faff1066b80 4 client.26270 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:07:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.511+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f270694fb0 con 0x55f2706bfe90 2024-01-11T18:07:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.512+0000 7fafce7fc700 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706bfe90 0x55f2706c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.513+0000 7fafbe7fc700 10 client.26270 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.513+0000 7fafce7fc700 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706bfe90 0x55f2706c22f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b346f413a16b4d38 server_cookie=1f37b34a06d954b6 in_seq=0 out_seq=0 2024-01-11T18:07:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.514+0000 7fafbe7fc700 1 -- 172.21.15.7:0/1570565473 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f270694fb0 con 0x55f2706bfe90 2024-01-11T18:07:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.514+0000 7fafbe7fc700 10 client.26270 handle_command_reply: tid=0 2024-01-11T18:07:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.514+0000 7faff1066b80 1 client.26270 shutdown 2024-01-11T18:07:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 msgr2=0x55f2706b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b4340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fafc80780b0 tx=0x7fafc8015c00 comp rx=0 tx=0).stop 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 shutdown_connections 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706bfe90 0x55f2706c22f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7fafbe7fc700 0 client.26270 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a3870 0x55f2706bb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b4340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 --2- 172.21.15.7:0/1570565473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a2eb0 0x55f2706b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 >> 172.21.15.7:0/1570565473 conn(0x55f270617740 msgr2=0x55f27069c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 shutdown_connections 2024-01-11T18:07:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.515+0000 7faff1066b80 1 -- 172.21.15.7:0/1570565473 wait complete. 2024-01-11T18:07:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.516+0000 7faff1066b80 20 client.26270 trim_cache size 0 max 16384 2024-01-11T18:07:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 Processor -- start 2024-01-11T18:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 -- start start 2024-01-11T18:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27046c640 con 0x55f2706a2eb0 2024-01-11T18:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.557+0000 7faff1066b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2706a6330 con 0x55f2706a2120 2024-01-11T18:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.558+0000 7faff1066b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2706bfc60 con 0x55f2706a3870 2024-01-11T18:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.558+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.558+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59302/0 (socket says 172.21.15.7:59302) 2024-01-11T18:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.558+0000 7fafd77fe700 1 -- 172.21.15.7:0/1451882020 learned_addr learned my addr 172.21.15.7:0/1451882020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.558+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.560+0000 7fafd77fe700 1 -- 172.21.15.7:0/1451882020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 msgr2=0x55f2706bbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.560+0000 7fafd77fe700 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706bbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.560+0000 7fafd77fe700 1 -- 172.21.15.7:0/1451882020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 msgr2=0x55f2706bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.560+0000 7fafd77fe700 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.560+0000 7fafd77fe700 1 -- 172.21.15.7:0/1451882020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2705b71d0 con 0x55f2706a3870 2024-01-11T18:07:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.561+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:07:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.563+0000 7fafd77fe700 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706b38c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009e30 tx=0x7fafd001a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf05671cfb567683 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.564+0000 7fafccff9700 1 -- 172.21.15.7:0/1451882020 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0015e70 con 0x55f2706a3870 2024-01-11T18:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.564+0000 7fafccff9700 1 -- 172.21.15.7:0/1451882020 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafd0017ca0 con 0x55f2706a3870 2024-01-11T18:07:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.564+0000 7fafccff9700 1 -- 172.21.15.7:0/1451882020 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0015e70 con 0x55f2706a3870 2024-01-11T18:07:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.566+0000 7faff1066b80 1 -- 172.21.15.7:0/1451882020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 msgr2=0x55f2706b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.566+0000 7faff1066b80 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706b38c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009e30 tx=0x7fafd001a070 comp rx=0 tx=0).stop 2024-01-11T18:07:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 -- 172.21.15.7:0/1451882020 shutdown_connections 2024-01-11T18:07:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706b38c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 --2- 172.21.15.7:0/1451882020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 -- 172.21.15.7:0/1451882020 >> 172.21.15.7:0/1451882020 conn(0x55f270617740 msgr2=0x55f270698c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 -- 172.21.15.7:0/1451882020 shutdown_connections 2024-01-11T18:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.567+0000 7faff1066b80 1 -- 172.21.15.7:0/1451882020 wait complete. 2024-01-11T18:07:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.568+0000 7faff1066b80 1 -- start start 2024-01-11T18:07:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.568+0000 7faff1066b80 1 Processor -- start 2024-01-11T18:07:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.569+0000 7faff1066b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.570+0000 7faff1066b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.570+0000 7faff1066b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7faff1066b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2706bb180 con 0x55f2706a2eb0 2024-01-11T18:07:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7faff1066b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2706bb2f0 con 0x55f2706a2120 2024-01-11T18:07:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7faff1066b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2706bb4b0 con 0x55f2706a3870 2024-01-11T18:07:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7faff1066b80 10 client.0 _ll_register_callbacks cb 0x55f2705966c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7fafd6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59304/0 (socket says 172.21.15.7:59304) 2024-01-11T18:07:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.571+0000 7fafd77fe700 1 -- 172.21.15.7:0/3537327340 learned_addr learned my addr 172.21.15.7:0/3537327340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafd77fe700 1 -- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 msgr2=0x55f2706b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafd77fe700 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafd77fe700 1 -- 172.21.15.7:0/3537327340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 msgr2=0x55f2706b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafd77fe700 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafd77fe700 1 -- 172.21.15.7:0/3537327340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f270416b00 con 0x55f2706a3870 2024-01-11T18:07:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:07:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.572+0000 7fafd77fe700 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706bac40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009710 tx=0x7fafd0006e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a41517e23967c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.573+0000 7fafbe7fc700 1 -- 172.21.15.7:0/3537327340 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd002d660 con 0x55f2706a3870 2024-01-11T18:07:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.573+0000 7fafbe7fc700 1 -- 172.21.15.7:0/3537327340 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fafd001e040 con 0x55f2706a3870 2024-01-11T18:07:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.573+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafc809a700 con 0x55f2706a3870 2024-01-11T18:07:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.575+0000 7fafbe7fc700 1 -- 172.21.15.7:0/3537327340 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafd00069b0 con 0x55f2706a3870 2024-01-11T18:07:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.575+0000 7faff1066b80 10 client.26293 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.575+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fafd0015480 con 0x55f2706a3870 2024-01-11T18:07:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7fafbe7fc700 1 -- 172.21.15.7:0/3537327340 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc809a700 con 0x55f2706a3870 2024-01-11T18:07:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7fafbe7fc700 1 -- 172.21.15.7:0/3537327340 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fafd00226a0 con 0x55f2706a3870 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7faff1066b80 10 client.26293 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7faff1066b80 10 client.26293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7faff1066b80 20 client.26293 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7faff1066b80 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706aed30 0x55f2706b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7faff1066b80 4 client.26293 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.577+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2705eeb60 con 0x55f2706aed30 2024-01-11T18:07:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.578+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706aed30 0x55f2706b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.578+0000 7fafd6ffd700 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706aed30 0x55f2706b1150 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebdee69b3e79a3d server_cookie=44e0f0d30442e33e in_seq=0 out_seq=0 2024-01-11T18:07:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.578+0000 7fafbe7fc700 10 client.26293 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.721+0000 7fafbe7fc700 1 -- 172.21.15.7:0/3537327340 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2705eeb60 con 0x55f2706aed30 2024-01-11T18:07:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.721+0000 7fafbe7fc700 10 client.26293 handle_command_reply: tid=0 2024-01-11T18:07:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.721+0000 7faff1066b80 1 client.26293 shutdown 2024-01-11T18:07:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.721+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 msgr2=0x55f2706bac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.721+0000 7faff1066b80 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706bac40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009710 tx=0x7fafd0006e10 comp rx=0 tx=0).stop 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 shutdown_connections 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f2706aed30 0x55f2706b1150 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2706a3870 0x55f2706bac40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2706a2120 0x55f2706b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 --2- 172.21.15.7:0/3537327340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2706a2eb0 0x55f2706b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 >> 172.21.15.7:0/3537327340 conn(0x55f270617740 msgr2=0x55f2706b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7fafbe7fc700 0 client.26293 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 shutdown_connections 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.722+0000 7faff1066b80 1 -- 172.21.15.7:0/3537327340 wait complete. 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.723+0000 7faff1066b80 20 client.26293 trim_cache size 0 max 16384 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:07:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fafc40a63d0 msgr2=0x7fafc40a8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fafc40a63d0 0x7fafc40a8890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fafd0006f60 tx=0x7fafd0009a50 comp rx=0 tx=0).stop 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd81013e0 msgr2=0x7fafd8197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd81013e0 0x7fafd8197370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fafc8007f30 tx=0x7fafc800d3b0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 shutdown_connections 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fafc40a63d0 0x7fafc40a8890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafd8102f70 0x7fafd819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafd81013e0 0x7fafd8197370 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 --2- 172.21.15.7:0/2157425565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd81025d0 0x7fafd81978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.724+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 >> 172.21.15.7:0/2157425565 conn(0x7fafd80fc930 msgr2=0x7fafd81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.725+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 shutdown_connections 2024-01-11T18:07:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:15.725+0000 7fafd7fff700 1 -- 172.21.15.7:0/2157425565 wait complete. 2024-01-11T18:07:15.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:15 smithi016 ceph-mon[43195]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-01-11T18:07:16.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:15 smithi038 ceph-mon[43582]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-01-11T18:07:16.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:15 smithi007 ceph-mon[33361]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-01-11T18:07:18.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:17 smithi016 ceph-mon[43195]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.6 MiB/s wr, 475 op/s 2024-01-11T18:07:18.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:17 smithi038 ceph-mon[43582]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.6 MiB/s wr, 475 op/s 2024-01-11T18:07:18.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:17 smithi007 ceph-mon[33361]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.6 MiB/s wr, 475 op/s 2024-01-11T18:07:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 530.0 s, 472.2 tps, lat 215.028 ms stddev 279.743 2024-01-11T18:07:19.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:19 smithi007 ceph-mon[33361]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-01-11T18:07:19.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:19 smithi016 ceph-mon[43195]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-01-11T18:07:19.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:19 smithi038 ceph-mon[43582]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-01-11T18:07:21.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:21 smithi038 ceph-mon[43582]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 554 op/s 2024-01-11T18:07:21.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:21 smithi007 ceph-mon[33361]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 554 op/s 2024-01-11T18:07:21.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:21 smithi016 ceph-mon[43195]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 554 op/s 2024-01-11T18:07:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 535.0 s, 615.7 tps, lat 156.687 ms stddev 196.173 2024-01-11T18:07:24.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:23 smithi016 ceph-mon[43195]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-01-11T18:07:24.271 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:23 smithi038 ceph-mon[43582]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-01-11T18:07:24.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:23 smithi007 ceph-mon[33361]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-01-11T18:07:26.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:25 smithi007 ceph-mon[33361]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-01-11T18:07:26.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:25 smithi016 ceph-mon[43195]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-01-11T18:07:26.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:25 smithi038 ceph-mon[43582]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-01-11T18:07:28.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:27 smithi016 ceph-mon[43195]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 524 op/s 2024-01-11T18:07:28.277 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:27 smithi038 ceph-mon[43582]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 524 op/s 2024-01-11T18:07:28.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:27 smithi007 ceph-mon[33361]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 524 op/s 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 -- 172.21.15.7:0/513708891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 msgr2=0x7fd20c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 --2- 172.21.15.7:0/513708891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c102240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b0a0 tx=0x7fd20400b3b0 comp rx=0 tx=0).stop 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 -- 172.21.15.7:0/513708891 shutdown_connections 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 --2- 172.21.15.7:0/513708891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c105150 0x7fd20c1055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 --2- 172.21.15.7:0/513708891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c102240 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 --2- 172.21.15.7:0/513708891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.994+0000 7fd2133ff700 1 -- 172.21.15.7:0/513708891 >> 172.21.15.7:0/513708891 conn(0x7fd20c0fba30 msgr2=0x7fd20c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.995+0000 7fd2133ff700 1 -- 172.21.15.7:0/513708891 shutdown_connections 2024-01-11T18:07:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:28.995+0000 7fd2133ff700 1 -- 172.21.15.7:0/513708891 wait complete. 2024-01-11T18:07:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.000+0000 7fd2133ff700 1 Processor -- start 2024-01-11T18:07:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.000+0000 7fd2133ff700 1 -- start start 2024-01-11T18:07:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd2133ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd2133ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd2133ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c105150 0x7fd20c1a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd2133ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1a5750 con 0x7fd20c105150 2024-01-11T18:07:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd2133ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1a5890 con 0x7fd20c0ffe10 2024-01-11T18:07:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd2133ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1a5a20 con 0x7fd20c102780 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd21099a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd21119b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd21119b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46996/0 (socket says 172.21.15.7:46996) 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd21099a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c19e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50284/0 (socket says 172.21.15.7:50284) 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.001+0000 7fd21119b700 1 -- 172.21.15.7:0/1261211913 learned_addr learned my addr 172.21.15.7:0/1261211913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21199c700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c105150 0x7fd20c1a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21099a700 1 -- 172.21.15.7:0/1261211913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 msgr2=0x7fd20c19dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21099a700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c19dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21099a700 1 -- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c105150 msgr2=0x7fd20c1a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21099a700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c105150 0x7fd20c1a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21099a700 1 -- 172.21.15.7:0/1261211913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009520 con 0x7fd20c102780 2024-01-11T18:07:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21119b700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c19dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:07:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.002+0000 7fd21099a700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c19e160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c970 tx=0x7fd20800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.003+0000 7fd2027fc700 1 -- 172.21.15.7:0/1261211913 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208004440 con 0x7fd20c102780 2024-01-11T18:07:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.003+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd204009d00 con 0x7fd20c102780 2024-01-11T18:07:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.003+0000 7fd2027fc700 1 -- 172.21.15.7:0/1261211913 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd20800ce60 con 0x7fd20c102780 2024-01-11T18:07:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.003+0000 7fd2027fc700 1 -- 172.21.15.7:0/1261211913 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208003c70 con 0x7fd20c102780 2024-01-11T18:07:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.003+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20c1a60d0 con 0x7fd20c102780 2024-01-11T18:07:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.010+0000 7fd2027fc700 1 -- 172.21.15.7:0/1261211913 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fd208019450 con 0x7fd20c102780 2024-01-11T18:07:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.014+0000 7fd2027fc700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd1f80a1fd0 0x7fd1f80a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.015+0000 7fd227145b80 1 Processor -- start 2024-01-11T18:07:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.015+0000 7fd2027fc700 1 -- 172.21.15.7:0/1261211913 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd2080cd8d0 con 0x7fd20c102780 2024-01-11T18:07:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.015+0000 7fd21119b700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd1f80a1fd0 0x7fd1f80a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.016+0000 7fd227145b80 1 -- start start 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.016+0000 7fd227145b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 0x5557ab1dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd227145b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd227145b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd227145b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab0d3030 con 0x5557ab1dcc10 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd227145b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab136ac0 con 0x5557ab1dd5d0 2024-01-11T18:07:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd227145b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab1343e0 con 0x5557ab1dbf20 2024-01-11T18:07:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd21119b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 0x5557ab1dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd21119b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 0x5557ab1dc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50290/0 (socket says 172.21.15.7:50290) 2024-01-11T18:07:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd21119b700 1 -- 172.21.15.7:0/4240670660 learned_addr learned my addr 172.21.15.7:0/4240670660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.017+0000 7fd21199c700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21199c700 1 -- 172.21.15.7:0/4240670660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 msgr2=0x5557ab1dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21099a700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21199c700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 0x5557ab1dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21199c700 1 -- 172.21.15.7:0/4240670660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 msgr2=0x5557ab1dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21199c700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21199c700 1 -- 172.21.15.7:0/4240670660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557aafa4350 con 0x5557ab1dd5d0 2024-01-11T18:07:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21119b700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 0x5557ab1dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:07:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21099a700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:07:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.018+0000 7fd21119b700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd1f80a1fd0 0x7fd1f80a4490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd20400bb90 tx=0x7fd2040077c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.019+0000 7fd21199c700 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1e3e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009d10 tx=0x7fd1fc0116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb8ee2839570521 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.019+0000 7fd1f77fe700 1 -- 172.21.15.7:0/4240670660 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc011e20 con 0x5557ab1dd5d0 2024-01-11T18:07:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.019+0000 7fd1f77fe700 1 -- 172.21.15.7:0/4240670660 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1fc009e60 con 0x5557ab1dd5d0 2024-01-11T18:07:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.019+0000 7fd1f77fe700 1 -- 172.21.15.7:0/4240670660 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc012450 con 0x5557ab1dd5d0 2024-01-11T18:07:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.020+0000 7fd227145b80 1 -- 172.21.15.7:0/4240670660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 msgr2=0x5557ab1e3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.020+0000 7fd227145b80 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1e3e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009d10 tx=0x7fd1fc0116c0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 -- 172.21.15.7:0/4240670660 shutdown_connections 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dbf20 0x5557ab1dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1e3e60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 --2- 172.21.15.7:0/4240670660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 -- 172.21.15.7:0/4240670660 >> 172.21.15.7:0/4240670660 conn(0x5557ab151740 msgr2=0x5557ab1d6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 -- 172.21.15.7:0/4240670660 shutdown_connections 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.021+0000 7fd227145b80 1 -- 172.21.15.7:0/4240670660 wait complete. 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.022+0000 7fd227145b80 1 -- start start 2024-01-11T18:07:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.023+0000 7fd227145b80 1 Processor -- start 2024-01-11T18:07:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.024+0000 7fd227145b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab1f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.024+0000 7fd227145b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd227145b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd227145b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab12fcd0 con 0x5557ab1dbf20 2024-01-11T18:07:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd227145b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab09e340 con 0x5557ab1dd5d0 2024-01-11T18:07:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd227145b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab09dfe0 con 0x5557ab1dcc10 2024-01-11T18:07:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd227145b80 10 client.0 _ll_register_callbacks cb 0x5557ab0d06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1f6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50298/0 (socket says 172.21.15.7:50298) 2024-01-11T18:07:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 -- 172.21.15.7:0/2325911521 learned_addr learned my addr 172.21.15.7:0/2325911521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21199c700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21119b700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab1f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 -- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 msgr2=0x5557ab1fdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 -- 172.21.15.7:0/2325911521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 msgr2=0x5557ab1f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab1f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.025+0000 7fd21099a700 1 -- 172.21.15.7:0/2325911521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20c106100 con 0x5557ab1dcc10 2024-01-11T18:07:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.026+0000 7fd21099a700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1f6f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2080074a0 tx=0x7fd20809b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edad51c764f6801a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.026+0000 7fd1f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:07:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.027+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/2325911521 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208020500 con 0x5557ab1dcc10 2024-01-11T18:07:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.027+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/2325911521 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd20809f170 con 0x5557ab1dcc10 2024-01-11T18:07:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.027+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557ab1ced60 con 0x5557ab1dcc10 2024-01-11T18:07:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.031+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/2325911521 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2080a35d0 con 0x5557ab1dcc10 2024-01-11T18:07:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.031+0000 7fd227145b80 10 client.26302 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.031+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fd1fc0099c0 con 0x5557ab1dcc10 2024-01-11T18:07:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.032+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/2325911521 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557ab1ced60 con 0x5557ab1dcc10 2024-01-11T18:07:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.032+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/2325911521 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd2080a3140 con 0x5557ab1dcc10 2024-01-11T18:07:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.032+0000 7fd227145b80 10 client.26302 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.032+0000 7fd227145b80 10 client.26302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.032+0000 7fd227145b80 20 client.26302 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.032+0000 7fd227145b80 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab202760 0x5557ab204b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.033+0000 7fd227145b80 4 client.26302 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:07:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.033+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557ab128b60 con 0x5557ab202760 2024-01-11T18:07:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.033+0000 7fd21119b700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab202760 0x5557ab204b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.034+0000 7fd21119b700 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab202760 0x5557ab204b80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5e609054b391a6 server_cookie=17d9cff65a619adf in_seq=0 out_seq=0 2024-01-11T18:07:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.034+0000 7fd1f4ff9700 10 client.26302 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.035+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/2325911521 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557ab128b60 con 0x5557ab202760 2024-01-11T18:07:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.035+0000 7fd1f4ff9700 10 client.26302 handle_command_reply: tid=0 2024-01-11T18:07:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.035+0000 7fd227145b80 1 client.26302 shutdown 2024-01-11T18:07:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.036+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 msgr2=0x5557ab1f6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.036+0000 7fd227145b80 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1f6f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2080074a0 tx=0x7fd20809b9f0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 shutdown_connections 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab202760 0x5557ab204b80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab1f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd1f4ff9700 0 client.26302 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 --2- 172.21.15.7:0/2325911521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1f6f30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 >> 172.21.15.7:0/2325911521 conn(0x5557ab151740 msgr2=0x5557ab1d5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 shutdown_connections 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 1 -- 172.21.15.7:0/2325911521 wait complete. 2024-01-11T18:07:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.037+0000 7fd227145b80 20 client.26302 trim_cache size 0 max 16384 2024-01-11T18:07:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 Processor -- start 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 -- start start 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab204b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557aafa6640 con 0x5557ab1dbf20 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab1ff080 con 0x5557ab1dd5d0 2024-01-11T18:07:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.060+0000 7fd227145b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab2022d0 con 0x5557ab1dcc10 2024-01-11T18:07:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.062+0000 7fd21199c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.062+0000 7fd21199c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47030/0 (socket says 172.21.15.7:47030) 2024-01-11T18:07:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.062+0000 7fd21099a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.062+0000 7fd21099a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1fdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50310/0 (socket says 172.21.15.7:50310) 2024-01-11T18:07:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.062+0000 7fd21099a700 1 -- 172.21.15.7:0/387909094 learned_addr learned my addr 172.21.15.7:0/387909094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.063+0000 7fd21099a700 1 -- 172.21.15.7:0/387909094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 msgr2=0x5557ab1d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.063+0000 7fd21099a700 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.063+0000 7fd21099a700 1 -- 172.21.15.7:0/387909094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 msgr2=0x5557ab204b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.063+0000 7fd21099a700 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab204b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.063+0000 7fd21099a700 1 -- 172.21.15.7:0/387909094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557aaf50b00 con 0x5557ab1dcc10 2024-01-11T18:07:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.064+0000 7fd21099a700 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1fdec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd20809b1d0 tx=0x7fd20809f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1e91e64f7f3e000 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.064+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/387909094 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20809b2c0 con 0x5557ab1dcc10 2024-01-11T18:07:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.064+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/387909094 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd208097580 con 0x5557ab1dcc10 2024-01-11T18:07:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.064+0000 7fd1f6ffd700 1 -- 172.21.15.7:0/387909094 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20809a510 con 0x5557ab1dcc10 2024-01-11T18:07:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 -- 172.21.15.7:0/387909094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 msgr2=0x5557ab1fdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1fdec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd20809b1d0 tx=0x7fd20809f510 comp rx=0 tx=0).stop 2024-01-11T18:07:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 -- 172.21.15.7:0/387909094 shutdown_connections 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dd5d0 0x5557ab1d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dbf20 0x5557ab204b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 --2- 172.21.15.7:0/387909094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dcc10 0x5557ab1fdec0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 -- 172.21.15.7:0/387909094 >> 172.21.15.7:0/387909094 conn(0x5557ab151740 msgr2=0x5557ab1d6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.065+0000 7fd227145b80 1 -- 172.21.15.7:0/387909094 shutdown_connections 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.066+0000 7fd227145b80 1 -- 172.21.15.7:0/387909094 wait complete. 2024-01-11T18:07:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.066+0000 7fd227145b80 1 -- start start 2024-01-11T18:07:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 Processor -- start 2024-01-11T18:07:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 0x5557ab1fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab1fcc60 con 0x5557ab1dcc10 2024-01-11T18:07:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab1fcdd0 con 0x5557ab1dbf20 2024-01-11T18:07:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ab1fcf90 con 0x5557ab1dd5d0 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.067+0000 7fd227145b80 10 client.0 _ll_register_callbacks cb 0x5557ab1fead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 0x5557ab1fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 0x5557ab1fb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47034/0 (socket says 172.21.15.7:47034) 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 -- 172.21.15.7:0/806979087 learned_addr learned my addr 172.21.15.7:0/806979087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21199c700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21099a700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 -- 172.21.15.7:0/806979087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dd5d0 msgr2=0x5557ab1fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 -- 172.21.15.7:0/806979087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 msgr2=0x5557ab1fb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1fb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 -- 172.21.15.7:0/806979087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc0099c0 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd21119b700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 0x5557ab1fb1d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5557ab1dbcc0 tx=0x7fd2040099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276b84f66a1f3e7d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.068+0000 7fd1f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/806979087 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204026070 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1fc009e60 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/806979087 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd204009400 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/806979087 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd20400e040 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 10 client.26294 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5557ab0f11d0 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/806979087 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1fc009e60 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/806979087 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd20401f740 con 0x5557ab1dbf20 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 10 client.26294 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 20 client.26294 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab2592a0 0x5557ab25b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 4 client.26294 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.069+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557ab128880 con 0x5557ab2592a0 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.070+0000 7fd21099a700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab2592a0 0x5557ab25b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.071+0000 7fd21099a700 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab2592a0 0x5557ab25b6c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe7843261923f3ca server_cookie=6af1d1f1a856fa40 in_seq=0 out_seq=0 2024-01-11T18:07:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.071+0000 7fd1f4ff9700 10 client.26294 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.083+0000 7fd1f4ff9700 1 -- 172.21.15.7:0/806979087 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557ab128880 con 0x5557ab2592a0 2024-01-11T18:07:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.084+0000 7fd1f4ff9700 10 client.26294 handle_command_reply: tid=0 2024-01-11T18:07:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.084+0000 7fd227145b80 1 client.26294 shutdown 2024-01-11T18:07:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.084+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 msgr2=0x5557ab1fb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.084+0000 7fd227145b80 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 0x5557ab1fb1d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5557ab1dbcc0 tx=0x7fd2040099e0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 shutdown_connections 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd227145b80 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5557ab2592a0 0x5557ab25b6c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd227145b80 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557ab1dd5d0 0x5557ab1fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd227145b80 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557ab1dbf20 0x5557ab1fb1d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd227145b80 1 --2- 172.21.15.7:0/806979087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557ab1dcc10 0x5557ab1fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 >> 172.21.15.7:0/806979087 conn(0x5557ab151740 msgr2=0x5557ab2094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.085+0000 7fd1f4ff9700 0 client.26294 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.086+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 shutdown_connections 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.086+0000 7fd227145b80 1 -- 172.21.15.7:0/806979087 wait complete. 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.086+0000 7fd227145b80 20 client.26294 trim_cache size 0 max 16384 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:07:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.087+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd1f80a1fd0 msgr2=0x7fd1f80a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.087+0000 7fd2133ff700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd1f80a1fd0 0x7fd1f80a4490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd20400bb90 tx=0x7fd2040077c0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.087+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 msgr2=0x7fd20c19e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.087+0000 7fd2133ff700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c19e160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c970 tx=0x7fd20800cc80 comp rx=0 tx=0).stop 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.088+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 shutdown_connections 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.088+0000 7fd2133ff700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fd1f80a1fd0 0x7fd1f80a4490 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd20400bb90 tx=0x7fd2040077c0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.088+0000 7fd2133ff700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c105150 0x7fd20c1a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.088+0000 7fd2133ff700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20c0ffe10 0x7fd20c19dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.088+0000 7fd2133ff700 1 --2- 172.21.15.7:0/1261211913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd20c102780 0x7fd20c19e160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.088+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 >> 172.21.15.7:0/1261211913 conn(0x7fd20c0fba30 msgr2=0x7fd20c103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.089+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 shutdown_connections 2024-01-11T18:07:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:29.090+0000 7fd2133ff700 1 -- 172.21.15.7:0/1261211913 wait complete. 2024-01-11T18:07:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 540.0 s, 490.9 tps, lat 212.253 ms stddev 233.013 2024-01-11T18:07:29.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:29 smithi038 ceph-mon[43582]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 540 op/s 2024-01-11T18:07:29.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:29 smithi007 ceph-mon[33361]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 540 op/s 2024-01-11T18:07:29.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:29 smithi016 ceph-mon[43195]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 540 op/s 2024-01-11T18:07:32.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:31 smithi016 ceph-mon[43195]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 525 op/s 2024-01-11T18:07:32.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:31 smithi038 ceph-mon[43582]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 525 op/s 2024-01-11T18:07:32.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:31 smithi007 ceph-mon[33361]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 525 op/s 2024-01-11T18:07:34.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 545.0 s, 482.4 tps, lat 210.915 ms stddev 257.552 2024-01-11T18:07:34.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:33 smithi016 ceph-mon[43195]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 547 op/s 2024-01-11T18:07:34.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:33 smithi038 ceph-mon[43582]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 547 op/s 2024-01-11T18:07:34.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:33 smithi007 ceph-mon[33361]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 547 op/s 2024-01-11T18:07:36.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:35 smithi038 ceph-mon[43582]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 519 op/s 2024-01-11T18:07:36.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:35 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:07:36.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:35 smithi007 ceph-mon[33361]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 519 op/s 2024-01-11T18:07:36.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:35 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:07:36.240 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:35 smithi016 ceph-mon[43195]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 MiB/s wr, 519 op/s 2024-01-11T18:07:36.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:35 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:07:37.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:37 smithi007 ceph-mon[33361]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-01-11T18:07:37.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:37 smithi016 ceph-mon[43195]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-01-11T18:07:37.520 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:37 smithi038 ceph-mon[43582]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-01-11T18:07:38.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:07:38.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:07:38.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:07:38.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:07:38.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:07:38.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:07:38.522 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:07:38.522 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:07:38.522 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:07:39.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 550.0 s, 418.8 tps, lat 228.996 ms stddev 310.621 2024-01-11T18:07:39.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:39 smithi007 ceph-mon[33361]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 517 op/s 2024-01-11T18:07:39.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:39 smithi016 ceph-mon[43195]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 517 op/s 2024-01-11T18:07:39.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:39 smithi038 ceph-mon[43582]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 517 op/s 2024-01-11T18:07:42.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:41 smithi016 ceph-mon[43195]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.8 MiB/s wr, 508 op/s 2024-01-11T18:07:42.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:41 smithi038 ceph-mon[43582]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.8 MiB/s wr, 508 op/s 2024-01-11T18:07:42.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:41 smithi007 ceph-mon[33361]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.8 MiB/s wr, 508 op/s 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.378+0000 7f692a5ff700 1 -- 172.21.15.7:0/4217752612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 msgr2=0x7f69241096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.378+0000 7f692a5ff700 1 --2- 172.21.15.7:0/4217752612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69241096f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b0a0 tx=0x7f691c00b3b0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 -- 172.21.15.7:0/4217752612 shutdown_connections 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 --2- 172.21.15.7:0/4217752612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 --2- 172.21.15.7:0/4217752612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69241096f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 --2- 172.21.15.7:0/4217752612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 0x7f6924106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 -- 172.21.15.7:0/4217752612 >> 172.21.15.7:0/4217752612 conn(0x7f69240fba30 msgr2=0x7f69240fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 -- 172.21.15.7:0/4217752612 shutdown_connections 2024-01-11T18:07:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.379+0000 7f692a5ff700 1 -- 172.21.15.7:0/4217752612 wait complete. 2024-01-11T18:07:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.380+0000 7f692a5ff700 1 Processor -- start 2024-01-11T18:07:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.381+0000 7f692a5ff700 1 -- start start 2024-01-11T18:07:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.383+0000 7f692a5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.383+0000 7f692a5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 0x7f692406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f692a5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69240702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f692a5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69241a7700 con 0x7f6924104950 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f692a5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69241a7840 con 0x7f6924107280 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f692a5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69241a79d0 con 0x7f6924102020 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39310/0 (socket says 172.21.15.7:39310) 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f6923fff700 1 -- 172.21.15.7:0/2840608137 learned_addr learned my addr 172.21.15.7:0/2840608137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.384+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69240702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.385+0000 7f6923fff700 1 -- 172.21.15.7:0/2840608137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 msgr2=0x7f69240702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.385+0000 7f6923fff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69240702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.385+0000 7f6923fff700 1 -- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 msgr2=0x7f692406fcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T18:07:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.385+0000 7f69237fe700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 0x7f692406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.387+0000 7f6923fff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 0x7f692406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.387+0000 7f6923fff700 1 -- 172.21.15.7:0/2840608137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009d00 con 0x7f6924102020 2024-01-11T18:07:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.387+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69240702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:07:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.387+0000 7f69237fe700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 0x7f692406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:07:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.394+0000 7f6923fff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924073260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6914009f20 tx=0x7f691400ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.395+0000 7f69217fa700 1 -- 172.21.15.7:0/2840608137 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914014070 con 0x7f6924102020 2024-01-11T18:07:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.395+0000 7f69217fa700 1 -- 172.21.15.7:0/2840608137 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6914007880 con 0x7f6924102020 2024-01-11T18:07:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.395+0000 7f69217fa700 1 -- 172.21.15.7:0/2840608137 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400ecc0 con 0x7f6924102020 2024-01-11T18:07:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.395+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69240ffe70 con 0x7f6924102020 2024-01-11T18:07:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.395+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69241003d0 con 0x7f6924102020 2024-01-11T18:07:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.407+0000 7f69217fa700 1 -- 172.21.15.7:0/2840608137 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f69140079f0 con 0x7f6924102020 2024-01-11T18:07:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.408+0000 7f693e403b80 1 Processor -- start 2024-01-11T18:07:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.408+0000 7f693e403b80 1 -- start start 2024-01-11T18:07:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.409+0000 7f693e403b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1631ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.409+0000 7f693e403b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc16324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.409+0000 7f693e403b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1632f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.409+0000 7f693e403b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc1520030 con 0x559cc1631690 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.409+0000 7f693e403b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc1583ac0 con 0x559cc1632ad0 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.409+0000 7f693e403b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc15813e0 con 0x559cc1631ff0 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc16324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc16324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39320/0 (socket says 172.21.15.7:39320) 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6923fff700 1 -- 172.21.15.7:0/3891505320 learned_addr learned my addr 172.21.15.7:0/3891505320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6928b9c700 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1631ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f69217fa700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f690c0a1f50 0x7f690c0a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f69237fe700 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1632f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f69217fa700 1 -- 172.21.15.7:0/2840608137 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f69140d0e70 con 0x7f6924102020 2024-01-11T18:07:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6928b9c700 1 -- 172.21.15.7:0/3891505320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 msgr2=0x559cc16324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6928b9c700 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc16324f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.410+0000 7f6928b9c700 1 -- 172.21.15.7:0/3891505320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 msgr2=0x559cc1632f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.411+0000 7f6928b9c700 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1632f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.411+0000 7f6928b9c700 1 -- 172.21.15.7:0/3891505320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cc13f1350 con 0x559cc1631690 2024-01-11T18:07:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.411+0000 7f6928b9c700 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1631ab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b070 tx=0x7f691c008d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e33948980162e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.413+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f690c0a1f50 0x7f690c0a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.414+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f690c0a1f50 0x7f690c0a4410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f691c00a000 tx=0x7f691c029040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.414+0000 7f690a7fc700 1 -- 172.21.15.7:0/3891505320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00e050 con 0x559cc1631690 2024-01-11T18:07:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.415+0000 7f690a7fc700 1 -- 172.21.15.7:0/3891505320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f691c007960 con 0x559cc1631690 2024-01-11T18:07:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.415+0000 7f690a7fc700 1 -- 172.21.15.7:0/3891505320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c01dc90 con 0x559cc1631690 2024-01-11T18:07:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.416+0000 7f693e403b80 1 -- 172.21.15.7:0/3891505320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 msgr2=0x559cc1631ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.416+0000 7f693e403b80 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1631ab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b070 tx=0x7f691c008d40 comp rx=0 tx=0).stop 2024-01-11T18:07:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 -- 172.21.15.7:0/3891505320 shutdown_connections 2024-01-11T18:07:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1632f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1631ab0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 --2- 172.21.15.7:0/3891505320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc16324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 -- 172.21.15.7:0/3891505320 >> 172.21.15.7:0/3891505320 conn(0x559cc159e740 msgr2=0x559cc1623950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 -- 172.21.15.7:0/3891505320 shutdown_connections 2024-01-11T18:07:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.417+0000 7f693e403b80 1 -- 172.21.15.7:0/3891505320 wait complete. 2024-01-11T18:07:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.418+0000 7f693e403b80 1 -- start start 2024-01-11T18:07:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.418+0000 7f693e403b80 1 Processor -- start 2024-01-11T18:07:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1631ff0 0x559cc163b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1632ad0 0x559cc16426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc157ccd0 con 0x559cc1632ad0 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc14eb340 con 0x559cc1631ff0 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc14eafe0 con 0x559cc1631690 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f693e403b80 10 client.0 _ll_register_callbacks cb 0x559cc151d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39332/0 (socket says 172.21.15.7:39332) 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.419+0000 7f6923fff700 1 -- 172.21.15.7:0/1527925224 learned_addr learned my addr 172.21.15.7:0/1527925224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.420+0000 7f6923fff700 1 -- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1631ff0 msgr2=0x559cc163b660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:07:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.420+0000 7f6923fff700 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1631ff0 0x559cc163b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.420+0000 7f6923fff700 1 -- 172.21.15.7:0/1527925224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1632ad0 msgr2=0x559cc16426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.420+0000 7f6923fff700 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1632ad0 0x559cc16426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.420+0000 7f6923fff700 1 -- 172.21.15.7:0/1527925224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692410a2e0 con 0x559cc1631690 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.421+0000 7f6923fff700 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163b120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ef0 tx=0x7f691409d2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26407e760f5b9aa5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.421+0000 7f69037fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.421+0000 7f69037fe700 1 -- 172.21.15.7:0/1527925224 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400ee20 con 0x559cc1631690 2024-01-11T18:07:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.421+0000 7f69037fe700 1 -- 172.21.15.7:0/1527925224 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f691409d8a0 con 0x559cc1631690 2024-01-11T18:07:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.422+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cc161bd60 con 0x559cc1631690 2024-01-11T18:07:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.426+0000 7f69037fe700 1 -- 172.21.15.7:0/1527925224 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69140d2030 con 0x559cc1631690 2024-01-11T18:07:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.427+0000 7f693e403b80 10 client.26320 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.427+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x559cc139db00 con 0x559cc1631690 2024-01-11T18:07:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.428+0000 7f69037fe700 1 -- 172.21.15.7:0/1527925224 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69140a19f0 con 0x559cc1631690 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f69037fe700 1 -- 172.21.15.7:0/1527925224 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f691409db00 con 0x559cc1631690 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f693e403b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f693e403b80 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f693e403b80 20 client.26320 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f693e403b80 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc162d0d0 0x559cc162f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f693e403b80 4 client.26320 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.429+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cc161bd60 con 0x559cc162d0d0 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.430+0000 7f69237fe700 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc162d0d0 0x559cc162f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.430+0000 7f69037fe700 10 client.26320 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.430+0000 7f69237fe700 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc162d0d0 0x559cc162f570 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f8ccb102ab18d8 server_cookie=645bb6f7f1284d1 in_seq=0 out_seq=0 2024-01-11T18:07:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.431+0000 7f69037fe700 1 -- 172.21.15.7:0/1527925224 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559cc161bd60 con 0x559cc162d0d0 2024-01-11T18:07:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.431+0000 7f69037fe700 10 client.26320 handle_command_reply: tid=0 2024-01-11T18:07:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.432+0000 7f693e403b80 1 client.26320 shutdown 2024-01-11T18:07:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 msgr2=0x559cc163b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163b120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6914009ef0 tx=0x7f691409d2b0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 shutdown_connections 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc162d0d0 0x559cc162f570 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163b120 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1632ad0 0x559cc16426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 --2- 172.21.15.7:0/1527925224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1631ff0 0x559cc163b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 >> 172.21.15.7:0/1527925224 conn(0x559cc159e740 msgr2=0x559cc1622f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f69037fe700 0 client.26320 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 shutdown_connections 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 1 -- 172.21.15.7:0/1527925224 wait complete. 2024-01-11T18:07:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.433+0000 7f693e403b80 20 client.26320 trim_cache size 0 max 16384 2024-01-11T18:07:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.467+0000 7f693e403b80 1 Processor -- start 2024-01-11T18:07:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.467+0000 7f693e403b80 1 -- start start 2024-01-11T18:07:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.468+0000 7f693e403b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1628d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.468+0000 7f693e403b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc1642630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.468+0000 7f693e403b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc163a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.468+0000 7f693e403b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc13f3640 con 0x559cc1631690 2024-01-11T18:07:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.468+0000 7f693e403b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc1639bc0 con 0x559cc1632ad0 2024-01-11T18:07:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.468+0000 7f693e403b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc163c160 con 0x559cc1631ff0 2024-01-11T18:07:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.469+0000 7f6928b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc163a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.469+0000 7f69237fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc1642630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.469+0000 7f69237fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc1642630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39334/0 (socket says 172.21.15.7:39334) 2024-01-11T18:07:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.469+0000 7f6928b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc163a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42520/0 (socket says 172.21.15.7:42520) 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.469+0000 7f69237fe700 1 -- 172.21.15.7:0/2071351339 learned_addr learned my addr 172.21.15.7:0/2071351339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.469+0000 7f6923fff700 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1628d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.470+0000 7f6928b9c700 1 -- 172.21.15.7:0/2071351339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 msgr2=0x559cc1642630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.470+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc1642630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.470+0000 7f6928b9c700 1 -- 172.21.15.7:0/2071351339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 msgr2=0x559cc1628d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.470+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1628d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.470+0000 7f6928b9c700 1 -- 172.21.15.7:0/2071351339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cc153e1d0 con 0x559cc1632ad0 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.471+0000 7f6928b9c700 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc163a6a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b0a0 tx=0x7f691c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a709aba63c3905 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.471+0000 7f6923fff700 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1628d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.471+0000 7f6909ffb700 1 -- 172.21.15.7:0/2071351339 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c0090d0 con 0x559cc1632ad0 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.471+0000 7f6909ffb700 1 -- 172.21.15.7:0/2071351339 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f691c01d450 con 0x559cc1632ad0 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.471+0000 7f6909ffb700 1 -- 172.21.15.7:0/2071351339 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c037670 con 0x559cc1632ad0 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 -- 172.21.15.7:0/2071351339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 msgr2=0x559cc163a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc163a6a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b0a0 tx=0x7f691c0099e0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 -- 172.21.15.7:0/2071351339 shutdown_connections 2024-01-11T18:07:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc163a6a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631690 0x559cc1628d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 --2- 172.21.15.7:0/2071351339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631ff0 0x559cc1642630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.472+0000 7f693e403b80 1 -- 172.21.15.7:0/2071351339 >> 172.21.15.7:0/2071351339 conn(0x559cc159e740 msgr2=0x559cc1639070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.473+0000 7f693e403b80 1 -- 172.21.15.7:0/2071351339 shutdown_connections 2024-01-11T18:07:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.473+0000 7f693e403b80 1 -- 172.21.15.7:0/2071351339 wait complete. 2024-01-11T18:07:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.473+0000 7f693e403b80 1 -- start start 2024-01-11T18:07:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.474+0000 7f693e403b80 1 Processor -- start 2024-01-11T18:07:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.475+0000 7f693e403b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.475+0000 7f693e403b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631ff0 0x559cc163eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.475+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.475+0000 7f6923fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39350/0 (socket says 172.21.15.7:39350) 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.475+0000 7f693e403b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1641d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.475+0000 7f693e403b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc1642250 con 0x559cc1631ff0 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.476+0000 7f693e403b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc16423c0 con 0x559cc1632ad0 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.476+0000 7f693e403b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc1642580 con 0x559cc1631690 2024-01-11T18:07:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.476+0000 7f693e403b80 10 client.0 _ll_register_callbacks cb 0x559cc151d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.476+0000 7f6923fff700 1 -- 172.21.15.7:0/1181392317 learned_addr learned my addr 172.21.15.7:0/1181392317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f69237fe700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631ff0 0x559cc163eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6928b9c700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1641d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6923fff700 1 -- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 msgr2=0x559cc1641d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6923fff700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1641d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6923fff700 1 -- 172.21.15.7:0/1181392317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631ff0 msgr2=0x559cc163eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6923fff700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631ff0 0x559cc163eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6923fff700 1 -- 172.21.15.7:0/1181392317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cc13f1350 con 0x559cc1631690 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f6923fff700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163e860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f691409bbf0 tx=0x7f6914007ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d13e4b02dc679efd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.477+0000 7f69037fe700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:07:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.478+0000 7f69037fe700 1 -- 172.21.15.7:0/1181392317 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140d2070 con 0x559cc1631690 2024-01-11T18:07:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.478+0000 7f69037fe700 1 -- 172.21.15.7:0/1181392317 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f69140d3070 con 0x559cc1631690 2024-01-11T18:07:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.478+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f691c007960 con 0x559cc1631690 2024-01-11T18:07:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.480+0000 7f69037fe700 1 -- 172.21.15.7:0/1181392317 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6914004f30 con 0x559cc1631690 2024-01-11T18:07:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.480+0000 7f693e403b80 10 client.26326 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.480+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x559cc13f5c70 con 0x559cc1631690 2024-01-11T18:07:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.480+0000 7f69037fe700 1 -- 172.21.15.7:0/1181392317 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f691c007960 con 0x559cc1631690 2024-01-11T18:07:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.480+0000 7f69037fe700 1 -- 172.21.15.7:0/1181392317 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f691409db00 con 0x559cc1631690 2024-01-11T18:07:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f693e403b80 10 client.26326 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f693e403b80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f693e403b80 20 client.26326 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f693e403b80 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc16a6300 0x559cc16a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f693e403b80 4 client.26326 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:07:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559cc1575b60 con 0x559cc16a6300 2024-01-11T18:07:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.481+0000 7f69237fe700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc16a6300 0x559cc16a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.484+0000 7f69237fe700 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc16a6300 0x559cc16a8720 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad52a5fd64fca1d server_cookie=acd86ea937333656 in_seq=0 out_seq=0 2024-01-11T18:07:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.484+0000 7f69037fe700 10 client.26326 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.512+0000 7f69037fe700 1 -- 172.21.15.7:0/1181392317 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559cc1575b60 con 0x559cc16a6300 2024-01-11T18:07:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.512+0000 7f69037fe700 10 client.26326 handle_command_reply: tid=0 2024-01-11T18:07:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.512+0000 7f693e403b80 1 client.26326 shutdown 2024-01-11T18:07:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 msgr2=0x559cc163e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163e860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f691409bbf0 tx=0x7f6914007ee0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 shutdown_connections 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559cc16a6300 0x559cc16a8720 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559cc1631690 0x559cc163e860 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cc1632ad0 0x559cc1641d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 --2- 172.21.15.7:0/1181392317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cc1631ff0 0x559cc163eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 >> 172.21.15.7:0/1181392317 conn(0x559cc159e740 msgr2=0x559cc1638f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.513+0000 7f69037fe700 0 client.26326 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.514+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 shutdown_connections 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.514+0000 7f693e403b80 1 -- 172.21.15.7:0/1181392317 wait complete. 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.514+0000 7f693e403b80 20 client.26326 trim_cache size 0 max 16384 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:07:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:07:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.515+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f690c0a1f50 msgr2=0x7f690c0a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f690c0a1f50 0x7f690c0a4410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f691c00a000 tx=0x7f691c029040 comp rx=0 tx=0).stop 2024-01-11T18:07:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 msgr2=0x7f6924073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924073260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6914009f20 tx=0x7f691400ff50 comp rx=0 tx=0).stop 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 shutdown_connections 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f690c0a1f50 0x7f690c0a4410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6924102020 0x7f6924073260 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924107280 0x7f69240702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 --2- 172.21.15.7:0/2840608137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6924104950 0x7f692406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.516+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 >> 172.21.15.7:0/2840608137 conn(0x7f69240fba30 msgr2=0x7f69241028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.517+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 shutdown_connections 2024-01-11T18:07:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:42.517+0000 7f692a5ff700 1 -- 172.21.15.7:0/2840608137 wait complete. 2024-01-11T18:07:43.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:42 smithi016 ceph-mon[43195]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2024-01-11T18:07:43.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:42 smithi038 ceph-mon[43582]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2024-01-11T18:07:43.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:42 smithi007 ceph-mon[33361]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2024-01-11T18:07:44.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 555.0 s, 564.4 tps, lat 184.554 ms stddev 228.826 2024-01-11T18:07:46.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:45 smithi007 ceph-mon[33361]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 445 op/s 2024-01-11T18:07:46.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:45 smithi016 ceph-mon[43195]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 445 op/s 2024-01-11T18:07:46.273 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:45 smithi038 ceph-mon[43582]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 445 op/s 2024-01-11T18:07:47.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:47 smithi007 ceph-mon[33361]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-01-11T18:07:47.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:47 smithi016 ceph-mon[43195]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-01-11T18:07:47.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:47 smithi038 ceph-mon[43582]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-01-11T18:07:49.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 560.0 s, 475.6 tps, lat 205.589 ms stddev 232.692 2024-01-11T18:07:49.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:49 smithi007 ceph-mon[33361]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-01-11T18:07:49.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:49 smithi016 ceph-mon[43195]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-01-11T18:07:49.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:49 smithi038 ceph-mon[43582]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-01-11T18:07:51.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:51 smithi007 ceph-mon[33361]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-01-11T18:07:51.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:51 smithi016 ceph-mon[43195]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-01-11T18:07:51.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:51 smithi038 ceph-mon[43582]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-01-11T18:07:53.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:53 smithi007 ceph-mon[33361]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 533 op/s 2024-01-11T18:07:53.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:53 smithi016 ceph-mon[43195]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 533 op/s 2024-01-11T18:07:53.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:53 smithi038 ceph-mon[43582]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 533 op/s 2024-01-11T18:07:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 565.0 s, 485.8 tps, lat 204.348 ms stddev 259.626 2024-01-11T18:07:56.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:55 smithi007 ceph-mon[33361]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 MiB/s wr, 441 op/s 2024-01-11T18:07:56.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:55 smithi016 ceph-mon[43195]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 MiB/s wr, 441 op/s 2024-01-11T18:07:56.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:55 smithi038 ceph-mon[43582]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 MiB/s wr, 441 op/s 2024-01-11T18:07:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 -- 172.21.15.7:0/1148079897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6970096890 msgr2=0x7f6970098c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 --2- 172.21.15.7:0/1148079897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6970096890 0x7f6970098c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6978067030 tx=0x7f69780503f0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 -- 172.21.15.7:0/1148079897 shutdown_connections 2024-01-11T18:07:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 --2- 172.21.15.7:0/1148079897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697009baf0 0x7f697009df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 --2- 172.21.15.7:0/1148079897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6970096890 0x7f6970098c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 --2- 172.21.15.7:0/1148079897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69700991c0 0x7f697009b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.606+0000 7f697e19d700 1 -- 172.21.15.7:0/1148079897 >> 172.21.15.7:0/1148079897 conn(0x7f6970090260 msgr2=0x7f69700926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.607+0000 7f697e19d700 1 -- 172.21.15.7:0/1148079897 shutdown_connections 2024-01-11T18:07:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.607+0000 7f697e19d700 1 -- 172.21.15.7:0/1148079897 wait complete. 2024-01-11T18:07:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.609+0000 7f697e19d700 1 Processor -- start 2024-01-11T18:07:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.609+0000 7f697e19d700 1 -- start start 2024-01-11T18:07:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.610+0000 7f697e19d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6970096890 0x7f697013cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697e19d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 0x7f697013d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697e19d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697009baf0 0x7f69701442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697c99a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 0x7f697013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697e19d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970144870 con 0x7f697009baf0 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697e19d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69701449e0 con 0x7f69700991c0 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697e19d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970144ba0 con 0x7f6970096890 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697d19b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6970096890 0x7f697013cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697c99a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 0x7f697013d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39582/0 (socket says 172.21.15.7:39582) 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.611+0000 7f697c99a700 1 -- 172.21.15.7:0/1557925110 learned_addr learned my addr 172.21.15.7:0/1557925110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.612+0000 7f697d99c700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697009baf0 0x7f69701442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.612+0000 7f697c99a700 1 -- 172.21.15.7:0/1557925110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6970096890 msgr2=0x7f697013cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.612+0000 7f697c99a700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6970096890 0x7f697013cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.612+0000 7f697c99a700 1 -- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697009baf0 msgr2=0x7f69701442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.612+0000 7f697c99a700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697009baf0 0x7f69701442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.612+0000 7f697c99a700 1 -- 172.21.15.7:0/1557925110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978050260 con 0x7f69700991c0 2024-01-11T18:07:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.613+0000 7f697c99a700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 0x7f697013d270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f696800d980 tx=0x7f696800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.613+0000 7f696e7fc700 1 -- 172.21.15.7:0/1557925110 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800bb60 con 0x7f69700991c0 2024-01-11T18:07:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.613+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6970144e80 con 0x7f69700991c0 2024-01-11T18:07:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.613+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6970145300 con 0x7f69700991c0 2024-01-11T18:07:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.617+0000 7f696e7fc700 1 -- 172.21.15.7:0/1557925110 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6968007640 con 0x7f69700991c0 2024-01-11T18:07:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.617+0000 7f696e7fc700 1 -- 172.21.15.7:0/1557925110 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968004e90 con 0x7f69700991c0 2024-01-11T18:07:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.625+0000 7f6993175b80 1 Processor -- start 2024-01-11T18:07:56.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.625+0000 7f696e7fc700 1 -- 172.21.15.7:0/1557925110 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f696800bcc0 con 0x7f69700991c0 2024-01-11T18:07:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.626+0000 7f6993175b80 1 -- start start 2024-01-11T18:07:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.627+0000 7f6993175b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x555750a008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.627+0000 7f6993175b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00df0 0x555750a012f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.627+0000 7f6993175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.627+0000 7f6993175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557508f3030 con 0x555750a01880 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.627+0000 7f6993175b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555750956ac0 con 0x555750a00df0 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.627+0000 7f6993175b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557509543e0 con 0x555750a00490 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.628+0000 7f697d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x555750a008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.628+0000 7f697c99a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.628+0000 7f697d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x555750a008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42104/0 (socket says 172.21.15.7:42104) 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.628+0000 7f697d99c700 1 -- 172.21.15.7:0/2628037982 learned_addr learned my addr 172.21.15.7:0/2628037982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.628+0000 7f697d19b700 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00df0 0x555750a012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.629+0000 7f696e7fc700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f69640a1f60 0x7f69640a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.629+0000 7f697d19b700 1 -- 172.21.15.7:0/2628037982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 msgr2=0x555750a008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.629+0000 7f697d19b700 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x555750a008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.629+0000 7f697d19b700 1 -- 172.21.15.7:0/2628037982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 msgr2=0x555750a01ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.629+0000 7f697d19b700 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a01ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.630+0000 7f696e7fc700 1 -- 172.21.15.7:0/1557925110 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f69680cda80 con 0x7f69700991c0 2024-01-11T18:07:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.629+0000 7f697d19b700 1 -- 172.21.15.7:0/2628037982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557507c4350 con 0x555750a00df0 2024-01-11T18:07:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.631+0000 7f697d19b700 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00df0 0x555750a012f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f697013e200 tx=0x7f697807b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c673ac000e77140d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.631+0000 7f697d99c700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f69640a1f60 0x7f69640a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.631+0000 7f69637fe700 1 -- 172.21.15.7:0/2628037982 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697807b400 con 0x555750a00df0 2024-01-11T18:07:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.632+0000 7f69637fe700 1 -- 172.21.15.7:0/2628037982 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697807bcd0 con 0x555750a00df0 2024-01-11T18:07:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.632+0000 7f69637fe700 1 -- 172.21.15.7:0/2628037982 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978072440 con 0x555750a00df0 2024-01-11T18:07:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.633+0000 7f6993175b80 1 -- 172.21.15.7:0/2628037982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00df0 msgr2=0x555750a012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.633+0000 7f697d99c700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f69640a1f60 0x7f69640a4420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f697400dcb0 tx=0x7f6974004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.633+0000 7f6993175b80 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00df0 0x555750a012f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f697013e200 tx=0x7f697807b670 comp rx=0 tx=0).stop 2024-01-11T18:07:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.635+0000 7f6993175b80 1 -- 172.21.15.7:0/2628037982 shutdown_connections 2024-01-11T18:07:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.635+0000 7f6993175b80 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x555750a008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.635+0000 7f6993175b80 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a01ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.636+0000 7f6993175b80 1 --2- 172.21.15.7:0/2628037982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00df0 0x555750a012f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.636+0000 7f6993175b80 1 -- 172.21.15.7:0/2628037982 >> 172.21.15.7:0/2628037982 conn(0x555750971740 msgr2=0x5557509f6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.636+0000 7f6993175b80 1 -- 172.21.15.7:0/2628037982 shutdown_connections 2024-01-11T18:07:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.636+0000 7f6993175b80 1 -- 172.21.15.7:0/2628037982 wait complete. 2024-01-11T18:07:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.637+0000 7f6993175b80 1 -- start start 2024-01-11T18:07:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.637+0000 7f6993175b80 1 Processor -- start 2024-01-11T18:07:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.638+0000 7f6993175b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x5557509fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.638+0000 7f6993175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x5557509fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.638+0000 7f6993175b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 0x5557509fe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f697d19b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x5557509fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f697d19b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x5557509fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42106/0 (socket says 172.21.15.7:42106) 2024-01-11T18:07:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f6993175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575094fcd0 con 0x555750a00df0 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f6993175b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557508be340 con 0x555750a01880 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f6993175b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557508bdfe0 con 0x555750a00490 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f6993175b80 10 client.0 _ll_register_callbacks cb 0x5557508f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f697d99c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 0x5557509fe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f697d99c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 0x5557509fe3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39598/0 (socket says 172.21.15.7:39598) 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.639+0000 7f697d99c700 1 -- 172.21.15.7:0/1328614402 learned_addr learned my addr 172.21.15.7:0/1328614402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f697d99c700 1 -- 172.21.15.7:0/1328614402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 msgr2=0x5557509fced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f697d99c700 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x5557509fced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f697d99c700 1 -- 172.21.15.7:0/1328614402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 msgr2=0x5557509fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f697d99c700 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x5557509fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f697d99c700 1 -- 172.21.15.7:0/1328614402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697009eb50 con 0x555750a01880 2024-01-11T18:07:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f6960ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:07:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.640+0000 7f697d99c700 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 0x5557509fe3d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6974011e60 tx=0x7f69740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90149b5031aae76 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.641+0000 7f6960ff9700 1 -- 172.21.15.7:0/1328614402 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697401d650 con 0x555750a01880 2024-01-11T18:07:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.641+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557509eefb0 con 0x555750a01880 2024-01-11T18:07:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.641+0000 7f6960ff9700 1 -- 172.21.15.7:0/1328614402 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f697401d7b0 con 0x555750a01880 2024-01-11T18:07:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.641+0000 7f6960ff9700 1 -- 172.21.15.7:0/1328614402 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f697401d650 con 0x555750a01880 2024-01-11T18:07:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.641+0000 7f6993175b80 10 client.26318 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.641+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555750770b00 con 0x555750a01880 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6960ff9700 1 -- 172.21.15.7:0/1328614402 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557509eefb0 con 0x555750a01880 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6960ff9700 1 -- 172.21.15.7:0/1328614402 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f697401ea90 con 0x555750a01880 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6993175b80 10 client.26318 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6993175b80 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6993175b80 20 client.26318 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6993175b80 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a1a0d0 0x555750a1c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6993175b80 4 client.26318 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555750948b60 con 0x555750a1a0d0 2024-01-11T18:07:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.642+0000 7f697d19b700 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a1a0d0 0x555750a1c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.643+0000 7f697d19b700 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a1a0d0 0x555750a1c4f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616475b50f475177 server_cookie=ee1766764c36081d in_seq=0 out_seq=0 2024-01-11T18:07:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.643+0000 7f6960ff9700 10 client.26318 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.644+0000 7f6960ff9700 1 -- 172.21.15.7:0/1328614402 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555750948b60 con 0x555750a1a0d0 2024-01-11T18:07:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.644+0000 7f6960ff9700 10 client.26318 handle_command_reply: tid=0 2024-01-11T18:07:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.644+0000 7f6993175b80 1 client.26318 shutdown 2024-01-11T18:07:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.644+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 msgr2=0x5557509fe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.644+0000 7f6993175b80 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 0x5557509fe3d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6974011e60 tx=0x7f69740047c0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 shutdown_connections 2024-01-11T18:07:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a1a0d0 0x555750a1c4f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00490 0x5557509fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a01880 0x5557509fe3d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 --2- 172.21.15.7:0/1328614402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x5557509fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 >> 172.21.15.7:0/1328614402 conn(0x555750971740 msgr2=0x5557509f61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6960ff9700 0 client.26318 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 shutdown_connections 2024-01-11T18:07:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.646+0000 7f6993175b80 1 -- 172.21.15.7:0/1328614402 wait complete. 2024-01-11T18:07:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.647+0000 7f6993175b80 20 client.26318 trim_cache size 0 max 16384 2024-01-11T18:07:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.685+0000 7f6993175b80 1 Processor -- start 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.685+0000 7f6993175b80 1 -- start start 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.686+0000 7f6993175b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x5557509fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.686+0000 7f6993175b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 0x555750a011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.686+0000 7f6993175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.686+0000 7f6993175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557507c6640 con 0x555750a01880 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.686+0000 7f6993175b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555750a084b0 con 0x555750a00490 2024-01-11T18:07:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.686+0000 7f6993175b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557509f88f0 con 0x555750a00df0 2024-01-11T18:07:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697d19b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x5557509fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697d19b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x5557509fea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39612/0 (socket says 172.21.15.7:39612) 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 0x555750a011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 0x555750a011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42120/0 (socket says 172.21.15.7:42120) 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 -- 172.21.15.7:0/1589391686 learned_addr learned my addr 172.21.15.7:0/1589391686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697d99c700 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 -- 172.21.15.7:0/1589391686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 msgr2=0x5557509fea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x5557509fea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 -- 172.21.15.7:0/1589391686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 msgr2=0x555750a20e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a20e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.687+0000 7f697c99a700 1 -- 172.21.15.7:0/1589391686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557509111d0 con 0x555750a00df0 2024-01-11T18:07:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.688+0000 7f697c99a700 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 0x555750a011d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f696800dfd0 tx=0x7f69680d1050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37f7b13fed88b46 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.688+0000 7f6962ffd700 1 -- 172.21.15.7:0/1589391686 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69680d6040 con 0x555750a00df0 2024-01-11T18:07:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.688+0000 7f6962ffd700 1 -- 172.21.15.7:0/1589391686 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69680d7040 con 0x555750a00df0 2024-01-11T18:07:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.688+0000 7f6962ffd700 1 -- 172.21.15.7:0/1589391686 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696809e670 con 0x555750a00df0 2024-01-11T18:07:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 -- 172.21.15.7:0/1589391686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 msgr2=0x555750a011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 0x555750a011d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f696800dfd0 tx=0x7f69680d1050 comp rx=0 tx=0).stop 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 -- 172.21.15.7:0/1589391686 shutdown_connections 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a01880 0x555750a20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x5557509fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 --2- 172.21.15.7:0/1589391686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a00df0 0x555750a011d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.689+0000 7f6993175b80 1 -- 172.21.15.7:0/1589391686 >> 172.21.15.7:0/1589391686 conn(0x555750971740 msgr2=0x5557509f2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.690+0000 7f6993175b80 1 -- 172.21.15.7:0/1589391686 shutdown_connections 2024-01-11T18:07:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.690+0000 7f6993175b80 1 -- 172.21.15.7:0/1589391686 wait complete. 2024-01-11T18:07:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.690+0000 7f6993175b80 1 -- start start 2024-01-11T18:07:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.691+0000 7f6993175b80 1 Processor -- start 2024-01-11T18:07:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f6993175b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x555750a06e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f6993175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x555750a6c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f6993175b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 0x555750a6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f6993175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555750a6f2d0 con 0x555750a00df0 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f6993175b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555750a6f440 con 0x555750a00490 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f697d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 0x555750a6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f697d99c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 0x555750a6ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:42132/0 (socket says 172.21.15.7:42132) 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.692+0000 7f697d99c700 1 -- 172.21.15.7:0/2259697674 learned_addr learned my addr 172.21.15.7:0/2259697674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:07:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555750a6f600 con 0x555750a01880 2024-01-11T18:07:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f6993175b80 10 client.0 _ll_register_callbacks cb 0x5557508f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:07:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f697c99a700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x555750a6c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f697d19b700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x555750a06e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f697d99c700 1 -- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 msgr2=0x555750a06e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f697d99c700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x555750a06e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f697d99c700 1 -- 172.21.15.7:0/2259697674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 msgr2=0x555750a6c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:07:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.693+0000 7f697d99c700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x555750a6c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.694+0000 7f697d99c700 1 -- 172.21.15.7:0/2259697674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6970145300 con 0x555750a01880 2024-01-11T18:07:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.694+0000 7f697d99c700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 0x555750a6ebc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f697401ef40 tx=0x7f6974012a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b52e2d218073f738 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:07:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.694+0000 7f697d19b700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x555750a06e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:07:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.694+0000 7f6960ff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:07:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.695+0000 7f6960ff9700 1 -- 172.21.15.7:0/2259697674 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974036070 con 0x555750a01880 2024-01-11T18:07:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.695+0000 7f6960ff9700 1 -- 172.21.15.7:0/2259697674 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f697401e760 con 0x555750a01880 2024-01-11T18:07:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.695+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f697807bcd0 con 0x555750a01880 2024-01-11T18:07:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.697+0000 7f6960ff9700 1 -- 172.21.15.7:0/2259697674 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f697401e920 con 0x555750a01880 2024-01-11T18:07:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.697+0000 7f6993175b80 10 client.26341 fetch_fsmap learned FSMap version 43 2024-01-11T18:07:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.697+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5557507c8c70 con 0x555750a01880 2024-01-11T18:07:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6960ff9700 1 -- 172.21.15.7:0/2259697674 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697807bcd0 con 0x555750a01880 2024-01-11T18:07:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6960ff9700 1 -- 172.21.15.7:0/2259697674 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 43) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f697401b530 con 0x555750a01880 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6993175b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 43 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6993175b80 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6993175b80 20 client.26341 populate_metadata read hostname 'smithi007' 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6993175b80 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a71040 0x555750a73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6993175b80 4 client.26341 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.698+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555750948880 con 0x555750a71040 2024-01-11T18:07:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.699+0000 7f697d19b700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a71040 0x555750a73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:07:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.700+0000 7f697d19b700 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a71040 0x555750a73460 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1383fcae2efea3 server_cookie=aeb997b1fa48afde in_seq=0 out_seq=0 2024-01-11T18:07:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.700+0000 7f6960ff9700 10 client.26341 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.742+0000 7f6960ff9700 1 -- 172.21.15.7:0/2259697674 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555750948880 con 0x555750a71040 2024-01-11T18:07:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.742+0000 7f6960ff9700 10 client.26341 handle_command_reply: tid=0 2024-01-11T18:07:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.742+0000 7f6993175b80 1 client.26341 shutdown 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 msgr2=0x555750a6ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 0x555750a6ebc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f697401ef40 tx=0x7f6974012a60 comp rx=0 tx=0).stop 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 shutdown_connections 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555750a71040 0x555750a73460 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555750a01880 0x555750a6ebc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555750a00490 0x555750a06e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6960ff9700 0 client.26341 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 --2- 172.21.15.7:0/2259697674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555750a00df0 0x555750a6c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 >> 172.21.15.7:0/2259697674 conn(0x555750971740 msgr2=0x555750a20e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 shutdown_connections 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.743+0000 7f6993175b80 1 -- 172.21.15.7:0/2259697674 wait complete. 2024-01-11T18:07:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.744+0000 7f6993175b80 20 client.26341 trim_cache size 0 max 16384 2024-01-11T18:07:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:07:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:07:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:07:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f69640a1f60 msgr2=0x7f69640a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f69640a1f60 0x7f69640a4420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f697400dcb0 tx=0x7f6974004750 comp rx=0 tx=0).stop 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 msgr2=0x7f697013d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 0x7f697013d270 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f696800d980 tx=0x7f696800dd40 comp rx=0 tx=0).stop 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 shutdown_connections 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f69640a1f60 0x7f69640a4420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6970096890 0x7f697013cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697009baf0 0x7f69701442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 --2- 172.21.15.7:0/1557925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69700991c0 0x7f697013d270 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 >> 172.21.15.7:0/1557925110 conn(0x7f6970090260 msgr2=0x7f69700971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:07:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 shutdown_connections 2024-01-11T18:07:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:07:56.745+0000 7f697e19d700 1 -- 172.21.15.7:0/1557925110 wait complete. 2024-01-11T18:07:57.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:57 smithi007 ceph-mon[33361]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 MiB/s wr, 477 op/s 2024-01-11T18:07:57.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:57 smithi016 ceph-mon[43195]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 MiB/s wr, 477 op/s 2024-01-11T18:07:57.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:57 smithi038 ceph-mon[43582]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 MiB/s wr, 477 op/s 2024-01-11T18:07:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 570.0 s, 443.8 tps, lat 223.453 ms stddev 330.442 2024-01-11T18:07:59.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:07:59 smithi007 ceph-mon[33361]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 MiB/s wr, 497 op/s 2024-01-11T18:07:59.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:07:59 smithi016 ceph-mon[43195]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 MiB/s wr, 497 op/s 2024-01-11T18:07:59.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:07:59 smithi038 ceph-mon[43582]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 MiB/s wr, 497 op/s 2024-01-11T18:08:01.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:01 smithi007 ceph-mon[33361]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.1 MiB/s wr, 481 op/s 2024-01-11T18:08:01.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:01 smithi016 ceph-mon[43195]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.1 MiB/s wr, 481 op/s 2024-01-11T18:08:01.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:01 smithi038 ceph-mon[43582]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.1 MiB/s wr, 481 op/s 2024-01-11T18:08:03.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:03 smithi007 ceph-mon[33361]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-01-11T18:08:03.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:03 smithi016 ceph-mon[43195]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-01-11T18:08:03.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:03 smithi038 ceph-mon[43582]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-01-11T18:08:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 575.0 s, 683.8 tps, lat 149.807 ms stddev 201.536 2024-01-11T18:08:06.027 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:05 smithi038 ceph-mon[43582]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-01-11T18:08:06.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:05 smithi007 ceph-mon[33361]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-01-11T18:08:06.241 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:05 smithi016 ceph-mon[43195]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.3 MiB/s wr, 417 op/s 2024-01-11T18:08:07.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:07 smithi007 ceph-mon[33361]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-01-11T18:08:07.490 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:07 smithi016 ceph-mon[43195]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-01-11T18:08:07.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:07 smithi038 ceph-mon[43582]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 480 op/s 2024-01-11T18:08:09.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 580.0 s, 466.4 tps, lat 205.530 ms stddev 242.834 2024-01-11T18:08:09.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:09 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:08:09.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:09 smithi016 ceph-mon[43195]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 521 op/s 2024-01-11T18:08:09.770 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:09 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:08:09.771 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:09 smithi038 ceph-mon[43582]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 521 op/s 2024-01-11T18:08:09.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:09 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:08:09.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:09 smithi007 ceph-mon[33361]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 521 op/s 2024-01-11T18:08:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.009+0000 7ff06b3ff700 1 -- 172.21.15.7:0/681140656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 msgr2=0x7ff064100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.009+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/681140656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff064100270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00b0a0 tx=0x7ff05c00b3b0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.010+0000 7ff06b3ff700 1 -- 172.21.15.7:0/681140656 shutdown_connections 2024-01-11T18:08:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.010+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/681140656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff064101170 0x7ff0641015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.010+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/681140656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff064100270 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.010+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/681140656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0641007b0 0x7ff064100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.010+0000 7ff06b3ff700 1 -- 172.21.15.7:0/681140656 >> 172.21.15.7:0/681140656 conn(0x7ff0640fba30 msgr2=0x7ff0640fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.012+0000 7ff06b3ff700 1 -- 172.21.15.7:0/681140656 shutdown_connections 2024-01-11T18:08:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.012+0000 7ff06b3ff700 1 -- 172.21.15.7:0/681140656 wait complete. 2024-01-11T18:08:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.014+0000 7ff06b3ff700 1 Processor -- start 2024-01-11T18:08:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.014+0000 7ff06b3ff700 1 -- start start 2024-01-11T18:08:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.016+0000 7ff06b3ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff06406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.016+0000 7ff06b3ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0641007b0 0x7ff0640704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.016+0000 7ff06b3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101170 0x7ff0641a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.016+0000 7ff06b3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff064070ab0 con 0x7ff064101170 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06b3ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff064070c20 con 0x7ff0640ffe50 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06899a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0641007b0 0x7ff0640704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06899a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0641007b0 0x7ff0640704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50840/0 (socket says 172.21.15.7:50840) 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06899a700 1 -- 172.21.15.7:0/4128315059 learned_addr learned my addr 172.21.15.7:0/4128315059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06919b700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff06406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06999c700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101170 0x7ff0641a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.017+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641a9ec0 con 0x7ff0641007b0 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.019+0000 7ff06919b700 1 -- 172.21.15.7:0/4128315059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0641007b0 msgr2=0x7ff0640704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.019+0000 7ff06919b700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0641007b0 0x7ff0640704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.019+0000 7ff06919b700 1 -- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101170 msgr2=0x7ff0641a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.019+0000 7ff06919b700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101170 0x7ff0641a9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.019+0000 7ff06919b700 1 -- 172.21.15.7:0/4128315059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c009d00 con 0x7ff0640ffe50 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.019+0000 7ff06999c700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101170 0x7ff0641a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:08:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.020+0000 7ff06919b700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff06406ffa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff05c000f80 tx=0x7ff05c00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.023+0000 7ff05a7fc700 1 -- 172.21.15.7:0/4128315059 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00e040 con 0x7ff0640ffe50 2024-01-11T18:08:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.023+0000 7ff05a7fc700 1 -- 172.21.15.7:0/4128315059 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff05c003e10 con 0x7ff0640ffe50 2024-01-11T18:08:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.023+0000 7ff05a7fc700 1 -- 172.21.15.7:0/4128315059 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c01b750 con 0x7ff0640ffe50 2024-01-11T18:08:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.023+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0641aa140 con 0x7ff0640ffe50 2024-01-11T18:08:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.023+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0641aa5f0 con 0x7ff0640ffe50 2024-01-11T18:08:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.033+0000 7ff07f04bb80 1 Processor -- start 2024-01-11T18:08:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.033+0000 7ff07f04bb80 1 -- start start 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f65d8030 con 0x5565f66e9690 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f663bac0 con 0x5565f66e9ff0 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f66393e0 con 0x5565f66eaad0 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff06899a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff06899a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66eaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50846/0 (socket says 172.21.15.7:50846) 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.034+0000 7ff06899a700 1 -- 172.21.15.7:0/379842194 learned_addr learned my addr 172.21.15.7:0/379842194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06999c700 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06899a700 1 -- 172.21.15.7:0/379842194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 msgr2=0x5565f66ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06899a700 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06899a700 1 -- 172.21.15.7:0/379842194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 msgr2=0x5565f66e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06899a700 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06899a700 1 -- 172.21.15.7:0/379842194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f64a9350 con 0x5565f66eaad0 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.035+0000 7ff06999c700 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:08:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.036+0000 7ff06899a700 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66eaf50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff060009e50 tx=0x7ff060011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b024c5433b95103c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.040+0000 7ff04f7fe700 1 -- 172.21.15.7:0/379842194 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060004a30 con 0x5565f66eaad0 2024-01-11T18:08:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.040+0000 7ff04f7fe700 1 -- 172.21.15.7:0/379842194 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0600054e0 con 0x5565f66eaad0 2024-01-11T18:08:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.040+0000 7ff04f7fe700 1 -- 172.21.15.7:0/379842194 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060019930 con 0x5565f66eaad0 2024-01-11T18:08:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.041+0000 7ff07f04bb80 1 -- 172.21.15.7:0/379842194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 msgr2=0x5565f66eaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.041+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66eaf50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff060009e50 tx=0x7ff060011720 comp rx=0 tx=0).stop 2024-01-11T18:08:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.042+0000 7ff07f04bb80 1 -- 172.21.15.7:0/379842194 shutdown_connections 2024-01-11T18:08:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.042+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66eaf50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.042+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.042+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/379842194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.042+0000 7ff07f04bb80 1 -- 172.21.15.7:0/379842194 >> 172.21.15.7:0/379842194 conn(0x5565f6656740 msgr2=0x5565f66db950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.043+0000 7ff05a7fc700 1 -- 172.21.15.7:0/4128315059 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7ff05c01da30 con 0x7ff0640ffe50 2024-01-11T18:08:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.044+0000 7ff07f04bb80 1 -- 172.21.15.7:0/379842194 shutdown_connections 2024-01-11T18:08:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.044+0000 7ff07f04bb80 1 -- 172.21.15.7:0/379842194 wait complete. 2024-01-11T18:08:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.044+0000 7ff07f04bb80 1 -- start start 2024-01-11T18:08:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.045+0000 7ff07f04bb80 1 Processor -- start 2024-01-11T18:08:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.047+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.047+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.047+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.047+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6634cd0 con 0x5565f66e9690 2024-01-11T18:08:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.047+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f65a3340 con 0x5565f66e9ff0 2024-01-11T18:08:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.047+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f65a2fe0 con 0x5565f66eaad0 2024-01-11T18:08:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.048+0000 7ff07f04bb80 10 client.0 _ll_register_callbacks cb 0x5565f65d56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.048+0000 7ff06899a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.048+0000 7ff06899a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66e7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50856/0 (socket says 172.21.15.7:50856) 2024-01-11T18:08:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.048+0000 7ff06899a700 1 -- 172.21.15.7:0/2195340033 learned_addr learned my addr 172.21.15.7:0/2195340033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.048+0000 7ff06919b700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.048+0000 7ff05a7fc700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff0500a1ea0 0x7ff0500a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.049+0000 7ff05a7fc700 1 -- 172.21.15.7:0/4128315059 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff05c0ce680 con 0x7ff0640ffe50 2024-01-11T18:08:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.051+0000 7ff06999c700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.051+0000 7ff06919b700 1 -- 172.21.15.7:0/2195340033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 msgr2=0x5565f66e7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.051+0000 7ff06919b700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66e7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.051+0000 7ff06919b700 1 -- 172.21.15.7:0/2195340033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 msgr2=0x5565f66e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.051+0000 7ff06919b700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.051+0000 7ff06919b700 1 -- 172.21.15.7:0/2195340033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff060009a80 con 0x5565f66e9ff0 2024-01-11T18:08:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.052+0000 7ff06999c700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff0500a1ea0 0x7ff0500a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.053+0000 7ff06999c700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff0500a1ea0 0x7ff0500a4360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff05400dd40 tx=0x7ff054004860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.057+0000 7ff06919b700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66e2af0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0cbea0 tx=0x7ff05c09b080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcede8465ab8492 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.057+0000 7ff04cff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:08:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.057+0000 7ff04cff9700 1 -- 172.21.15.7:0/2195340033 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c09ab60 con 0x5565f66e9ff0 2024-01-11T18:08:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.057+0000 7ff04cff9700 1 -- 172.21.15.7:0/2195340033 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff05c09b900 con 0x5565f66e9ff0 2024-01-11T18:08:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.057+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565f66d3d60 con 0x5565f66e9ff0 2024-01-11T18:08:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.058+0000 7ff04cff9700 1 -- 172.21.15.7:0/2195340033 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff05c09b200 con 0x5565f66e9ff0 2024-01-11T18:08:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.058+0000 7ff07f04bb80 10 client.26330 fetch_fsmap learned FSMap version 44 2024-01-11T18:08:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.058+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5565f6455b00 con 0x5565f66e9ff0 2024-01-11T18:08:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.058+0000 7ff04cff9700 1 -- 172.21.15.7:0/2195340033 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565f66d3d60 con 0x5565f66e9ff0 2024-01-11T18:08:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.059+0000 7ff04cff9700 1 -- 172.21.15.7:0/2195340033 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff05c0a3070 con 0x5565f66e9ff0 2024-01-11T18:08:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff07f04bb80 10 client.26330 fetch_fsmap finished waiting for FSMap version 44 2024-01-11T18:08:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff07f04bb80 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff07f04bb80 20 client.26330 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f6707480 0x5565f67098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff07f04bb80 4 client.26330 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:08:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565f662db60 con 0x5565f6707480 2024-01-11T18:08:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.061+0000 7ff06899a700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f6707480 0x5565f67098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.062+0000 7ff06899a700 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f6707480 0x5565f67098a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e90438fc847f148 server_cookie=445fcc55c9421df2 in_seq=0 out_seq=0 2024-01-11T18:08:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.062+0000 7ff04cff9700 10 client.26330 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.063+0000 7ff04cff9700 1 -- 172.21.15.7:0/2195340033 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565f662db60 con 0x5565f6707480 2024-01-11T18:08:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.063+0000 7ff04cff9700 10 client.26330 handle_command_reply: tid=0 2024-01-11T18:08:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.063+0000 7ff07f04bb80 1 client.26330 shutdown 2024-01-11T18:08:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.063+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 msgr2=0x5565f66e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.063+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66e2af0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0cbea0 tx=0x7ff05c09b080 comp rx=0 tx=0).stop 2024-01-11T18:08:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 shutdown_connections 2024-01-11T18:08:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f6707480 0x5565f67098a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f66e7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/2195340033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66e2af0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 >> 172.21.15.7:0/2195340033 conn(0x5565f6656740 msgr2=0x5565f66daf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.064+0000 7ff04cff9700 0 client.26330 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.065+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 shutdown_connections 2024-01-11T18:08:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.065+0000 7ff07f04bb80 1 -- 172.21.15.7:0/2195340033 wait complete. 2024-01-11T18:08:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.065+0000 7ff07f04bb80 20 client.26330 trim_cache size 0 max 16384 2024-01-11T18:08:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.108+0000 7ff07f04bb80 1 Processor -- start 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 -- start start 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f6709490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f64ab640 con 0x5565f66e9690 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f66e8ca0 con 0x5565f66e9ff0 2024-01-11T18:08:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.109+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f66e2120 con 0x5565f66eaad0 2024-01-11T18:08:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.111+0000 7ff06999c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f6709490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.112+0000 7ff06999c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f6709490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50866/0 (socket says 172.21.15.7:50866) 2024-01-11T18:08:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.112+0000 7ff06999c700 1 -- 172.21.15.7:0/3641711179 learned_addr learned my addr 172.21.15.7:0/3641711179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.112+0000 7ff06899a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.112+0000 7ff06919b700 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.112+0000 7ff06899a700 1 -- 172.21.15.7:0/3641711179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 msgr2=0x5565f6709490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06899a700 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f6709490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06899a700 1 -- 172.21.15.7:0/3641711179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 msgr2=0x5565f66e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06899a700 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06899a700 1 -- 172.21.15.7:0/3641711179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f65f61d0 con 0x5565f66e9ff0 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06999c700 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f6709490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06919b700 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.113+0000 7ff06899a700 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66d79b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5565f670a180 tx=0x7ff060005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799e75f8be2e49f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.114+0000 7ff04effd700 1 -- 172.21.15.7:0/3641711179 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060029070 con 0x5565f66e9ff0 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.114+0000 7ff04effd700 1 -- 172.21.15.7:0/3641711179 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565f662db60 con 0x5565f66e9ff0 2024-01-11T18:08:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.114+0000 7ff04effd700 1 -- 172.21.15.7:0/3641711179 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060029070 con 0x5565f66e9ff0 2024-01-11T18:08:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.118+0000 7ff07f04bb80 1 -- 172.21.15.7:0/3641711179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 msgr2=0x5565f66d79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.118+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66d79b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5565f670a180 tx=0x7ff060005ce0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.118+0000 7ff07f04bb80 1 -- 172.21.15.7:0/3641711179 shutdown_connections 2024-01-11T18:08:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.119+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f6709490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.119+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f66e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.119+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/3641711179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f66d79b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.119+0000 7ff07f04bb80 1 -- 172.21.15.7:0/3641711179 >> 172.21.15.7:0/3641711179 conn(0x5565f6656740 msgr2=0x5565f66db920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.120+0000 7ff07f04bb80 1 -- 172.21.15.7:0/3641711179 shutdown_connections 2024-01-11T18:08:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.120+0000 7ff07f04bb80 1 -- 172.21.15.7:0/3641711179 wait complete. 2024-01-11T18:08:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.120+0000 7ff07f04bb80 1 -- start start 2024-01-11T18:08:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.121+0000 7ff07f04bb80 1 Processor -- start 2024-01-11T18:08:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.122+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f6716cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.122+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f6717230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.122+0000 7ff07f04bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f675c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f675c9d0 con 0x5565f66e9690 2024-01-11T18:08:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f675cb10 con 0x5565f66e9ff0 2024-01-11T18:08:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff07f04bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f675ccd0 con 0x5565f66eaad0 2024-01-11T18:08:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff06999c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f675c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff06999c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f675c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:50876/0 (socket says 172.21.15.7:50876) 2024-01-11T18:08:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff06999c700 1 -- 172.21.15.7:0/1465196872 learned_addr learned my addr 172.21.15.7:0/1465196872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.123+0000 7ff06899a700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f6717230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06999c700 1 -- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 msgr2=0x5565f6717230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06999c700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f6717230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06999c700 1 -- 172.21.15.7:0/1465196872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 msgr2=0x5565f6716cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06919b700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f6716cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06899a700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f6717230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06999c700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f6716cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06999c700 1 -- 172.21.15.7:0/1465196872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565f6455b00 con 0x5565f66eaad0 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff07f04bb80 10 client.0 _ll_register_callbacks cb 0x5565f65d56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0600054e0 con 0x5565f66eaad0 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff06999c700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f675c490 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff054017640 tx=0x7ff054004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77e959d31294f61 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.124+0000 7ff04cff9700 10 client.26356 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:08:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.126+0000 7ff04cff9700 1 -- 172.21.15.7:0/1465196872 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054006d20 con 0x5565f66eaad0 2024-01-11T18:08:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.126+0000 7ff04cff9700 1 -- 172.21.15.7:0/1465196872 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff054005a50 con 0x5565f66eaad0 2024-01-11T18:08:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.126+0000 7ff04cff9700 1 -- 172.21.15.7:0/1465196872 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff054005c10 con 0x5565f66eaad0 2024-01-11T18:08:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.126+0000 7ff07f04bb80 10 client.26356 fetch_fsmap learned FSMap version 44 2024-01-11T18:08:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.126+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=44}) v3 -- 0x7ff05400e040 con 0x5565f66eaad0 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff04cff9700 1 -- 172.21.15.7:0/1465196872 <== mon.1 v2:172.21.15.38:3300/0 4 ==== fsmap(e 44) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff054029020 con 0x5565f66eaad0 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff07f04bb80 10 client.26356 fetch_fsmap finished waiting for FSMap version 44 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff07f04bb80 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff07f04bb80 20 client.26356 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f675e5d0 0x5565f67609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff07f04bb80 4 client.26356 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:08:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.127+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565f662d880 con 0x5565f675e5d0 2024-01-11T18:08:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.130+0000 7ff06919b700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f675e5d0 0x5565f67609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.130+0000 7ff06919b700 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f675e5d0 0x5565f67609f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245d8d5ee262e16a server_cookie=a8b576b32034a2fc in_seq=0 out_seq=0 2024-01-11T18:08:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.130+0000 7ff04cff9700 10 client.26356 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.254+0000 7ff04cff9700 1 -- 172.21.15.7:0/1465196872 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565f662d880 con 0x5565f675e5d0 2024-01-11T18:08:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.254+0000 7ff04cff9700 10 client.26356 handle_command_reply: tid=0 2024-01-11T18:08:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.254+0000 7ff07f04bb80 1 client.26356 shutdown 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 msgr2=0x5565f675c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f675c490 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff054017640 tx=0x7ff054004750 comp rx=0 tx=0).stop 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 shutdown_connections 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5565f675e5d0 0x5565f67609f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff04cff9700 0 client.26356 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5565f66eaad0 0x5565f675c490 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5565f66e9690 0x5565f6716cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 --2- 172.21.15.7:0/1465196872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565f66e9ff0 0x5565f6717230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 >> 172.21.15.7:0/1465196872 conn(0x5565f6656740 msgr2=0x5565f6709490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 shutdown_connections 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.255+0000 7ff07f04bb80 1 -- 172.21.15.7:0/1465196872 wait complete. 2024-01-11T18:08:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.256+0000 7ff07f04bb80 20 client.26356 trim_cache size 0 max 16384 2024-01-11T18:08:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:08:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:08:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:08:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.258+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff0500a1ea0 msgr2=0x7ff0500a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.258+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff0500a1ea0 0x7ff0500a4360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff05400dd40 tx=0x7ff054004860 comp rx=0 tx=0).stop 2024-01-11T18:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.258+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 msgr2=0x7ff06406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.258+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff06406ffa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff05c000f80 tx=0x7ff05c00bce0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 shutdown_connections 2024-01-11T18:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7ff0500a1ea0 0x7ff0500a4360 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101170 0x7ff0641a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0640ffe50 0x7ff06406ffa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 --2- 172.21.15.7:0/4128315059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0641007b0 0x7ff0640704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 >> 172.21.15.7:0/4128315059 conn(0x7ff0640fba30 msgr2=0x7ff0640fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.261+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 shutdown_connections 2024-01-11T18:08:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:10.262+0000 7ff06b3ff700 1 -- 172.21.15.7:0/4128315059 wait complete. 2024-01-11T18:08:11.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:11 smithi007 ceph-mon[33361]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 566 op/s 2024-01-11T18:08:11.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:11 smithi016 ceph-mon[43195]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 566 op/s 2024-01-11T18:08:11.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:11 smithi038 ceph-mon[43582]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 566 op/s 2024-01-11T18:08:13.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:13 smithi007 ceph-mon[33361]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 584 op/s 2024-01-11T18:08:13.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:13 smithi016 ceph-mon[43195]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 584 op/s 2024-01-11T18:08:13.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:13 smithi038 ceph-mon[43582]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 584 op/s 2024-01-11T18:08:14.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 585.0 s, 536.8 tps, lat 193.888 ms stddev 233.584 2024-01-11T18:08:16.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:15 smithi038 ceph-mon[43582]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 485 op/s 2024-01-11T18:08:16.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:15 smithi007 ceph-mon[33361]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 485 op/s 2024-01-11T18:08:16.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:15 smithi016 ceph-mon[43195]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 485 op/s 2024-01-11T18:08:17.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:17 smithi007 ceph-mon[33361]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 584 op/s 2024-01-11T18:08:17.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:17 smithi016 ceph-mon[43195]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 584 op/s 2024-01-11T18:08:17.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:17 smithi038 ceph-mon[43582]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 584 op/s 2024-01-11T18:08:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 590.0 s, 718.6 tps, lat 137.638 ms stddev 191.020 2024-01-11T18:08:19.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:19 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:08:19.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:19 smithi016 ceph-mon[43195]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-01-11T18:08:19.769 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:19 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:08:19.769 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:19 smithi038 ceph-mon[43582]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-01-11T18:08:19.813 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:19 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:08:19.813 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:19 smithi007 ceph-mon[33361]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-01-11T18:08:21.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:21 smithi007 ceph-mon[33361]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-01-11T18:08:21.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:21 smithi016 ceph-mon[43195]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-01-11T18:08:21.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:21 smithi038 ceph-mon[43582]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-01-11T18:08:23.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:23 smithi007 ceph-mon[33361]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-01-11T18:08:23.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:23 smithi016 ceph-mon[43195]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-01-11T18:08:23.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:23 smithi038 ceph-mon[43582]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 -- 172.21.15.7:0/2379712945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 msgr2=0x7f5a701099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2379712945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a701099f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009f90 tx=0x7f5a6800b330 comp rx=0 tx=0).stop 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 -- 172.21.15.7:0/2379712945 shutdown_connections 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2379712945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a701099f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2379712945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a701015b0 0x7f5a701019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2379712945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 -- 172.21.15.7:0/2379712945 >> 172.21.15.7:0/2379712945 conn(0x7f5a700fcb00 msgr2=0x7f5a700fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.535+0000 7f5a74fff700 1 -- 172.21.15.7:0/2379712945 shutdown_connections 2024-01-11T18:08:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.537+0000 7f5a74fff700 1 -- 172.21.15.7:0/2379712945 wait complete. 2024-01-11T18:08:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.542+0000 7f5a74fff700 1 Processor -- start 2024-01-11T18:08:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 -- start start 2024-01-11T18:08:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a701015b0 0x7f5a700753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a70073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700746e0 con 0x7f5a701015b0 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a70074850 con 0x7f5a701027a0 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a74fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a701a16e0 con 0x7f5a70103140 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.543+0000 7f5a6dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43944/0 (socket says 172.21.15.7:43944) 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/2606948561 learned_addr learned my addr 172.21.15.7:0/2606948561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a70073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a701015b0 0x7f5a700753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/2606948561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 msgr2=0x7f5a70073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a70073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a701015b0 msgr2=0x7f5a700753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a701015b0 0x7f5a700753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/2606948561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68009c40 con 0x7f5a701027a0 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.544+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a70073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:08:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.546+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70073a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e9c0 tx=0x7f5a6400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.546+0000 7f5a5f7fe700 1 -- 172.21.15.7:0/2606948561 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400cbd0 con 0x7f5a701027a0 2024-01-11T18:08:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.547+0000 7f5a5f7fe700 1 -- 172.21.15.7:0/2606948561 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a6400cd30 con 0x7f5a701027a0 2024-01-11T18:08:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.547+0000 7f5a5f7fe700 1 -- 172.21.15.7:0/2606948561 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64005580 con 0x7f5a701027a0 2024-01-11T18:08:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.547+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a701a19c0 con 0x7f5a701027a0 2024-01-11T18:08:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.547+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a701a1f00 con 0x7f5a701027a0 2024-01-11T18:08:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.556+0000 7f5a5f7fe700 1 -- 172.21.15.7:0/2606948561 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f5a64004750 con 0x7f5a701027a0 2024-01-11T18:08:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a5f7fe700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a580a20a0 0x7f5a580a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a88d58b80 1 Processor -- start 2024-01-11T18:08:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a5f7fe700 1 -- 172.21.15.7:0/2606948561 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a64014070 con 0x7f5a701027a0 2024-01-11T18:08:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a88d58b80 1 -- start start 2024-01-11T18:08:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a580a20a0 0x7f5a580a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.570+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdca71030 con 0x559bdcb82730 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcad4ac0 con 0x559bdcb83420 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcad23e0 con 0x559bdcb83de0 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46262/0 (socket says 172.21.15.7:46262) 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/59941979 learned_addr learned my addr 172.21.15.7:0/59941979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/59941979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 msgr2=0x559bdcb8a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/59941979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 msgr2=0x559bdcb82b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.571+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb82b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.572+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/59941979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bdc942350 con 0x559bdcb83420 2024-01-11T18:08:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.573+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb838a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5a6409a5b0 tx=0x7f5a64009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab0946f13021f4d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.573+0000 7f5a5cff9700 1 -- 172.21.15.7:0/59941979 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6409a8b0 con 0x559bdcb83420 2024-01-11T18:08:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.573+0000 7f5a5cff9700 1 -- 172.21.15.7:0/59941979 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a6409ea00 con 0x559bdcb83420 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.573+0000 7f5a5cff9700 1 -- 172.21.15.7:0/59941979 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6401f340 con 0x559bdcb83420 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 -- 172.21.15.7:0/59941979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 msgr2=0x559bdcb838a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb838a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5a6409a5b0 tx=0x7f5a64009740 comp rx=0 tx=0).stop 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a580a20a0 0x7f5a580a4560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x559bdcb824d0 tx=0x7f5a6000a350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 -- 172.21.15.7:0/59941979 shutdown_connections 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb82b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/59941979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb838a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.574+0000 7f5a88d58b80 1 -- 172.21.15.7:0/59941979 >> 172.21.15.7:0/59941979 conn(0x559bdcaef740 msgr2=0x559bdcb74880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.575+0000 7f5a88d58b80 1 -- 172.21.15.7:0/59941979 shutdown_connections 2024-01-11T18:08:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.576+0000 7f5a88d58b80 1 -- 172.21.15.7:0/59941979 wait complete. 2024-01-11T18:08:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.576+0000 7f5a88d58b80 1 -- start start 2024-01-11T18:08:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.577+0000 7f5a88d58b80 1 Processor -- start 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb82730 0x559bdcb8c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb8c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcacdcd0 con 0x559bdcb83420 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdca3c340 con 0x559bdcb82730 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdca3bfe0 con 0x559bdcb83de0 2024-01-11T18:08:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a88d58b80 10 client.0 _ll_register_callbacks cb 0x559bdca6e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a6ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a6ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46276/0 (socket says 172.21.15.7:46276) 2024-01-11T18:08:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/3912008661 learned_addr learned my addr 172.21.15.7:0/3912008661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.578+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb82730 0x559bdcb8c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.579+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb82730 msgr2=0x559bdcb8c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.579+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb82730 0x559bdcb8c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.579+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/3912008661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 msgr2=0x559bdcb8c6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.579+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb8c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.579+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/3912008661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a701a1f00 con 0x559bdcb83de0 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.580+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb936f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5a68007d60 tx=0x7f5a68007d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9833597445ccfc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.580+0000 7f5a55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:08:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.580+0000 7f5a55ffb700 1 -- 172.21.15.7:0/3912008661 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800e040 con 0x559bdcb83de0 2024-01-11T18:08:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.581+0000 7f5a55ffb700 1 -- 172.21.15.7:0/3912008661 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a68022070 con 0x559bdcb83de0 2024-01-11T18:08:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.581+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bdcb6cd60 con 0x559bdcb83de0 2024-01-11T18:08:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.581+0000 7f5a55ffb700 1 -- 172.21.15.7:0/3912008661 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68004e40 con 0x559bdcb83de0 2024-01-11T18:08:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.581+0000 7f5a88d58b80 10 client.26368 fetch_fsmap learned FSMap version 45 2024-01-11T18:08:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.581+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x559bdc8eeb00 con 0x559bdcb83de0 2024-01-11T18:08:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.582+0000 7f5a55ffb700 1 -- 172.21.15.7:0/3912008661 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559bdcb6cd60 con 0x559bdcb83de0 2024-01-11T18:08:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a55ffb700 1 -- 172.21.15.7:0/3912008661 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a6800b050 con 0x559bdcb83de0 2024-01-11T18:08:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a88d58b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:08:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a88d58b80 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a88d58b80 20 client.26368 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcb97fb0 0x559bdcb9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a88d58b80 4 client.26368 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:08:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.583+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bdcac6b60 con 0x559bdcb97fb0 2024-01-11T18:08:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.584+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcb97fb0 0x559bdcb9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.585+0000 7f5a55ffb700 10 client.26368 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.585+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcb97fb0 0x559bdcb9a3d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0f87052495ea09 server_cookie=e9dfc5639cb4bb4b in_seq=0 out_seq=0 2024-01-11T18:08:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.586+0000 7f5a55ffb700 1 -- 172.21.15.7:0/3912008661 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bdcac6b60 con 0x559bdcb97fb0 2024-01-11T18:08:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.586+0000 7f5a55ffb700 10 client.26368 handle_command_reply: tid=0 2024-01-11T18:08:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 client.26368 shutdown 2024-01-11T18:08:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 msgr2=0x559bdcb936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb936f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5a68007d60 tx=0x7f5a68007d90 comp rx=0 tx=0).stop 2024-01-11T18:08:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 shutdown_connections 2024-01-11T18:08:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcb97fb0 0x559bdcb9a3d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb936f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb82730 0x559bdcb8c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3912008661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb8c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 >> 172.21.15.7:0/3912008661 conn(0x559bdcaef740 msgr2=0x559bdcb73f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.587+0000 7f5a55ffb700 0 client.26368 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.588+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 shutdown_connections 2024-01-11T18:08:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.588+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3912008661 wait complete. 2024-01-11T18:08:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.588+0000 7f5a88d58b80 20 client.26368 trim_cache size 0 max 16384 2024-01-11T18:08:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.621+0000 7f5a88d58b80 1 Processor -- start 2024-01-11T18:08:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 -- start start 2024-01-11T18:08:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdc944640 con 0x559bdcb82730 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcb948a0 con 0x559bdcb83420 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.622+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcb9f5e0 con 0x559bdcb83de0 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.623+0000 7f5a6e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.623+0000 7f5a6ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.623+0000 7f5a6e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb6c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57454/0 (socket says 172.21.15.7:57454) 2024-01-11T18:08:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.623+0000 7f5a6e59c700 1 -- 172.21.15.7:0/3283874918 learned_addr learned my addr 172.21.15.7:0/3283874918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.623+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.624+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/3283874918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 msgr2=0x559bdcb8b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.624+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.624+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/3283874918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 msgr2=0x559bdcb6c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.624+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb6c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.624+0000 7f5a6dd9b700 1 -- 172.21.15.7:0/3283874918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bdca8f1d0 con 0x559bdcb83420 2024-01-11T18:08:23.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.625+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb936b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400c5b0 tx=0x7f5a64009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5adb3fd2f50acfc9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:23.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.626+0000 7f5a57fff700 1 -- 172.21.15.7:0/3283874918 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6409a8b0 con 0x559bdcb83420 2024-01-11T18:08:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.626+0000 7f5a57fff700 1 -- 172.21.15.7:0/3283874918 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a640a2c40 con 0x559bdcb83420 2024-01-11T18:08:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.626+0000 7f5a57fff700 1 -- 172.21.15.7:0/3283874918 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6401f340 con 0x559bdcb83420 2024-01-11T18:08:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.627+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3283874918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 msgr2=0x559bdcb936b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.627+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb936b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400c5b0 tx=0x7f5a64009550 comp rx=0 tx=0).stop 2024-01-11T18:08:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.627+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3283874918 shutdown_connections 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.627+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb83de0 0x559bdcb8b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.628+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb82730 0x559bdcb6c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.628+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/3283874918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83420 0x559bdcb936b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.628+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3283874918 >> 172.21.15.7:0/3283874918 conn(0x559bdcaef740 msgr2=0x559bdcb709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.628+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3283874918 shutdown_connections 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.628+0000 7f5a88d58b80 1 -- 172.21.15.7:0/3283874918 wait complete. 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.628+0000 7f5a88d58b80 1 -- start start 2024-01-11T18:08:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.629+0000 7f5a88d58b80 1 Processor -- start 2024-01-11T18:08:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.629+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 0x559bdcb90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.629+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a88d58b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83de0 0x559bdcb92110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcb92650 con 0x559bdcb83420 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcb927c0 con 0x559bdcb83de0 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a88d58b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdcb92980 con 0x559bdcb82730 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a88d58b80 10 client.0 _ll_register_callbacks cb 0x559bdca6e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 0x559bdcb90c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 0x559bdcb90c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46292/0 (socket says 172.21.15.7:46292) 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6e59c700 1 -- 172.21.15.7:0/2521878240 learned_addr learned my addr 172.21.15.7:0/2521878240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83de0 0x559bdcb92110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6dd9b700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/2521878240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 msgr2=0x559bdcb90c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 0x559bdcb90c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/2521878240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 msgr2=0x559bdcb91150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb91150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.630+0000 7f5a6ed9d700 1 -- 172.21.15.7:0/2521878240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bdc942350 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.631+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 0x559bdcb90c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.631+0000 7f5a6ed9d700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83de0 0x559bdcb92110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5a68007d30 tx=0x7f5a680096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3be2287c0a55f78 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.631+0000 7f5a55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.631+0000 7f5a55ffb700 1 -- 172.21.15.7:0/2521878240 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800e070 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.631+0000 7f5a55ffb700 1 -- 172.21.15.7:0/2521878240 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a68022040 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.631+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a6409ea00 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a55ffb700 1 -- 172.21.15.7:0/2521878240 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6800b0b0 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 10 client.26351 fetch_fsmap learned FSMap version 45 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x559bdc946c70 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a55ffb700 1 -- 172.21.15.7:0/2521878240 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a6409ea00 con 0x559bdcb83de0 2024-01-11T18:08:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a55ffb700 1 -- 172.21.15.7:0/2521878240 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a680058c0 con 0x559bdcb83de0 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 10 client.26351 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 20 client.26351 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcbee810 0x559bdcbf0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 4 client.26351 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.632+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559bdcac6880 con 0x559bdcbee810 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.633+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcbee810 0x559bdcbf0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.633+0000 7f5a6e59c700 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcbee810 0x559bdcbf0c30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77ca9b9d4e6072b server_cookie=46174fcc3093c376 in_seq=0 out_seq=0 2024-01-11T18:08:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.633+0000 7f5a55ffb700 10 client.26351 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.649+0000 7f5a55ffb700 1 -- 172.21.15.7:0/2521878240 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559bdcac6880 con 0x559bdcbee810 2024-01-11T18:08:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.649+0000 7f5a55ffb700 10 client.26351 handle_command_reply: tid=0 2024-01-11T18:08:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.650+0000 7f5a88d58b80 1 client.26351 shutdown 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83de0 msgr2=0x559bdcb92110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83de0 0x559bdcb92110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5a68007d30 tx=0x7f5a680096c0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 shutdown_connections 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559bdcbee810 0x559bdcbf0c30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559bdcb82730 0x559bdcb90c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a55ffb700 0 client.26351 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bdcb83de0 0x559bdcb92110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 --2- 172.21.15.7:0/2521878240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559bdcb83420 0x559bdcb91150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.651+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 >> 172.21.15.7:0/2521878240 conn(0x559bdcaef740 msgr2=0x559bdcb8b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.652+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 shutdown_connections 2024-01-11T18:08:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.652+0000 7f5a88d58b80 1 -- 172.21.15.7:0/2521878240 wait complete. 2024-01-11T18:08:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.652+0000 7f5a88d58b80 20 client.26351 trim_cache size 0 max 16384 2024-01-11T18:08:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:08:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:08:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:08:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a580a20a0 msgr2=0x7f5a580a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a580a20a0 0x7f5a580a4560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x559bdcb824d0 tx=0x7f5a6000a350 comp rx=0 tx=0).stop 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 msgr2=0x7f5a70073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70073a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e9c0 tx=0x7f5a6400ed80 comp rx=0 tx=0).stop 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 shutdown_connections 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f5a580a20a0 0x7f5a580a4560 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70103140 0x7f5a70073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a701015b0 0x7f5a700753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 --2- 172.21.15.7:0/2606948561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701027a0 0x7f5a70073a00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 >> 172.21.15.7:0/2606948561 conn(0x7f5a700fcb00 msgr2=0x7f5a70105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 shutdown_connections 2024-01-11T18:08:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:23.653+0000 7f5a74fff700 1 -- 172.21.15.7:0/2606948561 wait complete. 2024-01-11T18:08:24.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 595.0 s, 405.6 tps, lat 246.405 ms stddev 385.793 2024-01-11T18:08:26.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:25 smithi007 ceph-mon[33361]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-01-11T18:08:26.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:25 smithi016 ceph-mon[43195]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-01-11T18:08:26.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:25 smithi038 ceph-mon[43582]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-01-11T18:08:27.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:27 smithi007 ceph-mon[33361]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 602 op/s 2024-01-11T18:08:27.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:27 smithi016 ceph-mon[43195]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 602 op/s 2024-01-11T18:08:27.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:27 smithi038 ceph-mon[43582]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 602 op/s 2024-01-11T18:08:29.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 600.0 s, 519.2 tps, lat 193.144 ms stddev 219.472 2024-01-11T18:08:29.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:29 smithi007 ceph-mon[33361]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-01-11T18:08:29.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:29 smithi016 ceph-mon[43195]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-01-11T18:08:29.524 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:29 smithi038 ceph-mon[43582]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-01-11T18:08:31.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:31 smithi007 ceph-mon[33361]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.4 MiB/s wr, 620 op/s 2024-01-11T18:08:31.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:31 smithi016 ceph-mon[43195]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.4 MiB/s wr, 620 op/s 2024-01-11T18:08:31.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:31 smithi038 ceph-mon[43582]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.4 MiB/s wr, 620 op/s 2024-01-11T18:08:33.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:33 smithi007 ceph-mon[33361]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-01-11T18:08:33.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:33 smithi016 ceph-mon[43195]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-01-11T18:08:33.520 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:33 smithi038 ceph-mon[43582]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-01-11T18:08:34.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 605.0 s, 463.8 tps, lat 212.574 ms stddev 241.615 2024-01-11T18:08:36.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:35 smithi007 ceph-mon[33361]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-01-11T18:08:36.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:35 smithi016 ceph-mon[43195]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-01-11T18:08:36.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:35 smithi038 ceph-mon[43582]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.905+0000 7f47abfff700 1 -- 172.21.15.7:0/2702644132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 msgr2=0x7f47a410d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.905+0000 7f47abfff700 1 --2- 172.21.15.7:0/2702644132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a410d410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4794009990 tx=0x7f4794009ca0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 -- 172.21.15.7:0/2702644132 shutdown_connections 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 --2- 172.21.15.7:0/2702644132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 --2- 172.21.15.7:0/2702644132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 0x7f47a410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 --2- 172.21.15.7:0/2702644132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a410d410 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 -- 172.21.15.7:0/2702644132 >> 172.21.15.7:0/2702644132 conn(0x7f47a406e500 msgr2=0x7f47a4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 -- 172.21.15.7:0/2702644132 shutdown_connections 2024-01-11T18:08:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.906+0000 7f47abfff700 1 -- 172.21.15.7:0/2702644132 wait complete. 2024-01-11T18:08:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 Processor -- start 2024-01-11T18:08:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 -- start start 2024-01-11T18:08:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 0x7f47a4114bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a4115100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a41b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41181e0 con 0x7f47a4108610 2024-01-11T18:08:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a4118350 con 0x7f47a410d950 2024-01-11T18:08:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.909+0000 7f47abfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41b6a40 con 0x7f47a410af80 2024-01-11T18:08:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.910+0000 7f47aa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a41b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.910+0000 7f47aa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a41b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:32838/0 (socket says 172.21.15.7:32838) 2024-01-11T18:08:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.910+0000 7f47aa59c700 1 -- 172.21.15.7:0/3446161461 learned_addr learned my addr 172.21.15.7:0/3446161461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.910+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 0x7f47a4114bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.911+0000 7f47a959a700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a4115100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.911+0000 7f47aa59c700 1 -- 172.21.15.7:0/3446161461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 msgr2=0x7f47a4115100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.911+0000 7f47aa59c700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a4115100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.912+0000 7f47aa59c700 1 -- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 msgr2=0x7f47a4114bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.912+0000 7f47aa59c700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 0x7f47a4114bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.912+0000 7f47aa59c700 1 -- 172.21.15.7:0/3446161461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940095f0 con 0x7f47a410d950 2024-01-11T18:08:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.912+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 0x7f47a4114bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:08:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.912+0000 7f47a959a700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a4115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:08:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.912+0000 7f47aa59c700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a41b6500 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f479c00ba10 tx=0x7f479c00bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.913+0000 7f479affd700 1 -- 172.21.15.7:0/3446161461 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00d8e0 con 0x7f47a410d950 2024-01-11T18:08:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.913+0000 7f479affd700 1 -- 172.21.15.7:0/3446161461 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f479c009800 con 0x7f47a410d950 2024-01-11T18:08:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.913+0000 7f479affd700 1 -- 172.21.15.7:0/3446161461 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00c400 con 0x7f47a410d950 2024-01-11T18:08:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.913+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a41b6ca0 con 0x7f47a410d950 2024-01-11T18:08:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.913+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a41b7160 con 0x7f47a410d950 2024-01-11T18:08:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.918+0000 7f479affd700 1 -- 172.21.15.7:0/3446161461 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f479c015600 con 0x7f47a410d950 2024-01-11T18:08:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.922+0000 7f479affd700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47900a2020 0x7f47900a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.922+0000 7f479affd700 1 -- 172.21.15.7:0/3446161461 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f479c011070 con 0x7f47a410d950 2024-01-11T18:08:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.923+0000 7f47bfcbab80 1 Processor -- start 2024-01-11T18:08:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.924+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47900a2020 0x7f47900a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.924+0000 7f47bfcbab80 1 -- start start 2024-01-11T18:08:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.925+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703797340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.925+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x558703798090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.925+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x55870379ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.925+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870368e030 con 0x5587037985d0 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.925+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587036f1ac0 con 0x558703796f20 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.925+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587036ef3e0 con 0x558703797c10 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x558703798090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x558703798090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38860/0 (socket says 172.21.15.7:38860) 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 -- 172.21.15.7:0/2591971009 learned_addr learned my addr 172.21.15.7:0/2591971009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 -- 172.21.15.7:0/2591971009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 msgr2=0x558703797340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:08:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703797340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 -- 172.21.15.7:0/2591971009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 msgr2=0x55870379ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x55870379ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.926+0000 7f47a959a700 1 -- 172.21.15.7:0/2591971009 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870355f350 con 0x558703797c10 2024-01-11T18:08:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.927+0000 7f47a959a700 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x558703798090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f479400b2e0 tx=0x7f4794005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf7d1f178117689d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:36.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.927+0000 7f478ffff700 1 -- 172.21.15.7:0/2591971009 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479401e070 con 0x558703797c10 2024-01-11T18:08:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.927+0000 7f478ffff700 1 -- 172.21.15.7:0/2591971009 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f479400bd70 con 0x558703797c10 2024-01-11T18:08:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.927+0000 7f478ffff700 1 -- 172.21.15.7:0/2591971009 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794004ae0 con 0x558703797c10 2024-01-11T18:08:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.928+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2591971009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 msgr2=0x558703798090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.928+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x558703798090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f479400b2e0 tx=0x7f4794005f00 comp rx=0 tx=0).stop 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.928+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2591971009 shutdown_connections 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x55870379ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703797340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2591971009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x558703798090 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2591971009 >> 172.21.15.7:0/2591971009 conn(0x55870370c740 msgr2=0x558703791950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47900a2020 0x7f47900a44e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x558703796cc0 tx=0x7f47a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2591971009 shutdown_connections 2024-01-11T18:08:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.929+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2591971009 wait complete. 2024-01-11T18:08:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.930+0000 7f47bfcbab80 1 -- start start 2024-01-11T18:08:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.930+0000 7f47bfcbab80 1 Processor -- start 2024-01-11T18:08:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703796f20 0x5587037a0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587037985d0 0x5587037a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587036eacd0 con 0x558703796f20 2024-01-11T18:08:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703659340 con 0x5587037985d0 2024-01-11T18:08:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703658fe0 con 0x558703797c10 2024-01-11T18:08:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.931+0000 7f47bfcbab80 10 client.0 _ll_register_callbacks cb 0x55870368b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703796f20 0x5587037a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38876/0 (socket says 172.21.15.7:38876) 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47aa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587037985d0 0x5587037a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 -- 172.21.15.7:0/1711879183 learned_addr learned my addr 172.21.15.7:0/1711879183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 -- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587037985d0 msgr2=0x5587037a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587037985d0 0x5587037a7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 -- 172.21.15.7:0/1711879183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703796f20 msgr2=0x5587037a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703796f20 0x5587037a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47a959a700 1 -- 172.21.15.7:0/1711879183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a410e900 con 0x558703797c10 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.932+0000 7f47aa59c700 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587037985d0 0x5587037a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.933+0000 7f47a959a700 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a0ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f47940092b0 tx=0x7f4794023ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c580e93e0eaa391 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.933+0000 7f478d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:08:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.933+0000 7f478d7fa700 1 -- 172.21.15.7:0/1711879183 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479401e040 con 0x558703797c10 2024-01-11T18:08:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.933+0000 7f478d7fa700 1 -- 172.21.15.7:0/1711879183 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f479400f660 con 0x558703797c10 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.933+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558703789d60 con 0x558703797c10 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.933+0000 7f478d7fa700 1 -- 172.21.15.7:0/1711879183 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479400f820 con 0x558703797c10 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 10 client.26383 fetch_fsmap learned FSMap version 45 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55870350bb00 con 0x558703797c10 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f478d7fa700 1 -- 172.21.15.7:0/1711879183 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558703789d60 con 0x558703797c10 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f478d7fa700 1 -- 172.21.15.7:0/1711879183 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f479402cbd0 con 0x558703797c10 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 10 client.26383 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 10 client.26383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 20 client.26383 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5587037ac300 0x5587037ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 4 client.26383 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:08:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.934+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587036e3b60 con 0x5587037ac300 2024-01-11T18:08:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.935+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5587037ac300 0x5587037ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.936+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5587037ac300 0x5587037ae720 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caca33101698dd1 server_cookie=94991d6e4e547169 in_seq=0 out_seq=0 2024-01-11T18:08:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.936+0000 7f478d7fa700 10 client.26383 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.936+0000 7f478d7fa700 1 -- 172.21.15.7:0/1711879183 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587036e3b60 con 0x5587037ac300 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.936+0000 7f478d7fa700 10 client.26383 handle_command_reply: tid=0 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 client.26383 shutdown 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 msgr2=0x5587037a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a0ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f47940092b0 tx=0x7f4794023ac0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 shutdown_connections 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5587037ac300 0x5587037ae720 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587037985d0 0x5587037a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703796f20 0x5587037a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/1711879183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a0ed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 >> 172.21.15.7:0/1711879183 conn(0x55870370c740 msgr2=0x558703790f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f478d7fa700 0 client.26383 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.937+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 shutdown_connections 2024-01-11T18:08:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.938+0000 7f47bfcbab80 1 -- 172.21.15.7:0/1711879183 wait complete. 2024-01-11T18:08:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.938+0000 7f47bfcbab80 20 client.26383 trim_cache size 0 max 16384 2024-01-11T18:08:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.963+0000 7f47bfcbab80 1 Processor -- start 2024-01-11T18:08:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.963+0000 7f47bfcbab80 1 -- start start 2024-01-11T18:08:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703798010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703797c10 0x5587037a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5587037985d0 0x55870379b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703561640 con 0x558703797c10 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703792810 con 0x558703796f20 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587036e3b60 con 0x5587037985d0 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703798010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703798010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:32858/0 (socket says 172.21.15.7:32858) 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47a9d9b700 1 -- 172.21.15.7:0/3546719483 learned_addr learned my addr 172.21.15.7:0/3546719483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.964+0000 7f47aa59c700 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5587037985d0 0x55870379b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a959a700 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703797c10 0x5587037a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a9d9b700 1 -- 172.21.15.7:0/3546719483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5587037985d0 msgr2=0x55870379b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5587037985d0 0x55870379b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a9d9b700 1 -- 172.21.15.7:0/3546719483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703797c10 msgr2=0x5587037a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703797c10 0x5587037a7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a9d9b700 1 -- 172.21.15.7:0/3546719483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587036ac1d0 con 0x558703796f20 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a959a700 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703797c10 0x5587037a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703798010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47a0011a50 tx=0x7f47a0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c7171ef4623a1d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f478f7fe700 1 -- 172.21.15.7:0/3546719483 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0004440 con 0x558703796f20 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f478f7fe700 1 -- 172.21.15.7:0/3546719483 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a00045a0 con 0x558703796f20 2024-01-11T18:08:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.965+0000 7f478f7fe700 1 -- 172.21.15.7:0/3546719483 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0012b30 con 0x558703796f20 2024-01-11T18:08:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 -- 172.21.15.7:0/3546719483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 msgr2=0x558703798010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703798010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47a0011a50 tx=0x7f47a0014000 comp rx=0 tx=0).stop 2024-01-11T18:08:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 -- 172.21.15.7:0/3546719483 shutdown_connections 2024-01-11T18:08:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5587037985d0 0x55870379b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x558703798010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/3546719483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558703797c10 0x5587037a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.966+0000 7f47bfcbab80 1 -- 172.21.15.7:0/3546719483 >> 172.21.15.7:0/3546719483 conn(0x55870370c740 msgr2=0x55870378d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.967+0000 7f47bfcbab80 1 -- 172.21.15.7:0/3546719483 shutdown_connections 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.967+0000 7f47bfcbab80 1 -- 172.21.15.7:0/3546719483 wait complete. 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.967+0000 7f47bfcbab80 1 -- start start 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.967+0000 7f47bfcbab80 1 Processor -- start 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x5587037a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x5587038007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703800d20 con 0x5587037985d0 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703800e60 con 0x558703796f20 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558703800ff0 con 0x558703797c10 2024-01-11T18:08:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.968+0000 7f47bfcbab80 10 client.0 _ll_register_callbacks cb 0x55870368b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47aa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x5587038007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x5587037a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x5587037a4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:32864/0 (socket says 172.21.15.7:32864) 2024-01-11T18:08:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 -- 172.21.15.7:0/2090333665 learned_addr learned my addr 172.21.15.7:0/2090333665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a959a700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 -- 172.21.15.7:0/2090333665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 msgr2=0x5587037a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 -- 172.21.15.7:0/2090333665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 msgr2=0x5587038007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x5587038007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a9d9b700 1 -- 172.21.15.7:0/2090333665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a0011c90 con 0x558703796f20 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47aa59c700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x5587038007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:08:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.969+0000 7f47a959a700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.970+0000 7f47a9d9b700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x5587037a4f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fd0 tx=0x7f47a0012830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1211acc444865c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.970+0000 7f478d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.970+0000 7f478d7fa700 1 -- 172.21.15.7:0/2090333665 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0027040 con 0x558703796f20 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.970+0000 7f478d7fa700 1 -- 172.21.15.7:0/2090333665 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f47a0018070 con 0x558703796f20 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.970+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f479400bd70 con 0x558703796f20 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.971+0000 7f478d7fa700 1 -- 172.21.15.7:0/2090333665 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a0012ea0 con 0x558703796f20 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.971+0000 7f47bfcbab80 10 client.26363 fetch_fsmap learned FSMap version 45 2024-01-11T18:08:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.971+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x558703563c70 con 0x558703796f20 2024-01-11T18:08:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.972+0000 7f478d7fa700 1 -- 172.21.15.7:0/2090333665 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f479400bd70 con 0x558703796f20 2024-01-11T18:08:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.972+0000 7f478d7fa700 1 -- 172.21.15.7:0/2090333665 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f47a0003680 con 0x558703796f20 2024-01-11T18:08:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.974+0000 7f47bfcbab80 10 client.26363 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:08:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.974+0000 7f47bfcbab80 10 client.26363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.974+0000 7f47bfcbab80 20 client.26363 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.974+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558703802960 0x558703804d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.974+0000 7f47bfcbab80 4 client.26363 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:08:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.974+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5587036e3880 con 0x558703802960 2024-01-11T18:08:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.975+0000 7f47a959a700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558703802960 0x558703804d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.976+0000 7f47a959a700 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558703802960 0x558703804d80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d48ebae67b02449 server_cookie=92589f45a720fc3f in_seq=0 out_seq=0 2024-01-11T18:08:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:36.976+0000 7f478d7fa700 10 client.26363 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.069+0000 7f478d7fa700 1 -- 172.21.15.7:0/2090333665 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5587036e3880 con 0x558703802960 2024-01-11T18:08:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.069+0000 7f478d7fa700 10 client.26363 handle_command_reply: tid=0 2024-01-11T18:08:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.069+0000 7f47bfcbab80 1 client.26363 shutdown 2024-01-11T18:08:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.070+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 msgr2=0x5587037a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.070+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x5587037a4f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fd0 tx=0x7f47a0012830 comp rx=0 tx=0).stop 2024-01-11T18:08:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 shutdown_connections 2024-01-11T18:08:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558703802960 0x558703804d80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587037985d0 0x5587038007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558703796f20 0x5587037a4f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 --2- 172.21.15.7:0/2090333665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558703797c10 0x5587037a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 >> 172.21.15.7:0/2090333665 conn(0x55870370c740 msgr2=0x55870379b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f478d7fa700 0 client.26363 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 shutdown_connections 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.071+0000 7f47bfcbab80 1 -- 172.21.15.7:0/2090333665 wait complete. 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.072+0000 7f47bfcbab80 20 client.26363 trim_cache size 0 max 16384 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:08:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:08:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47900a2020 msgr2=0x7f47900a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47900a2020 0x7f47900a44e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x558703796cc0 tx=0x7f47a0009260 comp rx=0 tx=0).stop 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 msgr2=0x7f47a41b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a41b6500 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f479c00ba10 tx=0x7f479c00bdd0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 shutdown_connections 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f47900a2020 0x7f47900a44e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47a410d950 0x7f47a41b6500 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a4108610 0x7f47a4114bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 --2- 172.21.15.7:0/3446161461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a410af80 0x7f47a4115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.074+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 >> 172.21.15.7:0/3446161461 conn(0x7f47a406e500 msgr2=0x7f47a4108a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.075+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 shutdown_connections 2024-01-11T18:08:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:37.075+0000 7f47abfff700 1 -- 172.21.15.7:0/3446161461 wait complete. 2024-01-11T18:08:37.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:37 smithi007 ceph-mon[33361]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.4 MiB/s wr, 523 op/s 2024-01-11T18:08:37.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:37 smithi016 ceph-mon[43195]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.4 MiB/s wr, 523 op/s 2024-01-11T18:08:37.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:37 smithi038 ceph-mon[43582]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.4 MiB/s wr, 523 op/s 2024-01-11T18:08:38.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:38 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:08:38.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:38 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:08:38.520 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:38 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:08:39.114 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 610.0 s, 432.4 tps, lat 231.152 ms stddev 259.335 2024-01-11T18:08:39.490 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:39 smithi016 ceph-mon[43195]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.6 MiB/s wr, 435 op/s 2024-01-11T18:08:39.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:39 smithi038 ceph-mon[43582]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.6 MiB/s wr, 435 op/s 2024-01-11T18:08:39.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:39 smithi007 ceph-mon[33361]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.6 MiB/s wr, 435 op/s 2024-01-11T18:08:41.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:40 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:40 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:40 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:40 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:40 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:40 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:40 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:40 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:40 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:41.992 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:41 smithi016 ceph-mon[43195]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-01-11T18:08:42.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:41 smithi038 ceph-mon[43582]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-01-11T18:08:42.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:41 smithi007 ceph-mon[33361]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-01-11T18:08:43.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:43 smithi016 ceph-mon[43195]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 MiB/s wr, 413 op/s 2024-01-11T18:08:43.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:43 smithi038 ceph-mon[43582]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 MiB/s wr, 413 op/s 2024-01-11T18:08:43.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:43 smithi007 ceph-mon[33361]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 MiB/s wr, 413 op/s 2024-01-11T18:08:44.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 615.0 s, 477.4 tps, lat 212.440 ms stddev 240.253 2024-01-11T18:08:44.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:44.987 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:44 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:45.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:45.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:44 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:45.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:45.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:44 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:46.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:45 smithi007 ceph-mon[33361]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.2 MiB/s wr, 437 op/s 2024-01-11T18:08:46.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:08:46.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:08:46.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:46.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:45 smithi016 ceph-mon[43195]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.2 MiB/s wr, 437 op/s 2024-01-11T18:08:46.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:08:46.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:08:46.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:46.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:45 smithi038 ceph-mon[43582]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.2 MiB/s wr, 437 op/s 2024-01-11T18:08:46.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:08:46.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:08:46.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:08:47.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:47 smithi007 ceph-mon[33361]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-01-11T18:08:47.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:47 smithi016 ceph-mon[43195]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-01-11T18:08:47.527 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:47 smithi038 ceph-mon[43582]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 485 op/s 2024-01-11T18:08:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 620.0 s, 509.2 tps, lat 193.021 ms stddev 219.119 2024-01-11T18:08:49.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:49 smithi007 ceph-mon[33361]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-01-11T18:08:49.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:49 smithi016 ceph-mon[43195]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-01-11T18:08:49.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:49 smithi038 ceph-mon[43582]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-01-11T18:08:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.339+0000 7f8d45fff700 1 -- 172.21.15.7:0/2698691910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40107280 msgr2=0x7f8d401096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.339+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2698691910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40107280 0x7f8d401096f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009f90 tx=0x7f8d3800b330 comp rx=0 tx=0).stop 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.340+0000 7f8d45fff700 1 -- 172.21.15.7:0/2698691910 shutdown_connections 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.340+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2698691910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40107280 0x7f8d401096f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.340+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2698691910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40102020 0x7f8d40104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.340+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2698691910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40104950 0x7f8d40106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.340+0000 7f8d45fff700 1 -- 172.21.15.7:0/2698691910 >> 172.21.15.7:0/2698691910 conn(0x7f8d400fba30 msgr2=0x7f8d400fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.341+0000 7f8d45fff700 1 -- 172.21.15.7:0/2698691910 shutdown_connections 2024-01-11T18:08:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.341+0000 7f8d45fff700 1 -- 172.21.15.7:0/2698691910 wait complete. 2024-01-11T18:08:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 Processor -- start 2024-01-11T18:08:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 -- start start 2024-01-11T18:08:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40102020 0x7f8d40101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 0x7f8d400fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40107280 0x7f8d40100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019b300 con 0x7f8d40107280 2024-01-11T18:08:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40100a50 con 0x7f8d40104950 2024-01-11T18:08:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.343+0000 7f8d45fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40100c10 con 0x7f8d40102020 2024-01-11T18:08:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40102020 0x7f8d40101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 0x7f8d400fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 0x7f8d400fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54634/0 (socket says 172.21.15.7:54634) 2024-01-11T18:08:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3effd700 1 -- 172.21.15.7:0/2396241788 learned_addr learned my addr 172.21.15.7:0/2396241788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40107280 0x7f8d40100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3effd700 1 -- 172.21.15.7:0/2396241788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40102020 msgr2=0x7f8d40101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.345+0000 7f8d3effd700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40102020 0x7f8d40101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.346+0000 7f8d3effd700 1 -- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40107280 msgr2=0x7f8d40100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.346+0000 7f8d3effd700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40107280 0x7f8d40100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.346+0000 7f8d3effd700 1 -- 172.21.15.7:0/2396241788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d38009c40 con 0x7f8d40104950 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.346+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40107280 0x7f8d40100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:08:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.346+0000 7f8d3effd700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 0x7f8d400fffd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d950 tx=0x7f8d3400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.347+0000 7f8d3cff9700 1 -- 172.21.15.7:0/2396241788 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34009930 con 0x7f8d40104950 2024-01-11T18:08:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.347+0000 7f8d3cff9700 1 -- 172.21.15.7:0/2396241788 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d34010460 con 0x7f8d40104950 2024-01-11T18:08:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.347+0000 7f8d3cff9700 1 -- 172.21.15.7:0/2396241788 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400f620 con 0x7f8d40104950 2024-01-11T18:08:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.347+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d401a5cb0 con 0x7f8d40104950 2024-01-11T18:08:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.347+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d401a6080 con 0x7f8d40104950 2024-01-11T18:08:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.353+0000 7f8d3cff9700 1 -- 172.21.15.7:0/2396241788 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f8d340108d0 con 0x7f8d40104950 2024-01-11T18:08:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.357+0000 7f8d3cff9700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8d280a1fd0 0x7f8d280a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.357+0000 7f8d3cff9700 1 -- 172.21.15.7:0/2396241788 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8d340cd6a0 con 0x7f8d40104950 2024-01-11T18:08:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.358+0000 7f8d59ca7b80 1 Processor -- start 2024-01-11T18:08:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.359+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8d280a1fd0 0x7f8d280a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.360+0000 7f8d59ca7b80 1 -- start start 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.360+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.361+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 0x55cef04cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.361+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.361+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef03c5030 con 0x55cef04cf5d0 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.361+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef0428ac0 con 0x55cef04cec10 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.361+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef04263e0 con 0x55cef04cdf20 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43760/0 (socket says 172.21.15.7:43760) 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/2957898014 learned_addr learned my addr 172.21.15.7:0/2957898014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8d280a1fd0 0x7f8d280a4490 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d30005fd0 tx=0x7f8d30009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/2957898014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 msgr2=0x55cef04cf090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 0x55cef04cf090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/2957898014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 msgr2=0x55cef04d5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.362+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/2957898014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cef0296350 con 0x55cef04cdf20 2024-01-11T18:08:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.363+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04ce340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fb0 tx=0x7f8d30014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16ee9b0eba7914c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.363+0000 7f8d25ffb700 1 -- 172.21.15.7:0/2957898014 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30003d10 con 0x55cef04cdf20 2024-01-11T18:08:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.363+0000 7f8d25ffb700 1 -- 172.21.15.7:0/2957898014 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d30003e70 con 0x55cef04cdf20 2024-01-11T18:08:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.364+0000 7f8d25ffb700 1 -- 172.21.15.7:0/2957898014 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30005700 con 0x55cef04cdf20 2024-01-11T18:08:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/2957898014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 msgr2=0x55cef04ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04ce340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fb0 tx=0x7f8d30014000 comp rx=0 tx=0).stop 2024-01-11T18:08:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/2957898014 shutdown_connections 2024-01-11T18:08:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04ce340 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/2957898014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 0x55cef04cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/2957898014 >> 172.21.15.7:0/2957898014 conn(0x55cef0443740 msgr2=0x55cef04c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/2957898014 shutdown_connections 2024-01-11T18:08:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.365+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/2957898014 wait complete. 2024-01-11T18:08:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.366+0000 7f8d59ca7b80 1 -- start start 2024-01-11T18:08:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.366+0000 7f8d59ca7b80 1 Processor -- start 2024-01-11T18:08:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.367+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.367+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.367+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04def90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.368+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef0421cd0 con 0x55cef04cf5d0 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.368+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef0390340 con 0x55cef04cdf20 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.368+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef038ffe0 con 0x55cef04cec10 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.368+0000 7f8d59ca7b80 10 client.0 _ll_register_callbacks cb 0x55cef03c26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.368+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.369+0000 7f8d3effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.369+0000 7f8d3effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04d7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43770/0 (socket says 172.21.15.7:43770) 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.369+0000 7f8d3effd700 1 -- 172.21.15.7:0/3338132500 learned_addr learned my addr 172.21.15.7:0/3338132500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.369+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d3effd700 1 -- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 msgr2=0x55cef04d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d3effd700 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d3effd700 1 -- 172.21.15.7:0/3338132500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 msgr2=0x55cef04def90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d3effd700 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04def90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d3effd700 1 -- 172.21.15.7:0/3338132500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d30017070 con 0x55cef04cec10 2024-01-11T18:08:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d3effd700 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04d7f50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009520 tx=0x7f8d34099bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba4393ee6c51d66 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.370+0000 7f8d1effd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:08:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d1effd700 1 -- 172.21.15.7:0/3338132500 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3409a070 con 0x55cef04cec10 2024-01-11T18:08:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d1effd700 1 -- 172.21.15.7:0/3338132500 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8d3409e1c0 con 0x55cef04cec10 2024-01-11T18:08:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cef04c0d60 con 0x55cef04cec10 2024-01-11T18:08:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d1effd700 1 -- 172.21.15.7:0/3338132500 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d340a2400 con 0x55cef04cec10 2024-01-11T18:08:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d59ca7b80 10 client.26401 fetch_fsmap learned FSMap version 45 2024-01-11T18:08:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55cef0242b00 con 0x55cef04cec10 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d1effd700 1 -- 172.21.15.7:0/3338132500 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cef04c0d60 con 0x55cef04cec10 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d1effd700 1 -- 172.21.15.7:0/3338132500 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d340a1f70 con 0x55cef04cec10 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.371+0000 7f8d59ca7b80 10 client.26401 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.372+0000 7f8d59ca7b80 10 client.26401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.372+0000 7f8d59ca7b80 20 client.26401 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.372+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef04e3770 0x55cef04e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.372+0000 7f8d59ca7b80 4 client.26401 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.372+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cef041ab60 con 0x55cef04e3770 2024-01-11T18:08:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.372+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef04e3770 0x55cef04e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.373+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef04e3770 0x55cef04e5b90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560bb5d6dd70f402 server_cookie=83e788a71bdd481e in_seq=0 out_seq=0 2024-01-11T18:08:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.373+0000 7f8d1effd700 10 client.26401 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d1effd700 1 -- 172.21.15.7:0/3338132500 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cef041ab60 con 0x55cef04e3770 2024-01-11T18:08:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d1effd700 10 client.26401 handle_command_reply: tid=0 2024-01-11T18:08:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 client.26401 shutdown 2024-01-11T18:08:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 msgr2=0x55cef04d7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04d7f50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009520 tx=0x7f8d34099bb0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 shutdown_connections 2024-01-11T18:08:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef04e3770 0x55cef04e5b90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/3338132500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04d7f50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 >> 172.21.15.7:0/3338132500 conn(0x55cef0443740 msgr2=0x55cef04c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.374+0000 7f8d1effd700 0 client.26401 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.375+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 shutdown_connections 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.375+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/3338132500 wait complete. 2024-01-11T18:08:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.375+0000 7f8d59ca7b80 20 client.26401 trim_cache size 0 max 16384 2024-01-11T18:08:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.408+0000 7f8d59ca7b80 1 Processor -- start 2024-01-11T18:08:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.408+0000 7f8d59ca7b80 1 -- start start 2024-01-11T18:08:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.409+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.409+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 0x55cef04def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.410+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.410+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef0298640 con 0x55cef04cf5d0 2024-01-11T18:08:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef04c5560 con 0x55cef04cec10 2024-01-11T18:08:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef04c9c80 con 0x55cef04cdf20 2024-01-11T18:08:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d3f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43786/0 (socket says 172.21.15.7:43786) 2024-01-11T18:08:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/220174209 learned_addr learned my addr 172.21.15.7:0/220174209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.411+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/220174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 msgr2=0x55cef04def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.412+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 0x55cef04def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.412+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/220174209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 msgr2=0x55cef04d6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.412+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.412+0000 7f8d3f7fe700 1 -- 172.21.15.7:0/220174209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cef03e31d0 con 0x55cef04cdf20 2024-01-11T18:08:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.412+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:08:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.421+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04c49b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fb0 tx=0x7f8d3003a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ee06e7f05a1087 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.426+0000 7f8d257fa700 1 -- 172.21.15.7:0/220174209 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3003d040 con 0x55cef04cdf20 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.427+0000 7f8d257fa700 1 -- 172.21.15.7:0/220174209 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d300428e0 con 0x55cef04cdf20 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.427+0000 7f8d257fa700 1 -- 172.21.15.7:0/220174209 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004940 con 0x55cef04cdf20 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/220174209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 msgr2=0x55cef04c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04c49b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fb0 tx=0x7f8d3003a040 comp rx=0 tx=0).stop 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/220174209 shutdown_connections 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cdf20 0x55cef04c49b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04d6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/220174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cec10 0x55cef04def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.428+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/220174209 >> 172.21.15.7:0/220174209 conn(0x55cef0443740 msgr2=0x55cef04c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.429+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/220174209 shutdown_connections 2024-01-11T18:08:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.430+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/220174209 wait complete. 2024-01-11T18:08:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.430+0000 7f8d59ca7b80 1 -- start start 2024-01-11T18:08:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.431+0000 7f8d59ca7b80 1 Processor -- start 2024-01-11T18:08:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.432+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.432+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04dd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.432+0000 7f8d59ca7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.432+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef04de660 con 0x55cef04cf5d0 2024-01-11T18:08:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.432+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef04de7d0 con 0x55cef04cdf20 2024-01-11T18:08:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.432+0000 7f8d59ca7b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef04de990 con 0x55cef04cec10 2024-01-11T18:08:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.433+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.433+0000 7f8d3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04de120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35202/0 (socket says 172.21.15.7:35202) 2024-01-11T18:08:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.433+0000 7f8d59ca7b80 10 client.0 _ll_register_callbacks cb 0x55cef03c26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:08:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.434+0000 7f8d3ffff700 1 -- 172.21.15.7:0/1330577094 learned_addr learned my addr 172.21.15.7:0/1330577094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:08:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.434+0000 7f8d3ffff700 1 -- 172.21.15.7:0/1330577094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 msgr2=0x55cef04dd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.434+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.434+0000 7f8d3ffff700 1 -- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 msgr2=0x55cef04dcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:08:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.434+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.434+0000 7f8d3ffff700 1 -- 172.21.15.7:0/1330577094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cef0296350 con 0x55cef04cf5d0 2024-01-11T18:08:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.435+0000 7f8d3ffff700 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04de120 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009f90 tx=0x7f8d3800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b499ffba826321da server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:08:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.435+0000 7f8d1effd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:08:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.436+0000 7f8d1effd700 1 -- 172.21.15.7:0/1330577094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800e040 con 0x55cef04cf5d0 2024-01-11T18:08:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.437+0000 7f8d1effd700 1 -- 172.21.15.7:0/1330577094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8d380095e0 con 0x55cef04cf5d0 2024-01-11T18:08:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.437+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d30003e70 con 0x55cef04cf5d0 2024-01-11T18:08:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.439+0000 7f8d1effd700 1 -- 172.21.15.7:0/1330577094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d380054c0 con 0x55cef04cf5d0 2024-01-11T18:08:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.439+0000 7f8d59ca7b80 10 client.16173 fetch_fsmap learned FSMap version 45 2024-01-11T18:08:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.439+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55cef029ac70 con 0x55cef04cf5d0 2024-01-11T18:08:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.439+0000 7f8d1effd700 1 -- 172.21.15.7:0/1330577094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d30003e70 con 0x55cef04cf5d0 2024-01-11T18:08:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d1effd700 1 -- 172.21.15.7:0/1330577094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d38023020 con 0x55cef04cf5d0 2024-01-11T18:08:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d59ca7b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:08:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d59ca7b80 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:08:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d59ca7b80 20 client.16173 populate_metadata read hostname 'smithi007' 2024-01-11T18:08:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef053ab00 0x55cef053cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:08:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d59ca7b80 4 client.16173 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:08:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.440+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cef041a880 con 0x55cef053ab00 2024-01-11T18:08:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.442+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef053ab00 0x55cef053cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:08:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.443+0000 7f8d3f7fe700 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef053ab00 0x55cef053cf20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e1c6f6da155a94 server_cookie=a4a1d42666597b0b in_seq=0 out_seq=0 2024-01-11T18:08:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.443+0000 7f8d1effd700 10 client.16173 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.596+0000 7f8d1effd700 1 -- 172.21.15.7:0/1330577094 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cef041a880 con 0x55cef053ab00 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.596+0000 7f8d1effd700 10 client.16173 handle_command_reply: tid=0 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.596+0000 7f8d59ca7b80 1 client.16173 shutdown 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 msgr2=0x55cef04de120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04de120 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009f90 tx=0x7f8d3800bcd0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 shutdown_connections 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55cef053ab00 0x55cef053cf20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cef04cf5d0 0x55cef04de120 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cef04cdf20 0x55cef04dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 --2- 172.21.15.7:0/1330577094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cef04cec10 0x55cef04dd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 >> 172.21.15.7:0/1330577094 conn(0x55cef0443740 msgr2=0x55cef04d6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.597+0000 7f8d1effd700 0 client.16173 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.598+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 shutdown_connections 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.598+0000 7f8d59ca7b80 1 -- 172.21.15.7:0/1330577094 wait complete. 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.598+0000 7f8d59ca7b80 20 client.16173 trim_cache size 0 max 16384 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:08:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:08:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8d280a1fd0 msgr2=0x7f8d280a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8d280a1fd0 0x7f8d280a4490 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d30005fd0 tx=0x7f8d30009310 comp rx=0 tx=0).stop 2024-01-11T18:08:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 msgr2=0x7f8d400fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:08:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 0x7f8d400fffd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d950 tx=0x7f8d3400dc60 comp rx=0 tx=0).stop 2024-01-11T18:08:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 shutdown_connections 2024-01-11T18:08:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8d280a1fd0 0x7f8d280a4490 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d40102020 0x7f8d40101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d40107280 0x7f8d40100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 --2- 172.21.15.7:0/2396241788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d40104950 0x7f8d400fffd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:08:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 >> 172.21.15.7:0/2396241788 conn(0x7f8d400fba30 msgr2=0x7f8d40102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:08:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 shutdown_connections 2024-01-11T18:08:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:08:50.599+0000 7f8d45fff700 1 -- 172.21.15.7:0/2396241788 wait complete. 2024-01-11T18:08:51.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:51 smithi007 ceph-mon[33361]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-01-11T18:08:51.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:51 smithi016 ceph-mon[43195]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-01-11T18:08:51.878 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:51 smithi038 ceph-mon[43582]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-01-11T18:08:53.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:53 smithi007 ceph-mon[33361]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-01-11T18:08:53.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:53 smithi016 ceph-mon[43195]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-01-11T18:08:53.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:53 smithi038 ceph-mon[43582]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-01-11T18:08:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 625.0 s, 511.4 tps, lat 200.766 ms stddev 229.999 2024-01-11T18:08:56.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:55 smithi016 ceph-mon[43195]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-01-11T18:08:56.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:55 smithi038 ceph-mon[43582]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-01-11T18:08:56.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:55 smithi007 ceph-mon[33361]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-01-11T18:08:57.490 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:57 smithi016 ceph-mon[43195]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-01-11T18:08:57.523 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:57 smithi038 ceph-mon[43582]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-01-11T18:08:57.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:57 smithi007 ceph-mon[33361]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-01-11T18:08:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 630.0 s, 480.8 tps, lat 203.355 ms stddev 242.746 2024-01-11T18:08:59.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:08:59 smithi007 ceph-mon[33361]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-01-11T18:08:59.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:08:59 smithi016 ceph-mon[43195]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-01-11T18:09:00.060 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:08:59 smithi038 ceph-mon[43582]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-01-11T18:09:01.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:01 smithi007 ceph-mon[33361]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 MiB/s wr, 546 op/s 2024-01-11T18:09:01.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:01 smithi016 ceph-mon[43195]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 MiB/s wr, 546 op/s 2024-01-11T18:09:01.521 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:01 smithi038 ceph-mon[43582]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 MiB/s wr, 546 op/s 2024-01-11T18:09:03.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:03 smithi016 ceph-mon[43195]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 447 op/s 2024-01-11T18:09:03.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:03 smithi038 ceph-mon[43582]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 447 op/s 2024-01-11T18:09:03.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:03 smithi007 ceph-mon[33361]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 447 op/s 2024-01-11T18:09:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 635.0 s, 450.1 tps, lat 215.496 ms stddev 246.455 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 -- 172.21.15.7:0/2022668945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c104950 msgr2=0x7fc92c106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 --2- 172.21.15.7:0/2022668945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c104950 0x7fc92c106d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098f0 tx=0x7fc920009c00 comp rx=0 tx=0).stop 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 -- 172.21.15.7:0/2022668945 shutdown_connections 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 --2- 172.21.15.7:0/2022668945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 0x7fc92c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 --2- 172.21.15.7:0/2022668945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 0x7fc92c104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 --2- 172.21.15.7:0/2022668945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c104950 0x7fc92c106d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.175+0000 7fc931bff700 1 -- 172.21.15.7:0/2022668945 >> 172.21.15.7:0/2022668945 conn(0x7fc92c0fba30 msgr2=0x7fc92c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.176+0000 7fc931bff700 1 -- 172.21.15.7:0/2022668945 shutdown_connections 2024-01-11T18:09:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.176+0000 7fc931bff700 1 -- 172.21.15.7:0/2022668945 wait complete. 2024-01-11T18:09:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.178+0000 7fc931bff700 1 Processor -- start 2024-01-11T18:09:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.178+0000 7fc931bff700 1 -- start start 2024-01-11T18:09:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc931bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 0x7fc92c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc931bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 0x7fc92c197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 0x7fc92c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 0x7fc92c197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44112/0 (socket says 172.21.15.7:44112) 2024-01-11T18:09:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc931bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c19cce0 0x7fc92c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc931bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c19f6a0 con 0x7fc92c107280 2024-01-11T18:09:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc931bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c19f7e0 con 0x7fc92c102020 2024-01-11T18:09:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.179+0000 7fc931bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c19f970 con 0x7fc92c19cce0 2024-01-11T18:09:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.180+0000 7fc92affd700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 0x7fc92c197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.180+0000 7fc92bfff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c19cce0 0x7fc92c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.180+0000 7fc92b7fe700 1 -- 172.21.15.7:0/696986696 learned_addr learned my addr 172.21.15.7:0/696986696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92affd700 1 -- 172.21.15.7:0/696986696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c19cce0 msgr2=0x7fc92c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92affd700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c19cce0 0x7fc92c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92affd700 1 -- 172.21.15.7:0/696986696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 msgr2=0x7fc92c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92affd700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 0x7fc92c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92affd700 1 -- 172.21.15.7:0/696986696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc920009520 con 0x7fc92c107280 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92bfff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c19cce0 0x7fc92c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:09:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.181+0000 7fc92affd700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 0x7fc92c197cd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc920008000 tx=0x7fc92000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.182+0000 7fc928ff9700 1 -- 172.21.15.7:0/696986696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92001c070 con 0x7fc92c107280 2024-01-11T18:09:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.182+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c19fbf0 con 0x7fc92c107280 2024-01-11T18:09:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.182+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c19ffc0 con 0x7fc92c107280 2024-01-11T18:09:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.183+0000 7fc928ff9700 1 -- 172.21.15.7:0/696986696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92000f7d0 con 0x7fc92c107280 2024-01-11T18:09:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.183+0000 7fc928ff9700 1 -- 172.21.15.7:0/696986696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920003900 con 0x7fc92c107280 2024-01-11T18:09:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.193+0000 7fc94595db80 1 Processor -- start 2024-01-11T18:09:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.193+0000 7fc94595db80 1 -- start start 2024-01-11T18:09:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.194+0000 7fc94595db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4dffb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc94595db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4dffc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc94595db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e002e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc94595db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4def2030 con 0x560e4dffaf20 2024-01-11T18:09:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc94595db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4df55ac0 con 0x560e4dffbc10 2024-01-11T18:09:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc94595db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4df533e0 con 0x560e4dffc5d0 2024-01-11T18:09:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4dffb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4dffc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4dffc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44128/0 (socket says 172.21.15.7:44128) 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc92b7fe700 1 -- 172.21.15.7:0/3084524709 learned_addr learned my addr 172.21.15.7:0/3084524709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.195+0000 7fc928ff9700 1 -- 172.21.15.7:0/696986696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc920020070 con 0x7fc92c107280 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.196+0000 7fc92bfff700 1 -- 172.21.15.7:0/3084524709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 msgr2=0x560e4e002e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.196+0000 7fc92bfff700 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e002e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.196+0000 7fc92bfff700 1 -- 172.21.15.7:0/3084524709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 msgr2=0x560e4dffc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.196+0000 7fc92bfff700 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4dffc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.196+0000 7fc92bfff700 1 -- 172.21.15.7:0/3084524709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e4ddc3350 con 0x560e4dffaf20 2024-01-11T18:09:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.196+0000 7fc92bfff700 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4dffb340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc9240076b0 tx=0x7fc9240079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a07c5fbe0c8272 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.197+0000 7fc911ffb700 1 -- 172.21.15.7:0/3084524709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924004bb0 con 0x560e4dffaf20 2024-01-11T18:09:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.197+0000 7fc911ffb700 1 -- 172.21.15.7:0/3084524709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92400f040 con 0x560e4dffaf20 2024-01-11T18:09:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.197+0000 7fc911ffb700 1 -- 172.21.15.7:0/3084524709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924005400 con 0x560e4dffaf20 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.197+0000 7fc94595db80 1 -- 172.21.15.7:0/3084524709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 msgr2=0x560e4dffb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.197+0000 7fc94595db80 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4dffb340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc9240076b0 tx=0x7fc9240079c0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc94595db80 1 -- 172.21.15.7:0/3084524709 shutdown_connections 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc94595db80 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e002e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc94595db80 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4dffb340 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc94595db80 1 --2- 172.21.15.7:0/3084524709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4dffc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc94595db80 1 -- 172.21.15.7:0/3084524709 >> 172.21.15.7:0/3084524709 conn(0x560e4df70740 msgr2=0x560e4dff5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc928ff9700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc9140a63e0 0x7fc9140a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc928ff9700 1 -- 172.21.15.7:0/696986696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc9200ce450 con 0x7fc92c107280 2024-01-11T18:09:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.198+0000 7fc94595db80 1 -- 172.21.15.7:0/3084524709 shutdown_connections 2024-01-11T18:09:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.199+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc9140a63e0 0x7fc9140a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.199+0000 7fc94595db80 1 -- 172.21.15.7:0/3084524709 wait complete. 2024-01-11T18:09:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.200+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc9140a63e0 0x7fc9140a88a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x560e4dff77f0 tx=0x7fc91c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.200+0000 7fc94595db80 1 -- start start 2024-01-11T18:09:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.203+0000 7fc94595db80 1 Processor -- start 2024-01-11T18:09:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4e00d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffbc10 0x560e4e00d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffc5d0 0x560e4e014710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4df4ecd0 con 0x560e4dffbc10 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4debd340 con 0x560e4dffc5d0 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4debcfe0 con 0x560e4dffaf20 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.204+0000 7fc94595db80 10 client.0 _ll_register_callbacks cb 0x560e4deef6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4e00d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4e00d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41960/0 (socket says 172.21.15.7:41960) 2024-01-11T18:09:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 -- 172.21.15.7:0/2782430447 learned_addr learned my addr 172.21.15.7:0/2782430447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92affd700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffbc10 0x560e4e00d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92bfff700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffc5d0 0x560e4e014710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 -- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffc5d0 msgr2=0x560e4e014710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffc5d0 0x560e4e014710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 -- 172.21.15.7:0/2782430447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffbc10 msgr2=0x560e4e00d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffbc10 0x560e4e00d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92b7fe700 1 -- 172.21.15.7:0/2782430447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc924009d80 con 0x560e4dffaf20 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92bfff700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffc5d0 0x560e4e014710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.205+0000 7fc92affd700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffbc10 0x560e4e00d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:09:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.206+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4e00d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc91c004620 tx=0x7fc91c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f48bfce453ace1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.206+0000 7fc90affd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:09:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.206+0000 7fc90affd700 1 -- 172.21.15.7:0/2782430447 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c017070 con 0x560e4dffaf20 2024-01-11T18:09:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.206+0000 7fc90affd700 1 -- 172.21.15.7:0/2782430447 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc91c01f430 con 0x560e4dffaf20 2024-01-11T18:09:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.206+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e4dfedd60 con 0x560e4dffaf20 2024-01-11T18:09:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.207+0000 7fc90affd700 1 -- 172.21.15.7:0/2782430447 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91c003ea0 con 0x560e4dffaf20 2024-01-11T18:09:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.207+0000 7fc94595db80 10 client.26413 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.207+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x560e4dd6fb00 con 0x560e4dffaf20 2024-01-11T18:09:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.208+0000 7fc90affd700 1 -- 172.21.15.7:0/2782430447 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e4dfedd60 con 0x560e4dffaf20 2024-01-11T18:09:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.208+0000 7fc90affd700 1 -- 172.21.15.7:0/2782430447 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc91c033880 con 0x560e4dffaf20 2024-01-11T18:09:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.209+0000 7fc94595db80 10 client.26413 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.209+0000 7fc94595db80 10 client.26413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.209+0000 7fc94595db80 20 client.26413 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.209+0000 7fc94595db80 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e0190e0 0x560e4e01b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.209+0000 7fc94595db80 4 client.26413 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:09:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.209+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e4df47b60 con 0x560e4e0190e0 2024-01-11T18:09:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.215+0000 7fc92affd700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e0190e0 0x560e4e01b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.218+0000 7fc90affd700 10 client.26413 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.218+0000 7fc92affd700 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e0190e0 0x560e4e01b500 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa54264a4beeb1f server_cookie=e497adfdb1e4c064 in_seq=0 out_seq=0 2024-01-11T18:09:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.219+0000 7fc90affd700 1 -- 172.21.15.7:0/2782430447 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e4df47b60 con 0x560e4e0190e0 2024-01-11T18:09:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.219+0000 7fc90affd700 10 client.26413 handle_command_reply: tid=0 2024-01-11T18:09:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.219+0000 7fc94595db80 1 client.26413 shutdown 2024-01-11T18:09:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 msgr2=0x560e4e00d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4e00d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc91c004620 tx=0x7fc91c0047c0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 shutdown_connections 2024-01-11T18:09:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e0190e0 0x560e4e01b500 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4e00d190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffc5d0 0x560e4e014710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 --2- 172.21.15.7:0/2782430447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffbc10 0x560e4e00d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 >> 172.21.15.7:0/2782430447 conn(0x560e4df70740 msgr2=0x560e4dff57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.222+0000 7fc90affd700 0 client.26413 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.223+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 shutdown_connections 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.223+0000 7fc94595db80 1 -- 172.21.15.7:0/2782430447 wait complete. 2024-01-11T18:09:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.223+0000 7fc94595db80 20 client.26413 trim_cache size 0 max 16384 2024-01-11T18:09:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 Processor -- start 2024-01-11T18:09:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 -- start start 2024-01-11T18:09:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4dfed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e0146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffc5d0 0x560e4e01fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4ddc5640 con 0x560e4dffc5d0 2024-01-11T18:09:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4dffca70 con 0x560e4dffbc10 2024-01-11T18:09:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.291+0000 7fc94595db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4dffb7a0 con 0x560e4dffaf20 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4dfed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4dfed8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41968/0 (socket says 172.21.15.7:41968) 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92b7fe700 1 -- 172.21.15.7:0/3939885394 learned_addr learned my addr 172.21.15.7:0/3939885394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92affd700 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e0146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92affd700 1 -- 172.21.15.7:0/3939885394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 msgr2=0x560e4dfed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92affd700 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4dfed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92affd700 1 -- 172.21.15.7:0/3939885394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffc5d0 msgr2=0x560e4e01fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92affd700 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffc5d0 0x560e4e01fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.292+0000 7fc92affd700 1 -- 172.21.15.7:0/3939885394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e4df101d0 con 0x560e4dffbc10 2024-01-11T18:09:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.293+0000 7fc92affd700 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e0146a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9200cbb90 tx=0x7fc920009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6858995c00181bdf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.293+0000 7fc9117fa700 1 -- 172.21.15.7:0/3939885394 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92009b5b0 con 0x560e4dffbc10 2024-01-11T18:09:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.294+0000 7fc9117fa700 1 -- 172.21.15.7:0/3939885394 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92009aeb0 con 0x560e4dffbc10 2024-01-11T18:09:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.294+0000 7fc9117fa700 1 -- 172.21.15.7:0/3939885394 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92009b5b0 con 0x560e4dffbc10 2024-01-11T18:09:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 -- 172.21.15.7:0/3939885394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 msgr2=0x560e4e0146a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e0146a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9200cbb90 tx=0x7fc920009260 comp rx=0 tx=0).stop 2024-01-11T18:09:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 -- 172.21.15.7:0/3939885394 shutdown_connections 2024-01-11T18:09:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffaf20 0x560e4dfed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffc5d0 0x560e4e01fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 --2- 172.21.15.7:0/3939885394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e0146a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.295+0000 7fc94595db80 1 -- 172.21.15.7:0/3939885394 >> 172.21.15.7:0/3939885394 conn(0x560e4df70740 msgr2=0x560e4e002bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.297+0000 7fc94595db80 1 -- 172.21.15.7:0/3939885394 shutdown_connections 2024-01-11T18:09:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.297+0000 7fc94595db80 1 -- 172.21.15.7:0/3939885394 wait complete. 2024-01-11T18:09:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.297+0000 7fc94595db80 1 -- start start 2024-01-11T18:09:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.297+0000 7fc94595db80 1 Processor -- start 2024-01-11T18:09:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.299+0000 7fc94595db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4e011c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.299+0000 7fc94595db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e012160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.299+0000 7fc94595db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e0130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc94595db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e013610 con 0x560e4dffaf20 2024-01-11T18:09:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc94595db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e013780 con 0x560e4dffbc10 2024-01-11T18:09:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc94595db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4e013940 con 0x560e4dffc5d0 2024-01-11T18:09:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e0130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e0130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41980/0 (socket says 172.21.15.7:41980) 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 -- 172.21.15.7:0/1073812390 learned_addr learned my addr 172.21.15.7:0/1073812390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc94595db80 10 client.0 _ll_register_callbacks cb 0x560e4deef6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92affd700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e012160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 -- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 msgr2=0x560e4e012160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e012160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 -- 172.21.15.7:0/1073812390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 msgr2=0x560e4e011c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.300+0000 7fc92bfff700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4e011c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.301+0000 7fc92bfff700 1 -- 172.21.15.7:0/1073812390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c19ffc0 con 0x560e4dffc5d0 2024-01-11T18:09:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.301+0000 7fc92affd700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e012160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.301+0000 7fc92bfff700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e0130d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc9240076b0 tx=0x7fc924009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b036cce0fad4818 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.302+0000 7fc90affd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:09:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.302+0000 7fc90affd700 1 -- 172.21.15.7:0/1073812390 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240172e0 con 0x560e4dffc5d0 2024-01-11T18:09:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.303+0000 7fc90affd700 1 -- 172.21.15.7:0/1073812390 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc924017440 con 0x560e4dffc5d0 2024-01-11T18:09:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.303+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc92400f040 con 0x560e4dffc5d0 2024-01-11T18:09:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.303+0000 7fc90affd700 1 -- 172.21.15.7:0/1073812390 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc924017a10 con 0x560e4dffc5d0 2024-01-11T18:09:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.303+0000 7fc94595db80 10 client.26419 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.303+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x560e4ddc7c70 con 0x560e4dffc5d0 2024-01-11T18:09:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.304+0000 7fc90affd700 1 -- 172.21.15.7:0/1073812390 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92400f040 con 0x560e4dffc5d0 2024-01-11T18:09:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc90affd700 1 -- 172.21.15.7:0/1073812390 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc924017b70 con 0x560e4dffc5d0 2024-01-11T18:09:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc94595db80 10 client.26419 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc94595db80 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc94595db80 20 client.26419 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc94595db80 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e00ab90 0x560e4e00afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc94595db80 4 client.26419 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:09:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.305+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e4df47880 con 0x560e4e00ab90 2024-01-11T18:09:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.306+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e00ab90 0x560e4e00afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.310+0000 7fc92b7fe700 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e00ab90 0x560e4e00afa0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e8439071b8ebea server_cookie=5e5b4c95fe964cbe in_seq=0 out_seq=0 2024-01-11T18:09:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.310+0000 7fc90affd700 10 client.26419 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.373+0000 7fc90affd700 1 -- 172.21.15.7:0/1073812390 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e4df47880 con 0x560e4e00ab90 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.373+0000 7fc90affd700 10 client.26419 handle_command_reply: tid=0 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.373+0000 7fc94595db80 1 client.26419 shutdown 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 msgr2=0x560e4e0130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e0130d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc9240076b0 tx=0x7fc924009440 comp rx=0 tx=0).stop 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 shutdown_connections 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x560e4e00ab90 0x560e4e00afa0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e4dffc5d0 0x560e4e0130d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e4dffaf20 0x560e4e011c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 --2- 172.21.15.7:0/1073812390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e4dffbc10 0x560e4e012160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 >> 172.21.15.7:0/1073812390 conn(0x560e4df70740 msgr2=0x560e4e01fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc90affd700 0 client.26419 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 shutdown_connections 2024-01-11T18:09:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.374+0000 7fc94595db80 1 -- 172.21.15.7:0/1073812390 wait complete. 2024-01-11T18:09:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.375+0000 7fc94595db80 20 client.26419 trim_cache size 0 max 16384 2024-01-11T18:09:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:09:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.375+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc9140a63e0 msgr2=0x7fc9140a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.376+0000 7fc931bff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc9140a63e0 0x7fc9140a88a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x560e4dff77f0 tx=0x7fc91c009310 comp rx=0 tx=0).stop 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.376+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 msgr2=0x7fc92c197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.376+0000 7fc931bff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 0x7fc92c197cd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc920008000 tx=0x7fc92000b790 comp rx=0 tx=0).stop 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.376+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 shutdown_connections 2024-01-11T18:09:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.376+0000 7fc931bff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc9140a63e0 0x7fc9140a88a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.376+0000 7fc931bff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c19cce0 0x7fc92c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.377+0000 7fc931bff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92c102020 0x7fc92c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.377+0000 7fc931bff700 1 --2- 172.21.15.7:0/696986696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc92c107280 0x7fc92c197cd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.377+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 >> 172.21.15.7:0/696986696 conn(0x7fc92c0fba30 msgr2=0x7fc92c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.377+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 shutdown_connections 2024-01-11T18:09:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:04.377+0000 7fc931bff700 1 -- 172.21.15.7:0/696986696 wait complete. 2024-01-11T18:09:06.062 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:05 smithi007 ceph-mon[33361]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-01-11T18:09:06.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:05 smithi016 ceph-mon[43195]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-01-11T18:09:06.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:05 smithi038 ceph-mon[43582]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-01-11T18:09:07.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:07 smithi007 ceph-mon[33361]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 521 op/s 2024-01-11T18:09:07.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:07 smithi016 ceph-mon[43195]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 521 op/s 2024-01-11T18:09:07.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:07 smithi038 ceph-mon[43582]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 521 op/s 2024-01-11T18:09:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 640.0 s, 417.8 tps, lat 245.462 ms stddev 301.872 2024-01-11T18:09:09.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:09 smithi007 ceph-mon[33361]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.1 MiB/s wr, 424 op/s 2024-01-11T18:09:09.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:09 smithi016 ceph-mon[43195]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.1 MiB/s wr, 424 op/s 2024-01-11T18:09:09.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:09 smithi038 ceph-mon[43582]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.1 MiB/s wr, 424 op/s 2024-01-11T18:09:11.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:11 smithi007 ceph-mon[33361]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.1 MiB/s wr, 552 op/s 2024-01-11T18:09:11.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:11 smithi016 ceph-mon[43195]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.1 MiB/s wr, 552 op/s 2024-01-11T18:09:11.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:11 smithi038 ceph-mon[43582]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.1 MiB/s wr, 552 op/s 2024-01-11T18:09:13.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:13 smithi007 ceph-mon[33361]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6.2 MiB/s wr, 474 op/s 2024-01-11T18:09:13.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:13 smithi016 ceph-mon[43195]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6.2 MiB/s wr, 474 op/s 2024-01-11T18:09:13.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:13 smithi038 ceph-mon[43582]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6.2 MiB/s wr, 474 op/s 2024-01-11T18:09:14.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 645.0 s, 456.0 tps, lat 221.151 ms stddev 237.267 2024-01-11T18:09:16.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:15 smithi016 ceph-mon[43195]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.4 MiB/s wr, 467 op/s 2024-01-11T18:09:16.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:15 smithi038 ceph-mon[43582]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.4 MiB/s wr, 467 op/s 2024-01-11T18:09:16.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:15 smithi007 ceph-mon[33361]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.4 MiB/s wr, 467 op/s 2024-01-11T18:09:17.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:17 smithi007 ceph-mon[33361]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 8.5 MiB/s wr, 549 op/s 2024-01-11T18:09:17.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:17 smithi016 ceph-mon[43195]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 8.5 MiB/s wr, 549 op/s 2024-01-11T18:09:17.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:17 smithi038 ceph-mon[43582]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 8.5 MiB/s wr, 549 op/s 2024-01-11T18:09:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.630+0000 7fad531ff700 1 -- 172.21.15.7:0/2611912177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 msgr2=0x7fad4c104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.630+0000 7fad531ff700 1 --2- 172.21.15.7:0/2611912177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c104410 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad44009f90 tx=0x7fad4400b330 comp rx=0 tx=0).stop 2024-01-11T18:09:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.631+0000 7fad531ff700 1 -- 172.21.15.7:0/2611912177 shutdown_connections 2024-01-11T18:09:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.631+0000 7fad531ff700 1 --2- 172.21.15.7:0/2611912177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.631+0000 7fad531ff700 1 --2- 172.21.15.7:0/2611912177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c104410 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.631+0000 7fad531ff700 1 --2- 172.21.15.7:0/2611912177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4c104950 0x7fad4c106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.631+0000 7fad531ff700 1 -- 172.21.15.7:0/2611912177 >> 172.21.15.7:0/2611912177 conn(0x7fad4c0fba30 msgr2=0x7fad4c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.632+0000 7fad531ff700 1 -- 172.21.15.7:0/2611912177 shutdown_connections 2024-01-11T18:09:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.632+0000 7fad531ff700 1 -- 172.21.15.7:0/2611912177 wait complete. 2024-01-11T18:09:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.633+0000 7fad531ff700 1 Processor -- start 2024-01-11T18:09:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 -- start start 2024-01-11T18:09:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4c104950 0x7fad4c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c1a12d0 con 0x7fad4c104950 2024-01-11T18:09:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c1a1410 con 0x7fad4c102020 2024-01-11T18:09:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.634+0000 7fad531ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c1a15a0 con 0x7fad4c107280 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:53746/0 (socket says 172.21.15.7:53746) 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 -- 172.21.15.7:0/3298545645 learned_addr learned my addr 172.21.15.7:0/3298545645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad50f9b700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 -- 172.21.15.7:0/3298545645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 msgr2=0x7fad4c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 -- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4c104950 msgr2=0x7fad4c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4c104950 0x7fad4c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 -- 172.21.15.7:0/3298545645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad44009c40 con 0x7fad4c107280 2024-01-11T18:09:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad50f9b700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:09:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.635+0000 7fad5179c700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1a0d00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e970 tx=0x7fad4000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.639+0000 7fad49ffb700 1 -- 172.21.15.7:0/3298545645 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000cbd0 con 0x7fad4c107280 2024-01-11T18:09:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.639+0000 7fad49ffb700 1 -- 172.21.15.7:0/3298545645 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4000ee50 con 0x7fad4c107280 2024-01-11T18:09:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.639+0000 7fad49ffb700 1 -- 172.21.15.7:0/3298545645 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40005690 con 0x7fad4c107280 2024-01-11T18:09:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.639+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c1a1880 con 0x7fad4c107280 2024-01-11T18:09:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.639+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c1a1dc0 con 0x7fad4c107280 2024-01-11T18:09:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.648+0000 7fad49ffb700 1 -- 172.21.15.7:0/3298545645 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fad40004750 con 0x7fad4c107280 2024-01-11T18:09:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.649+0000 7fad66e5bb80 1 Processor -- start 2024-01-11T18:09:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.649+0000 7fad66e5bb80 1 -- start start 2024-01-11T18:09:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.649+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de784fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fdff0 0x55de784fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de783ec030 con 0x55de784fdff0 2024-01-11T18:09:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7844fac0 con 0x55de784fead0 2024-01-11T18:09:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7844d3e0 con 0x55de784fd690 2024-01-11T18:09:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de784fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de784fdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:53762/0 (socket says 172.21.15.7:53762) 2024-01-11T18:09:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad4bfff700 1 -- 172.21.15.7:0/2581153666 learned_addr learned my addr 172.21.15.7:0/2581153666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.650+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fdff0 0x55de784fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad49ffb700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fad340a2020 0x7fad340a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad49ffb700 1 -- 172.21.15.7:0/3298545645 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fad40014070 con 0x7fad4c107280 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad50f9b700 1 -- 172.21.15.7:0/2581153666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 msgr2=0x55de784fdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad50f9b700 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de784fdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad50f9b700 1 -- 172.21.15.7:0/2581153666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 msgr2=0x55de784fef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad50f9b700 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.651+0000 7fad50f9b700 1 -- 172.21.15.7:0/2581153666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de782bd350 con 0x55de784fdff0 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.652+0000 7fad5179c700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fad340a2020 0x7fad340a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.652+0000 7fad50f9b700 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fdff0 0x55de784fe4f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad44000f80 tx=0x7fad44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e427c5e1cdc553d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.652+0000 7fad32ffd700 1 -- 172.21.15.7:0/2581153666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400e070 con 0x55de784fdff0 2024-01-11T18:09:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.652+0000 7fad32ffd700 1 -- 172.21.15.7:0/2581153666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad44004cc0 con 0x55de784fdff0 2024-01-11T18:09:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.652+0000 7fad32ffd700 1 -- 172.21.15.7:0/2581153666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44005370 con 0x55de784fdff0 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.653+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2581153666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fdff0 msgr2=0x55de784fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.653+0000 7fad5179c700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fad340a2020 0x7fad340a44e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e940 tx=0x7fad400050b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.653+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fdff0 0x55de784fe4f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad44000f80 tx=0x7fad44004ad0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.654+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2581153666 shutdown_connections 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.654+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de784fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.654+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.654+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2581153666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fdff0 0x55de784fe4f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.654+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2581153666 >> 172.21.15.7:0/2581153666 conn(0x55de7846a740 msgr2=0x55de784ef950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.655+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2581153666 shutdown_connections 2024-01-11T18:09:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.655+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2581153666 wait complete. 2024-01-11T18:09:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.656+0000 7fad66e5bb80 1 -- start start 2024-01-11T18:09:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.658+0000 7fad66e5bb80 1 Processor -- start 2024-01-11T18:09:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de78507110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de78507650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de78448cd0 con 0x55de784fead0 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de783b7340 con 0x55de784fdff0 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de783b6fe0 con 0x55de784fd690 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad66e5bb80 10 client.0 _ll_register_callbacks cb 0x55de783e96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de78507650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de78507650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55090/0 (socket says 172.21.15.7:55090) 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.659+0000 7fad50f9b700 1 -- 172.21.15.7:0/4084330653 learned_addr learned my addr 172.21.15.7:0/4084330653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad50f9b700 1 -- 172.21.15.7:0/4084330653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 msgr2=0x55de78507110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad50f9b700 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de78507110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad50f9b700 1 -- 172.21.15.7:0/4084330653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 msgr2=0x55de7850e690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:09:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad50f9b700 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad50f9b700 1 -- 172.21.15.7:0/4084330653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c100190 con 0x55de784fdff0 2024-01-11T18:09:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad50f9b700 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de78507650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad44007a70 tx=0x7fad44007990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f83d96eb46ce55 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad2bfff700 1 -- 172.21.15.7:0/4084330653 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400e040 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.660+0000 7fad2bfff700 1 -- 172.21.15.7:0/4084330653 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fad440052c0 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.661+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de784e7d60 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.661+0000 7fad2bfff700 1 -- 172.21.15.7:0/4084330653 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad44005480 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.661+0000 7fad66e5bb80 10 client.26393 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.662+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55de78269b00 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad2bfff700 1 -- 172.21.15.7:0/4084330653 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad44033030 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad2bfff700 1 -- 172.21.15.7:0/4084330653 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad4400b050 con 0x55de784fdff0 2024-01-11T18:09:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad66e5bb80 10 client.26393 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad66e5bb80 10 client.26393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad66e5bb80 20 client.26393 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de78526090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad66e5bb80 4 client.26393 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:09:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.663+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de78441b60 con 0x55de78523c70 2024-01-11T18:09:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.664+0000 7fad4bfff700 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de78526090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.667+0000 7fad2bfff700 10 client.26393 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.667+0000 7fad4bfff700 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de78526090 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb218c94c2baf81 server_cookie=5acdec399f695a82 in_seq=0 out_seq=0 2024-01-11T18:09:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.668+0000 7fad2bfff700 1 -- 172.21.15.7:0/4084330653 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de78441b60 con 0x55de78523c70 2024-01-11T18:09:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.669+0000 7fad2bfff700 10 client.26393 handle_command_reply: tid=0 2024-01-11T18:09:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.669+0000 7fad66e5bb80 1 client.26393 shutdown 2024-01-11T18:09:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.670+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 msgr2=0x55de78507650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.670+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de78507650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad44007a70 tx=0x7fad44007990 comp rx=0 tx=0).stop 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.671+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 shutdown_connections 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.671+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de78526090 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad2bfff700 0 client.26393 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de78507110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/4084330653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de78507650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 >> 172.21.15.7:0/4084330653 conn(0x55de7846a740 msgr2=0x55de784eef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 shutdown_connections 2024-01-11T18:09:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.672+0000 7fad66e5bb80 1 -- 172.21.15.7:0/4084330653 wait complete. 2024-01-11T18:09:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.673+0000 7fad66e5bb80 20 client.26393 trim_cache size 0 max 16384 2024-01-11T18:09:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.706+0000 7fad66e5bb80 1 Processor -- start 2024-01-11T18:09:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.706+0000 7fad66e5bb80 1 -- start start 2024-01-11T18:09:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fd690 0x55de7851fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 0x55de7850e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de782bf640 con 0x55de784fd690 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de784f0cd0 con 0x55de784fead0 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de78441b60 con 0x55de784fdff0 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 0x55de7850e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 0x55de7850e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:53768/0 (socket says 172.21.15.7:53768) 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 -- 172.21.15.7:0/1978756513 learned_addr learned my addr 172.21.15.7:0/1978756513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad4bfff700 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 -- 172.21.15.7:0/1978756513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 msgr2=0x55de784feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad4bfff700 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fd690 0x55de7851fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784feeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 -- 172.21.15.7:0/1978756513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fd690 msgr2=0x55de7851fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fd690 0x55de7851fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.707+0000 7fad50f9b700 1 -- 172.21.15.7:0/1978756513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7840a1d0 con 0x55de784fdff0 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.708+0000 7fad4bfff700 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fd690 0x55de7851fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.708+0000 7fad50f9b700 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 0x55de7850e650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fad44024dc0 tx=0x7fad44005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3fccdcb64656523 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.708+0000 7fad327fc700 1 -- 172.21.15.7:0/1978756513 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400e070 con 0x55de784fdff0 2024-01-11T18:09:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.708+0000 7fad327fc700 1 -- 172.21.15.7:0/1978756513 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad44033070 con 0x55de784fdff0 2024-01-11T18:09:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.708+0000 7fad327fc700 1 -- 172.21.15.7:0/1978756513 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44005970 con 0x55de784fdff0 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.710+0000 7fad66e5bb80 1 -- 172.21.15.7:0/1978756513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 msgr2=0x55de7850e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.710+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 0x55de7850e650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fad44024dc0 tx=0x7fad44005040 comp rx=0 tx=0).stop 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.710+0000 7fad66e5bb80 1 -- 172.21.15.7:0/1978756513 shutdown_connections 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.710+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fead0 0x55de784feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.710+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fd690 0x55de7851fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.711+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/1978756513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fdff0 0x55de7850e650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.711+0000 7fad66e5bb80 1 -- 172.21.15.7:0/1978756513 >> 172.21.15.7:0/1978756513 conn(0x55de7846a740 msgr2=0x55de784eb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.711+0000 7fad66e5bb80 1 -- 172.21.15.7:0/1978756513 shutdown_connections 2024-01-11T18:09:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.711+0000 7fad66e5bb80 1 -- 172.21.15.7:0/1978756513 wait complete. 2024-01-11T18:09:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.711+0000 7fad66e5bb80 1 -- start start 2024-01-11T18:09:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.712+0000 7fad66e5bb80 1 Processor -- start 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.712+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de7850e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.712+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de7850e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.713+0000 7fad66e5bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.713+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7850b1d0 con 0x55de784fead0 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.713+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7850b340 con 0x55de784fdff0 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.713+0000 7fad66e5bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7850b500 con 0x55de784fd690 2024-01-11T18:09:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.713+0000 7fad66e5bb80 10 client.0 _ll_register_callbacks cb 0x55de783e96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de7850e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de7850e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55116/0 (socket says 172.21.15.7:55116) 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 -- 172.21.15.7:0/2088236615 learned_addr learned my addr 172.21.15.7:0/2088236615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad4bfff700 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de7850e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 -- 172.21.15.7:0/2088236615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 msgr2=0x55de7850e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de7850e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 -- 172.21.15.7:0/2088236615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 msgr2=0x55de7850aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.714+0000 7fad50f9b700 1 -- 172.21.15.7:0/2088236615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de78269b00 con 0x55de784fdff0 2024-01-11T18:09:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.715+0000 7fad50f9b700 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de7850e540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d010 tx=0x7fad44007990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11096d9e3fd21e72 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.715+0000 7fad2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:09:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.715+0000 7fad2bfff700 1 -- 172.21.15.7:0/2088236615 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400e070 con 0x55de784fdff0 2024-01-11T18:09:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.715+0000 7fad2bfff700 1 -- 172.21.15.7:0/2088236615 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fad440052c0 con 0x55de784fdff0 2024-01-11T18:09:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.716+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad44004cc0 con 0x55de784fdff0 2024-01-11T18:09:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.716+0000 7fad2bfff700 1 -- 172.21.15.7:0/2088236615 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad4400e070 con 0x55de784fdff0 2024-01-11T18:09:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.716+0000 7fad66e5bb80 10 client.26399 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.716+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55de782c1c70 con 0x55de784fdff0 2024-01-11T18:09:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.717+0000 7fad2bfff700 1 -- 172.21.15.7:0/2088236615 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad44004cc0 con 0x55de784fdff0 2024-01-11T18:09:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.717+0000 7fad2bfff700 1 -- 172.21.15.7:0/2088236615 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fad4400bd20 con 0x55de784fdff0 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad66e5bb80 10 client.26399 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad66e5bb80 10 client.26399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad66e5bb80 20 client.26399 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de784fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad66e5bb80 4 client.26399 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de78441880 con 0x55de78523c70 2024-01-11T18:09:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.718+0000 7fad4bfff700 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de784fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.719+0000 7fad2bfff700 10 client.26399 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.719+0000 7fad4bfff700 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de784fad20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f9bf4bbb189dff server_cookie=f6c64d084ec16d39 in_seq=0 out_seq=0 2024-01-11T18:09:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.731+0000 7fad2bfff700 1 -- 172.21.15.7:0/2088236615 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de78441880 con 0x55de78523c70 2024-01-11T18:09:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.731+0000 7fad2bfff700 10 client.26399 handle_command_reply: tid=0 2024-01-11T18:09:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.731+0000 7fad66e5bb80 1 client.26399 shutdown 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 msgr2=0x55de7850e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de7850e540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d010 tx=0x7fad44007990 comp rx=0 tx=0).stop 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 shutdown_connections 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55de78523c70 0x55de784fad20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55de784fd690 0x55de7850e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de784fead0 0x55de7850aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 --2- 172.21.15.7:0/2088236615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de784fdff0 0x55de7850e540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad2bfff700 0 client.26399 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 >> 172.21.15.7:0/2088236615 conn(0x55de7846a740 msgr2=0x55de78506660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 shutdown_connections 2024-01-11T18:09:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.732+0000 7fad66e5bb80 1 -- 172.21.15.7:0/2088236615 wait complete. 2024-01-11T18:09:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.733+0000 7fad66e5bb80 20 client.26399 trim_cache size 0 max 16384 2024-01-11T18:09:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:09:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:09:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:09:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:09:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fad340a2020 msgr2=0x7fad340a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fad340a2020 0x7fad340a44e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e940 tx=0x7fad400050b0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 msgr2=0x7fad4c1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1a0d00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fad4000e970 tx=0x7fad4000ed30 comp rx=0 tx=0).stop 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 shutdown_connections 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fad340a2020 0x7fad340a44e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad4c107280 0x7fad4c1a0d00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c102020 0x7fad4c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 --2- 172.21.15.7:0/3298545645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad4c104950 0x7fad4c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 >> 172.21.15.7:0/3298545645 conn(0x7fad4c0fba30 msgr2=0x7fad4c102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 shutdown_connections 2024-01-11T18:09:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:17.735+0000 7fad531ff700 1 -- 172.21.15.7:0/3298545645 wait complete. 2024-01-11T18:09:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 650.0 s, 515.2 tps, lat 196.652 ms stddev 213.989 2024-01-11T18:09:19.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:19 smithi007 ceph-mon[33361]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.9 MiB/s wr, 451 op/s 2024-01-11T18:09:19.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:19 smithi016 ceph-mon[43195]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.9 MiB/s wr, 451 op/s 2024-01-11T18:09:19.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:19 smithi038 ceph-mon[43582]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.9 MiB/s wr, 451 op/s 2024-01-11T18:09:21.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:21 smithi007 ceph-mon[33361]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-01-11T18:09:21.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:21 smithi016 ceph-mon[43195]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-01-11T18:09:21.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:21 smithi038 ceph-mon[43582]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-01-11T18:09:23.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:23 smithi007 ceph-mon[33361]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 467 op/s 2024-01-11T18:09:23.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:23 smithi016 ceph-mon[43195]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 467 op/s 2024-01-11T18:09:23.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:23 smithi038 ceph-mon[43582]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 467 op/s 2024-01-11T18:09:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 655.0 s, 579.4 tps, lat 175.665 ms stddev 231.613 2024-01-11T18:09:26.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:25 smithi016 ceph-mon[43195]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 455 op/s 2024-01-11T18:09:26.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:25 smithi038 ceph-mon[43582]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 455 op/s 2024-01-11T18:09:26.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:25 smithi007 ceph-mon[33361]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 455 op/s 2024-01-11T18:09:27.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:27 smithi007 ceph-mon[33361]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 531 op/s 2024-01-11T18:09:27.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:27 smithi016 ceph-mon[43195]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 531 op/s 2024-01-11T18:09:27.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:27 smithi038 ceph-mon[43582]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 531 op/s 2024-01-11T18:09:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 660.0 s, 511.6 tps, lat 182.470 ms stddev 241.748 2024-01-11T18:09:29.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:29 smithi007 ceph-mon[33361]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 418 op/s 2024-01-11T18:09:29.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:29 smithi016 ceph-mon[43195]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 418 op/s 2024-01-11T18:09:29.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:29 smithi038 ceph-mon[43582]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 418 op/s 2024-01-11T18:09:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.000+0000 7f1e92bff700 1 -- 172.21.15.7:0/3011671531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c1027e0 msgr2=0x7f1e8c102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.000+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3011671531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c1027e0 0x7f1e8c102c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009990 tx=0x7f1e80009ca0 comp rx=0 tx=0).stop 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.001+0000 7f1e92bff700 1 -- 172.21.15.7:0/3011671531 shutdown_connections 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.001+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3011671531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c103180 0x7f1e8c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.001+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3011671531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e8c1015f0 0x7f1e8c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.001+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3011671531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c1027e0 0x7f1e8c102c40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.001+0000 7f1e92bff700 1 -- 172.21.15.7:0/3011671531 >> 172.21.15.7:0/3011671531 conn(0x7f1e8c0fcb60 msgr2=0x7f1e8c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.002+0000 7f1e92bff700 1 -- 172.21.15.7:0/3011671531 shutdown_connections 2024-01-11T18:09:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.002+0000 7f1e92bff700 1 -- 172.21.15.7:0/3011671531 wait complete. 2024-01-11T18:09:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.007+0000 7f1e92bff700 1 Processor -- start 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.007+0000 7f1e92bff700 1 -- start start 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.007+0000 7f1e92bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c1015f0 0x7f1e8c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.007+0000 7f1e92bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e8c1027e0 0x7f1e8c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.007+0000 7f1e92bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 0x7f1e8c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.007+0000 7f1e92bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c1a12b0 con 0x7f1e8c1027e0 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.008+0000 7f1e92bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c06b030 con 0x7f1e8c1015f0 2024-01-11T18:09:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.008+0000 7f1e92bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c06b1c0 con 0x7f1e8c103180 2024-01-11T18:09:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.008+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 0x7f1e8c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 0x7f1e8c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48120/0 (socket says 172.21.15.7:48120) 2024-01-11T18:09:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9099b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c1015f0 0x7f1e8c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 -- 172.21.15.7:0/3076655171 learned_addr learned my addr 172.21.15.7:0/3076655171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 -- 172.21.15.7:0/3076655171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c1015f0 msgr2=0x7f1e8c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c1015f0 0x7f1e8c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 -- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e8c1027e0 msgr2=0x7f1e8c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e8c1027e0 0x7f1e8c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.009+0000 7f1e9119c700 1 -- 172.21.15.7:0/3076655171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800095f0 con 0x7f1e8c103180 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.010+0000 7f1e9119c700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 0x7f1e8c1a0ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ef30 tx=0x7f1e8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.010+0000 7f1e89ffb700 1 -- 172.21.15.7:0/3076655171 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84018070 con 0x7f1e8c103180 2024-01-11T18:09:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.010+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c06b440 con 0x7f1e8c103180 2024-01-11T18:09:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.011+0000 7f1e89ffb700 1 -- 172.21.15.7:0/3076655171 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8400f040 con 0x7f1e8c103180 2024-01-11T18:09:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.011+0000 7f1e89ffb700 1 -- 172.21.15.7:0/3076655171 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84018070 con 0x7f1e8c103180 2024-01-11T18:09:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.011+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c06b9d0 con 0x7f1e8c103180 2024-01-11T18:09:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.018+0000 7f1e89ffb700 1 -- 172.21.15.7:0/3076655171 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1e8400cc20 con 0x7f1e8c103180 2024-01-11T18:09:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.018+0000 7f1ea69bcb80 1 Processor -- start 2024-01-11T18:09:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.020+0000 7f1e89ffb700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1e7c0a2030 0x7f1e7c0a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.020+0000 7f1e89ffb700 1 -- 172.21.15.7:0/3076655171 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e840cd4f0 con 0x7f1e8c103180 2024-01-11T18:09:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.021+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1e7c0a2030 0x7f1e7c0a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.022+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1e7c0a2030 0x7f1e7c0a44f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1e800095c0 tx=0x7f1e80009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.022+0000 7f1ea69bcb80 1 -- start start 2024-01-11T18:09:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d916ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91700090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d91706e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d915f6030 con 0x558d916ffc10 2024-01-11T18:09:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d91659ac0 con 0x558d916fef20 2024-01-11T18:09:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d916573e0 con 0x558d917005d0 2024-01-11T18:09:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d91706e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d91706e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48136/0 (socket says 172.21.15.7:48136) 2024-01-11T18:09:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.023+0000 7f1e9119c700 1 -- 172.21.15.7:0/3373541103 learned_addr learned my addr 172.21.15.7:0/3373541103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.024+0000 7f1e9119c700 1 -- 172.21.15.7:0/3373541103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 msgr2=0x558d916ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.024+0000 7f1e9099b700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d916ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.024+0000 7f1e9119c700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d916ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.025+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91700090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.025+0000 7f1e9119c700 1 -- 172.21.15.7:0/3373541103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 msgr2=0x558d91700090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.025+0000 7f1e9119c700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91700090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.025+0000 7f1e9119c700 1 -- 172.21.15.7:0/3373541103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d914c7350 con 0x558d917005d0 2024-01-11T18:09:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.025+0000 7f1e9099b700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d916ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.025+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91700090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.026+0000 7f1e9119c700 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d91706e60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e840a1d20 tx=0x7f1e840092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8f05b73b45f41c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.026+0000 7f1e76ffd700 1 -- 172.21.15.7:0/3373541103 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8409a650 con 0x558d917005d0 2024-01-11T18:09:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.026+0000 7f1e76ffd700 1 -- 172.21.15.7:0/3373541103 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e84013e80 con 0x558d917005d0 2024-01-11T18:09:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.032+0000 7f1e76ffd700 1 -- 172.21.15.7:0/3373541103 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840cad70 con 0x558d917005d0 2024-01-11T18:09:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3373541103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 msgr2=0x558d91706e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d91706e60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e840a1d20 tx=0x7f1e840092e0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3373541103 shutdown_connections 2024-01-11T18:09:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d91706e60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d916ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3373541103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91700090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.033+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3373541103 >> 172.21.15.7:0/3373541103 conn(0x558d91674740 msgr2=0x558d916f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.034+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3373541103 shutdown_connections 2024-01-11T18:09:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.034+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3373541103 wait complete. 2024-01-11T18:09:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.035+0000 7f1ea69bcb80 1 -- start start 2024-01-11T18:09:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.035+0000 7f1ea69bcb80 1 Processor -- start 2024-01-11T18:09:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.036+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d91707ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.036+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91708020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.036+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d91652cd0 con 0x558d916ffc10 2024-01-11T18:09:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d915c1340 con 0x558d916fef20 2024-01-11T18:09:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d915c0fe0 con 0x558d917005d0 2024-01-11T18:09:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1ea69bcb80 10 client.0 _ll_register_callbacks cb 0x558d915f36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:48144/0 (socket says 172.21.15.7:48144) 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1e9119c700 1 -- 172.21.15.7:0/1677754928 learned_addr learned my addr 172.21.15.7:0/1677754928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1e9119c700 1 -- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 msgr2=0x558d91707ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.037+0000 7f1e9119c700 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d91707ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.038+0000 7f1e9119c700 1 -- 172.21.15.7:0/1677754928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 msgr2=0x558d91708020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.038+0000 7f1e9119c700 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91708020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.038+0000 7f1e9119c700 1 -- 172.21.15.7:0/1677754928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c06b9d0 con 0x558d917005d0 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.039+0000 7f1e9119c700 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170f060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e840969c0 tx=0x7f1e840060e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6371bbc7f1f90c6a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.039+0000 7f1e6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:09:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.039+0000 7f1e6bfff700 1 -- 172.21.15.7:0/1677754928 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84099f50 con 0x558d917005d0 2024-01-11T18:09:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.039+0000 7f1e6bfff700 1 -- 172.21.15.7:0/1677754928 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e8409e0a0 con 0x558d917005d0 2024-01-11T18:09:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.039+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d916f1d60 con 0x558d917005d0 2024-01-11T18:09:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.044+0000 7f1e6bfff700 1 -- 172.21.15.7:0/1677754928 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e840830d0 con 0x558d917005d0 2024-01-11T18:09:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.044+0000 7f1ea69bcb80 10 client.26446 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.044+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x558d91473b00 con 0x558d917005d0 2024-01-11T18:09:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.044+0000 7f1e6bfff700 1 -- 172.21.15.7:0/1677754928 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d916f1d60 con 0x558d917005d0 2024-01-11T18:09:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.044+0000 7f1e6bfff700 1 -- 172.21.15.7:0/1677754928 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1e84098eb0 con 0x558d917005d0 2024-01-11T18:09:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.045+0000 7f1ea69bcb80 10 client.26446 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.045+0000 7f1ea69bcb80 10 client.26446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.045+0000 7f1ea69bcb80 20 client.26446 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.045+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d91725330 0x558d91727720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.045+0000 7f1ea69bcb80 4 client.26446 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:09:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.045+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d9164bb60 con 0x558d91725330 2024-01-11T18:09:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.046+0000 7f1e9099b700 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d91725330 0x558d91727720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.047+0000 7f1e6bfff700 10 client.26446 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.047+0000 7f1e9099b700 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d91725330 0x558d91727720 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0cf5d00789e42b server_cookie=f9558f615b1e2893 in_seq=0 out_seq=0 2024-01-11T18:09:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.048+0000 7f1e6bfff700 1 -- 172.21.15.7:0/1677754928 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d9164bb60 con 0x558d91725330 2024-01-11T18:09:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.049+0000 7f1e6bfff700 10 client.26446 handle_command_reply: tid=0 2024-01-11T18:09:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.049+0000 7f1ea69bcb80 1 client.26446 shutdown 2024-01-11T18:09:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 msgr2=0x558d9170f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170f060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e840969c0 tx=0x7f1e840060e0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 shutdown_connections 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d91725330 0x558d91727720 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170f060 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d91707ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/1677754928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d91708020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 >> 172.21.15.7:0/1677754928 conn(0x558d91674740 msgr2=0x558d916f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1e6bfff700 0 client.26446 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 shutdown_connections 2024-01-11T18:09:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.051+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/1677754928 wait complete. 2024-01-11T18:09:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.052+0000 7f1ea69bcb80 20 client.26446 trim_cache size 0 max 16384 2024-01-11T18:09:31.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.086+0000 7f1ea69bcb80 1 Processor -- start 2024-01-11T18:09:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.086+0000 7f1ea69bcb80 1 -- start start 2024-01-11T18:09:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 0x558d917107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d916ffc10 0x558d9170f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 0x558d9170fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d914c9640 con 0x558d916fef20 2024-01-11T18:09:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d917009b0 con 0x558d917005d0 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1ea69bcb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d916f6560 con 0x558d916ffc10 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 0x558d9170fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1e9099b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 0x558d917107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1e9099b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 0x558d917107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43814/0 (socket says 172.21.15.7:43814) 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1e9099b700 1 -- 172.21.15.7:0/911680110 learned_addr learned my addr 172.21.15.7:0/911680110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.088+0000 7f1e9099b700 1 -- 172.21.15.7:0/911680110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d916ffc10 msgr2=0x558d9170f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.088+0000 7f1e9099b700 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d916ffc10 0x558d9170f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.088+0000 7f1e9099b700 1 -- 172.21.15.7:0/911680110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 msgr2=0x558d9170fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.087+0000 7f1e9119c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 0x558d9170fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41870/0 (socket says 172.21.15.7:41870) 2024-01-11T18:09:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.088+0000 7f1e9099b700 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 0x558d9170fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.088+0000 7f1e9099b700 1 -- 172.21.15.7:0/911680110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d916141d0 con 0x558d916fef20 2024-01-11T18:09:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.088+0000 7f1e9119c700 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 0x558d9170fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.089+0000 7f1e9099b700 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 0x558d917107a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e780097f0 tx=0x7f1e78009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0b17cb686502b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.089+0000 7f1e767fc700 1 -- 172.21.15.7:0/911680110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780179d0 con 0x558d916fef20 2024-01-11T18:09:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.089+0000 7f1e767fc700 1 -- 172.21.15.7:0/911680110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e780190b0 con 0x558d916fef20 2024-01-11T18:09:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.089+0000 7f1e767fc700 1 -- 172.21.15.7:0/911680110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780226d0 con 0x558d916fef20 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.090+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/911680110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 msgr2=0x558d917107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.090+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 0x558d917107a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e780097f0 tx=0x7f1e78009190 comp rx=0 tx=0).stop 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/911680110 shutdown_connections 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d917005d0 0x558d9170fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916fef20 0x558d917107a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/911680110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d916ffc10 0x558d9170f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/911680110 >> 172.21.15.7:0/911680110 conn(0x558d91674740 msgr2=0x558d916f59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/911680110 shutdown_connections 2024-01-11T18:09:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.091+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/911680110 wait complete. 2024-01-11T18:09:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.092+0000 7f1ea69bcb80 1 -- start start 2024-01-11T18:09:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.092+0000 7f1ea69bcb80 1 Processor -- start 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d9170c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d9170c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1ea69bcb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1e9099b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d9170c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1e9099b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d9170c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41878/0 (socket says 172.21.15.7:41878) 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1e9099b700 1 -- 172.21.15.7:0/3124385635 learned_addr learned my addr 172.21.15.7:0/3124385635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9170dc90 con 0x558d916ffc10 2024-01-11T18:09:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9170de00 con 0x558d916fef20 2024-01-11T18:09:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.093+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9170dfc0 con 0x558d917005d0 2024-01-11T18:09:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d9170c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1ea69bcb80 10 client.0 _ll_register_callbacks cb 0x558d915f36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e9099b700 1 -- 172.21.15.7:0/3124385635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 msgr2=0x558d9170d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e9099b700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e9099b700 1 -- 172.21.15.7:0/3124385635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 msgr2=0x558d9170c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e9099b700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d9170c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e9099b700 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e78019e70 con 0x558d916fef20 2024-01-11T18:09:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.094+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d9170c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:09:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.095+0000 7f1e9099b700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d9170c200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800a010 tx=0x7f1e78019360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8b33d4f1c5c822 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.095+0000 7f1e6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:09:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.110+0000 7f1e6bfff700 1 -- 172.21.15.7:0/3124385635 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78026040 con 0x558d916fef20 2024-01-11T18:09:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.110+0000 7f1e6bfff700 1 -- 172.21.15.7:0/3124385635 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1e780177d0 con 0x558d916fef20 2024-01-11T18:09:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.110+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e84013e80 con 0x558d916fef20 2024-01-11T18:09:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.114+0000 7f1e6bfff700 1 -- 172.21.15.7:0/3124385635 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e78019600 con 0x558d916fef20 2024-01-11T18:09:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.114+0000 7f1ea69bcb80 10 client.26408 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.114+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x558d914cbc70 con 0x558d916fef20 2024-01-11T18:09:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.114+0000 7f1e6bfff700 1 -- 172.21.15.7:0/3124385635 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e78039020 con 0x558d916fef20 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1e6bfff700 1 -- 172.21.15.7:0/3124385635 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1e78006610 con 0x558d916fef20 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1ea69bcb80 10 client.26408 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1ea69bcb80 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1ea69bcb80 20 client.26408 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d9177bb70 0x558d9177df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1ea69bcb80 4 client.26408 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:09:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.115+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d9164b880 con 0x558d9177bb70 2024-01-11T18:09:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.116+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d9177bb70 0x558d9177df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.117+0000 7f1e8bfff700 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d9177bb70 0x558d9177df60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ba675fa0c8797d server_cookie=74e6b6deda46b99e in_seq=0 out_seq=0 2024-01-11T18:09:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.117+0000 7f1e6bfff700 10 client.26408 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.229+0000 7f1e6bfff700 1 -- 172.21.15.7:0/3124385635 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d9164b880 con 0x558d9177bb70 2024-01-11T18:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.230+0000 7f1e6bfff700 10 client.26408 handle_command_reply: tid=0 2024-01-11T18:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.230+0000 7f1ea69bcb80 1 client.26408 shutdown 2024-01-11T18:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.230+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 msgr2=0x558d9170c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.230+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d9170c200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800a010 tx=0x7f1e78019360 comp rx=0 tx=0).stop 2024-01-11T18:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.230+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 shutdown_connections 2024-01-11T18:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.230+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558d9177bb70 0x558d9177df60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.231+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558d917005d0 0x558d9170d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.231+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d916fef20 0x558d9170c200 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800a010 tx=0x7f1e78019360 comp rx=0 tx=0).stop 2024-01-11T18:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.231+0000 7f1ea69bcb80 1 --2- 172.21.15.7:0/3124385635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d916ffc10 0x558d9170c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.231+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 >> 172.21.15.7:0/3124385635 conn(0x558d91674740 msgr2=0x558d9170fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.231+0000 7f1e6bfff700 0 client.26408 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.231+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 shutdown_connections 2024-01-11T18:09:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.232+0000 7f1ea69bcb80 1 -- 172.21.15.7:0/3124385635 wait complete. 2024-01-11T18:09:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.232+0000 7f1ea69bcb80 20 client.26408 trim_cache size 0 max 16384 2024-01-11T18:09:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:09:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:09:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:09:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:09:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.233+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1e7c0a2030 msgr2=0x7f1e7c0a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.233+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1e7c0a2030 0x7f1e7c0a44f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1e800095c0 tx=0x7f1e80009530 comp rx=0 tx=0).stop 2024-01-11T18:09:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.233+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 msgr2=0x7f1e8c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.233+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 0x7f1e8c1a0ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ef30 tx=0x7f1e8400c3a0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.234+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 shutdown_connections 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.234+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1e7c0a2030 0x7f1e7c0a44f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.234+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e8c103180 0x7f1e8c1a0ce0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.234+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c1015f0 0x7f1e8c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.234+0000 7f1e92bff700 1 --2- 172.21.15.7:0/3076655171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e8c1027e0 0x7f1e8c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.234+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 >> 172.21.15.7:0/3076655171 conn(0x7f1e8c0fcb60 msgr2=0x7f1e8c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.235+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 shutdown_connections 2024-01-11T18:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:31.235+0000 7f1e92bff700 1 -- 172.21.15.7:0/3076655171 wait complete. 2024-01-11T18:09:31.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:31 smithi007 ceph-mon[33361]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 526 op/s 2024-01-11T18:09:31.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:31 smithi016 ceph-mon[43195]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 526 op/s 2024-01-11T18:09:31.528 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:31 smithi038 ceph-mon[43582]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 526 op/s 2024-01-11T18:09:33.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:33 smithi007 ceph-mon[33361]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 472 op/s 2024-01-11T18:09:33.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:33 smithi016 ceph-mon[43195]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 472 op/s 2024-01-11T18:09:33.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:33 smithi038 ceph-mon[43582]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 472 op/s 2024-01-11T18:09:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 665.0 s, 487.6 tps, lat 214.826 ms stddev 249.095 2024-01-11T18:09:36.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:35 smithi016 ceph-mon[43195]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 452 op/s 2024-01-11T18:09:36.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:35 smithi038 ceph-mon[43582]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 452 op/s 2024-01-11T18:09:36.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:35 smithi007 ceph-mon[33361]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 452 op/s 2024-01-11T18:09:37.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:37 smithi007 ceph-mon[33361]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 MiB/s wr, 528 op/s 2024-01-11T18:09:37.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:37 smithi016 ceph-mon[43195]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 MiB/s wr, 528 op/s 2024-01-11T18:09:37.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:37 smithi038 ceph-mon[43582]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 MiB/s wr, 528 op/s 2024-01-11T18:09:39.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 670.0 s, 594.6 tps, lat 167.870 ms stddev 214.706 2024-01-11T18:09:39.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:39 smithi016 ceph-mon[43195]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 414 op/s 2024-01-11T18:09:39.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:39 smithi038 ceph-mon[43582]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 414 op/s 2024-01-11T18:09:39.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:39 smithi007 ceph-mon[33361]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 414 op/s 2024-01-11T18:09:41.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:41 smithi007 ceph-mon[33361]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 492 op/s 2024-01-11T18:09:41.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:41 smithi016 ceph-mon[43195]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 492 op/s 2024-01-11T18:09:41.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:41 smithi038 ceph-mon[43582]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 492 op/s 2024-01-11T18:09:43.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:43 smithi016 ceph-mon[43195]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-01-11T18:09:43.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:43 smithi038 ceph-mon[43582]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-01-11T18:09:43.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:43 smithi007 ceph-mon[33361]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-01-11T18:09:44.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 675.0 s, 499.0 tps, lat 204.578 ms stddev 261.239 2024-01-11T18:09:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/442555944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a880ffe10 msgr2=0x7f8a88102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/442555944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a880ffe10 0x7f8a88102240 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009f90 tx=0x7f8a7800b330 comp rx=0 tx=0).stop 2024-01-11T18:09:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/442555944 shutdown_connections 2024-01-11T18:09:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/442555944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88105150 0x7f8a881055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/442555944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a880ffe10 0x7f8a88102240 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009f90 tx=0x7f8a7800b330 comp rx=0 tx=0).stop 2024-01-11T18:09:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/442555944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a88102780 0x7f8a88104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.528+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/442555944 >> 172.21.15.7:0/442555944 conn(0x7f8a880fba30 msgr2=0x7f8a880fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.532+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/442555944 shutdown_connections 2024-01-11T18:09:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.532+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/442555944 wait complete. 2024-01-11T18:09:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.534+0000 7f8a8d5ff700 1 Processor -- start 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.534+0000 7f8a8d5ff700 1 -- start start 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.534+0000 7f8a8d5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 0x7f8a8819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a8d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a88105150 0x7f8a8819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a8d5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a881a3450 0x7f8a881a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a8d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a881a5e10 con 0x7f8a88105150 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a8d5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8806b030 con 0x7f8a881a3450 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a8d5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8806b1c0 con 0x7f8a88102780 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 0x7f8a8819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 0x7f8a8819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59186/0 (socket says 172.21.15.7:59186) 2024-01-11T18:09:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 -- 172.21.15.7:0/45909945 learned_addr learned my addr 172.21.15.7:0/45909945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a88105150 0x7f8a8819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 -- 172.21.15.7:0/45909945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a881a3450 msgr2=0x7f8a881a5840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a881a3450 0x7f8a881a5840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 -- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a88105150 msgr2=0x7f8a8819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a88105150 0x7f8a8819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a86ffd700 1 -- 172.21.15.7:0/45909945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009c40 con 0x7f8a88102780 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.535+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a88105150 0x7f8a8819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.536+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 0x7f8a8819deb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a78006070 tx=0x7f8a78004d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.537+0000 7f8a84ff9700 1 -- 172.21.15.7:0/45909945 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800e040 con 0x7f8a88102780 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.537+0000 7f8a84ff9700 1 -- 172.21.15.7:0/45909945 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7800be00 con 0x7f8a88102780 2024-01-11T18:09:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.537+0000 7f8a84ff9700 1 -- 172.21.15.7:0/45909945 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7801b6c0 con 0x7f8a88102780 2024-01-11T18:09:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.537+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8806b440 con 0x7f8a88102780 2024-01-11T18:09:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.537+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8806b8a0 con 0x7f8a88102780 2024-01-11T18:09:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.549+0000 7f8a84ff9700 1 -- 172.21.15.7:0/45909945 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f8a7801b820 con 0x7f8a88102780 2024-01-11T18:09:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.552+0000 7f8a84ff9700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8a740a1f50 0x7f8a740a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.553+0000 7f8a84ff9700 1 -- 172.21.15.7:0/45909945 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8a780cec50 con 0x7f8a88102780 2024-01-11T18:09:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.554+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8a740a1f50 0x7f8a740a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.555+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8a740a1f50 0x7f8a740a4410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a70005950 tx=0x7f8a700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.556+0000 7f8aa13c8b80 1 Processor -- start 2024-01-11T18:09:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.557+0000 7f8aa13c8b80 1 -- start start 2024-01-11T18:09:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.559+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.560+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 0x555eaaa9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.560+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8a7e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 0x555eaaa9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8a7e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 0x555eaaa9a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:59192/0 (socket says 172.21.15.7:59192) 2024-01-11T18:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8a877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8a877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50460/0 (socket says 172.21.15.7:50460) 2024-01-11T18:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/2813575437 learned_addr learned my addr 172.21.15.7:0/2813575437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa990030 con 0x555eaaa98f20 2024-01-11T18:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa9f3ac0 con 0x555eaaa9a5d0 2024-01-11T18:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.561+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa9f13e0 con 0x555eaaa99c10 2024-01-11T18:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.562+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.565+0000 7f8a877fe700 1 -- 172.21.15.7:0/2813575437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 msgr2=0x555eaaa9a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.565+0000 7f8a877fe700 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 0x555eaaa9a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.565+0000 7f8a877fe700 1 -- 172.21.15.7:0/2813575437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 msgr2=0x555eaaa99340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.565+0000 7f8a877fe700 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa99340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.565+0000 7f8a877fe700 1 -- 172.21.15.7:0/2813575437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eaa861350 con 0x555eaaa9a5d0 2024-01-11T18:09:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.565+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 0x555eaaa9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:09:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.567+0000 7f8a877fe700 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa0e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009a90 tx=0x7f8a80009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01393c60a4a5ebf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.569+0000 7f8a7d7fa700 1 -- 172.21.15.7:0/2813575437 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80004230 con 0x555eaaa9a5d0 2024-01-11T18:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.569+0000 7f8a7d7fa700 1 -- 172.21.15.7:0/2813575437 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a80004cf0 con 0x555eaaa9a5d0 2024-01-11T18:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.569+0000 7f8a7d7fa700 1 -- 172.21.15.7:0/2813575437 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80005700 con 0x555eaaa9a5d0 2024-01-11T18:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 msgr2=0x555eaaaa0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa0e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009a90 tx=0x7f8a80009da0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 shutdown_connections 2024-01-11T18:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa0e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2813575437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa99c10 0x555eaaa9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.570+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 >> 172.21.15.7:0/2813575437 conn(0x555eaaa0e740 msgr2=0x555eaaa93950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.571+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 shutdown_connections 2024-01-11T18:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.572+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2813575437 wait complete. 2024-01-11T18:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.572+0000 7f8aa13c8b80 1 -- start start 2024-01-11T18:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.573+0000 7f8aa13c8b80 1 Processor -- start 2024-01-11T18:09:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa98f20 0x555eaaaa2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa99c10 0x555eaaaa2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa9eccd0 con 0x555eaaa99c10 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa95b340 con 0x555eaaa9a5d0 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa95afe0 con 0x555eaaa98f20 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8a877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8a877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50470/0 (socket says 172.21.15.7:50470) 2024-01-11T18:09:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.576+0000 7f8a877fe700 1 -- 172.21.15.7:0/559068555 learned_addr learned my addr 172.21.15.7:0/559068555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.577+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa99c10 0x555eaaaa2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.577+0000 7f8aa13c8b80 10 client.0 _ll_register_callbacks cb 0x555eaa98d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.578+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/559068555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa98f20 msgr2=0x555eaaaa2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.578+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa98f20 0x555eaaaa2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.578+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 msgr2=0x555eaaaa9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.578+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.578+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/559068555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7000fbe0 con 0x555eaaa99c10 2024-01-11T18:09:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.579+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa99c10 0x555eaaaa2f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x555eaaa957f0 tx=0x7f8a70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ff2feaf34a249a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.579+0000 7f8a6e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:09:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.580+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/559068555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70017040 con 0x555eaaa99c10 2024-01-11T18:09:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.580+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/559068555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8a70003b10 con 0x555eaaa99c10 2024-01-11T18:09:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.580+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eaaa8bd60 con 0x555eaaa99c10 2024-01-11T18:09:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.582+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/559068555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a70003cd0 con 0x555eaaa99c10 2024-01-11T18:09:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.582+0000 7f8aa13c8b80 10 client.16206 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.582+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f8a80009770 con 0x555eaaa99c10 2024-01-11T18:09:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.583+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/559068555 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555eaaa8bd60 con 0x555eaaa99c10 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/559068555 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a70011550 con 0x555eaaa99c10 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8aa13c8b80 10 client.16206 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8aa13c8b80 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8aa13c8b80 20 client.16206 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaaaae8c0 0x555eaaab0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8aa13c8b80 4 client.16206 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.584+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eaa9e5b60 con 0x555eaaaae8c0 2024-01-11T18:09:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.585+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaaaae8c0 0x555eaaab0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.587+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaaaae8c0 0x555eaaab0d00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61adae931c8b5bee server_cookie=bdf7cfff3914ae68 in_seq=0 out_seq=0 2024-01-11T18:09:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.587+0000 7f8a6e7fc700 10 client.16206 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.588+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/559068555 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555eaa9e5b60 con 0x555eaaaae8c0 2024-01-11T18:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.588+0000 7f8a6e7fc700 10 client.16206 handle_command_reply: tid=0 2024-01-11T18:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.588+0000 7f8aa13c8b80 1 client.16206 shutdown 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa99c10 msgr2=0x555eaaaa2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa99c10 0x555eaaaa2f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x555eaaa957f0 tx=0x7f8a70004750 comp rx=0 tx=0).stop 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 shutdown_connections 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaaaae8c0 0x555eaaab0d00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa98f20 0x555eaaaa2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/559068555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa99c10 0x555eaaaa2f50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 >> 172.21.15.7:0/559068555 conn(0x555eaaa0e740 msgr2=0x555eaaa92f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.592+0000 7f8a6e7fc700 0 client.16206 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.594+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 shutdown_connections 2024-01-11T18:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.594+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/559068555 wait complete. 2024-01-11T18:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.595+0000 7f8aa13c8b80 20 client.16206 trim_cache size 0 max 16384 2024-01-11T18:09:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.654+0000 7f8aa13c8b80 1 Processor -- start 2024-01-11T18:09:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.654+0000 7f8aa13c8b80 1 -- start start 2024-01-11T18:09:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.656+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa9ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.656+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.656+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaab08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.656+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaa863640 con 0x555eaaa98f20 2024-01-11T18:09:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.656+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaaa945c0 con 0x555eaaa99c10 2024-01-11T18:09:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.656+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaaaa3910 con 0x555eaaa9a5d0 2024-01-11T18:09:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.657+0000 7f8a86ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa9ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.657+0000 7f8a86ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa9ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40612/0 (socket says 172.21.15.7:40612) 2024-01-11T18:09:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.657+0000 7f8a86ffd700 1 -- 172.21.15.7:0/2306039747 learned_addr learned my addr 172.21.15.7:0/2306039747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.657+0000 7f8a877fe700 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaab08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.658+0000 7f8a86ffd700 1 -- 172.21.15.7:0/2306039747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 msgr2=0x555eaaab08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.658+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaab08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.658+0000 7f8a86ffd700 1 -- 172.21.15.7:0/2306039747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 msgr2=0x555eaaaa9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.658+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.658+0000 7f8a86ffd700 1 -- 172.21.15.7:0/2306039747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eaa80db00 con 0x555eaaa98f20 2024-01-11T18:09:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.660+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa9ce40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8a78007900 tx=0x7f8a780bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3fed11eacc85ec server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.661+0000 7f8a7cff9700 1 -- 172.21.15.7:0/2306039747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7809b070 con 0x555eaaa98f20 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.661+0000 7f8a7cff9700 1 -- 172.21.15.7:0/2306039747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7809be10 con 0x555eaaa98f20 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.661+0000 7f8a7cff9700 1 -- 172.21.15.7:0/2306039747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78025e90 con 0x555eaaa98f20 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.662+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2306039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 msgr2=0x555eaaa9ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.662+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa9ce40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8a78007900 tx=0x7f8a780bb040 comp rx=0 tx=0).stop 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2306039747 shutdown_connections 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaab08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaa9ce40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/2306039747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2306039747 >> 172.21.15.7:0/2306039747 conn(0x555eaaa0e740 msgr2=0x555eaaaa1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2306039747 shutdown_connections 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.663+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/2306039747 wait complete. 2024-01-11T18:09:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.664+0000 7f8aa13c8b80 1 -- start start 2024-01-11T18:09:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.664+0000 7f8aa13c8b80 1 Processor -- start 2024-01-11T18:09:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.665+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaaa72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.665+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.665+0000 7f8aa13c8b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.665+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaaaa8d40 con 0x555eaaa98f20 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.665+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaaaa8eb0 con 0x555eaaa99c10 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.665+0000 7f8aa13c8b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eaaaa9070 con 0x555eaaa9a5d0 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8aa13c8b80 10 client.0 _ll_register_callbacks cb 0x555eaa98d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8a7e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8a7e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2024-01-11T18:09:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/1098240388 learned_addr learned my addr 172.21.15.7:0/1098240388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8a877fe700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaaa72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.666+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/1098240388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 msgr2=0x555eaaaa8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.667+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.667+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/1098240388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 msgr2=0x555eaaaa72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.667+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaaa72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.667+0000 7f8a7e7fc700 1 -- 172.21.15.7:0/1098240388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8806b8a0 con 0x555eaaa99c10 2024-01-11T18:09:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.667+0000 7f8a7e7fc700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa77f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a700119b0 tx=0x7f8a7000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72633eb5d9c90030 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.667+0000 7f8a6e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:09:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.669+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/1098240388 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70003d20 con 0x555eaaa99c10 2024-01-11T18:09:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.669+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/1098240388 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8a70017030 con 0x555eaaa99c10 2024-01-11T18:09:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.669+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a80004cf0 con 0x555eaaa99c10 2024-01-11T18:09:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.670+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/1098240388 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a7000b8f0 con 0x555eaaa99c10 2024-01-11T18:09:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.670+0000 7f8aa13c8b80 10 client.26420 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.670+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x555eaa9ae1d0 con 0x555eaaa99c10 2024-01-11T18:09:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.671+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/1098240388 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a70042030 con 0x555eaaa99c10 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.671+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/1098240388 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a7003b020 con 0x555eaaa99c10 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.672+0000 7f8aa13c8b80 10 client.26420 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.672+0000 7f8aa13c8b80 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.672+0000 7f8aa13c8b80 20 client.26420 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.672+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaab05300 0x555eaab07720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.672+0000 7f8aa13c8b80 4 client.26420 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:09:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.672+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555eaa9e5880 con 0x555eaab05300 2024-01-11T18:09:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.675+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaab05300 0x555eaab07720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.676+0000 7f8a86ffd700 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaab05300 0x555eaab07720 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bc5deec4873797 server_cookie=d8dca1c716930775 in_seq=0 out_seq=0 2024-01-11T18:09:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.676+0000 7f8a6e7fc700 10 client.26420 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.875+0000 7f8a6e7fc700 1 -- 172.21.15.7:0/1098240388 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555eaa9e5880 con 0x555eaab05300 2024-01-11T18:09:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.875+0000 7f8a6e7fc700 10 client.26420 handle_command_reply: tid=0 2024-01-11T18:09:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.875+0000 7f8aa13c8b80 1 client.26420 shutdown 2024-01-11T18:09:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.875+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 msgr2=0x555eaaaa77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.875+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa77f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a700119b0 tx=0x7f8a7000bfd0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 shutdown_connections 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555eaab05300 0x555eaab07720 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8a6e7fc700 0 client.26420 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555eaaa9a5d0 0x555eaaaa8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555eaaa98f20 0x555eaaaa72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 --2- 172.21.15.7:0/1098240388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eaaa99c10 0x555eaaaa77f0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a700119b0 tx=0x7f8a7000bfd0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 >> 172.21.15.7:0/1098240388 conn(0x555eaaa0e740 msgr2=0x555eaaab08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 shutdown_connections 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.876+0000 7f8aa13c8b80 1 -- 172.21.15.7:0/1098240388 wait complete. 2024-01-11T18:09:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.877+0000 7f8aa13c8b80 20 client.26420 trim_cache size 0 max 16384 2024-01-11T18:09:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:09:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:09:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:09:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8a740a1f50 msgr2=0x7f8a740a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8a740a1f50 0x7f8a740a4410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a70005950 tx=0x7f8a700058e0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 msgr2=0x7f8a8819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 0x7f8a8819deb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a78006070 tx=0x7f8a78004d90 comp rx=0 tx=0).stop 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 shutdown_connections 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8a740a1f50 0x7f8a740a4410 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a88102780 0x7f8a8819deb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a881a3450 0x7f8a881a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 --2- 172.21.15.7:0/45909945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a88105150 0x7f8a8819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 >> 172.21.15.7:0/45909945 conn(0x7f8a880fba30 msgr2=0x7f8a88103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 shutdown_connections 2024-01-11T18:09:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:44.878+0000 7f8a8d5ff700 1 -- 172.21.15.7:0/45909945 wait complete. 2024-01-11T18:09:46.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:45 smithi016 ceph-mon[43195]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 463 op/s 2024-01-11T18:09:46.238 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:45 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:09:46.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:45 smithi038 ceph-mon[43582]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 463 op/s 2024-01-11T18:09:46.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:45 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:09:46.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:45 smithi007 ceph-mon[33361]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 463 op/s 2024-01-11T18:09:46.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:45 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:09:47.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:47 smithi016 ceph-mon[43195]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 554 op/s 2024-01-11T18:09:47.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:47 smithi038 ceph-mon[43582]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 554 op/s 2024-01-11T18:09:47.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:47 smithi007 ceph-mon[33361]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 554 op/s 2024-01-11T18:09:48.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:09:48.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:09:48.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:09:48.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:09:48.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:09:48.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:09:48.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:09:48.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:09:48.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:09:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 680.0 s, 430.0 tps, lat 223.904 ms stddev 258.708 2024-01-11T18:09:49.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:49 smithi016 ceph-mon[43195]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 444 op/s 2024-01-11T18:09:49.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:49 smithi038 ceph-mon[43582]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 444 op/s 2024-01-11T18:09:49.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:49 smithi007 ceph-mon[33361]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 444 op/s 2024-01-11T18:09:51.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:51 smithi016 ceph-mon[43195]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 542 op/s 2024-01-11T18:09:51.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:51 smithi038 ceph-mon[43582]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 542 op/s 2024-01-11T18:09:51.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:51 smithi007 ceph-mon[33361]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.2 MiB/s wr, 542 op/s 2024-01-11T18:09:53.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:53 smithi016 ceph-mon[43195]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 504 op/s 2024-01-11T18:09:53.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:53 smithi038 ceph-mon[43582]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 504 op/s 2024-01-11T18:09:53.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:53 smithi007 ceph-mon[33361]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.4 MiB/s wr, 504 op/s 2024-01-11T18:09:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 685.0 s, 458.2 tps, lat 218.895 ms stddev 252.220 2024-01-11T18:09:56.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:55 smithi016 ceph-mon[43195]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 452 op/s 2024-01-11T18:09:56.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:55 smithi038 ceph-mon[43582]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 452 op/s 2024-01-11T18:09:56.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:55 smithi007 ceph-mon[33361]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 452 op/s 2024-01-11T18:09:57.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:57 smithi016 ceph-mon[43195]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-01-11T18:09:57.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:57 smithi038 ceph-mon[43582]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-01-11T18:09:57.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:57 smithi007 ceph-mon[33361]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 534 op/s 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.132+0000 7f30151ff700 1 -- 172.21.15.7:0/1424496463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f301010d950 msgr2=0x7f301010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.132+0000 7f30151ff700 1 --2- 172.21.15.7:0/1424496463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f301010d950 0x7f301010ddd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f300800b0a0 tx=0x7f300800b3b0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.132+0000 7f30151ff700 1 -- 172.21.15.7:0/1424496463 shutdown_connections 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.133+0000 7f30151ff700 1 --2- 172.21.15.7:0/1424496463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f301010d950 0x7f301010ddd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.133+0000 7f30151ff700 1 --2- 172.21.15.7:0/1424496463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3010108610 0x7f301010aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.133+0000 7f30151ff700 1 --2- 172.21.15.7:0/1424496463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f301010af80 0x7f301010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.133+0000 7f30151ff700 1 -- 172.21.15.7:0/1424496463 >> 172.21.15.7:0/1424496463 conn(0x7f301006e500 msgr2=0x7f3010070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.134+0000 7f30151ff700 1 -- 172.21.15.7:0/1424496463 shutdown_connections 2024-01-11T18:09:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.135+0000 7f30151ff700 1 -- 172.21.15.7:0/1424496463 wait complete. 2024-01-11T18:09:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.137+0000 7f30151ff700 1 Processor -- start 2024-01-11T18:09:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 -- start start 2024-01-11T18:09:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3010108610 0x7f30101a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301010af80 0x7f30101a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f301010d950 0x7f30101ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101adda0 con 0x7f301010d950 2024-01-11T18:09:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101adee0 con 0x7f301010af80 2024-01-11T18:09:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.138+0000 7f30151ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101ae070 con 0x7f3010108610 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301010af80 0x7f30101a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3010108610 0x7f30101a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301010af80 0x7f30101a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37250/0 (socket says 172.21.15.7:37250) 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300e59c700 1 -- 172.21.15.7:0/2866347375 learned_addr learned my addr 172.21.15.7:0/2866347375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300ed9d700 1 -- 172.21.15.7:0/2866347375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301010af80 msgr2=0x7f30101a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301010af80 0x7f30101a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300ed9d700 1 -- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f301010d950 msgr2=0x7f30101ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f301010d950 0x7f30101ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.139+0000 7f300ed9d700 1 -- 172.21.15.7:0/2866347375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3008009d00 con 0x7f3010108610 2024-01-11T18:09:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.140+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3010108610 0x7f30101a6270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f300000c970 tx=0x7f300000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.140+0000 7f3007fff700 1 -- 172.21.15.7:0/2866347375 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000007b00 con 0x7f3010108610 2024-01-11T18:09:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.140+0000 7f3007fff700 1 -- 172.21.15.7:0/2866347375 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f300000ce60 con 0x7f3010108610 2024-01-11T18:09:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.140+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30101ae350 con 0x7f3010108610 2024-01-11T18:09:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.140+0000 7f3007fff700 1 -- 172.21.15.7:0/2866347375 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000005440 con 0x7f3010108610 2024-01-11T18:09:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.140+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30101ae860 con 0x7f3010108610 2024-01-11T18:09:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.147+0000 7f3007fff700 1 -- 172.21.15.7:0/2866347375 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f30000055a0 con 0x7f3010108610 2024-01-11T18:09:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 Processor -- start 2024-01-11T18:09:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 -- start start 2024-01-11T18:09:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c78e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 0x562f6c78f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78f5d0 0x562f6c795e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c685030 con 0x562f6c78f5d0 2024-01-11T18:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c6e8ac0 con 0x562f6c78df20 2024-01-11T18:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.150+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c6e63e0 con 0x562f6c78ec10 2024-01-11T18:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.151+0000 7f300ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 0x562f6c78f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.151+0000 7f300ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 0x562f6c78f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41384/0 (socket says 172.21.15.7:41384) 2024-01-11T18:09:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.151+0000 7f300ed9d700 1 -- 172.21.15.7:0/2472660991 learned_addr learned my addr 172.21.15.7:0/2472660991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.151+0000 7f300f59e700 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c78e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.151+0000 7f3007fff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2ff40a2020 0x7f2ff40a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.152+0000 7f3007fff700 1 -- 172.21.15.7:0/2866347375 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f30000cd660 con 0x7f3010108610 2024-01-11T18:09:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.152+0000 7f300e59c700 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78f5d0 0x562f6c795e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300f59e700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2ff40a2020 0x7f2ff40a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300ed9d700 1 -- 172.21.15.7:0/2472660991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 msgr2=0x562f6c78e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c78e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300ed9d700 1 -- 172.21.15.7:0/2472660991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78f5d0 msgr2=0x562f6c795e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78f5d0 0x562f6c795e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300ed9d700 1 -- 172.21.15.7:0/2472660991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6c556350 con 0x562f6c78ec10 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.153+0000 7f300e59c700 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78f5d0 0x562f6c795e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.154+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 0x562f6c78f090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f300009b000 tx=0x7f300009acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b12df2aeba7a34df server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.154+0000 7f300f59e700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2ff40a2020 0x7f2ff40a44e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x562f6c78dcc0 tx=0x7f30080099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.154+0000 7f30057fa700 1 -- 172.21.15.7:0/2472660991 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000004e80 con 0x562f6c78ec10 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.154+0000 7f30057fa700 1 -- 172.21.15.7:0/2472660991 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30000a2500 con 0x562f6c78ec10 2024-01-11T18:09:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.154+0000 7f30057fa700 1 -- 172.21.15.7:0/2472660991 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000a2330 con 0x562f6c78ec10 2024-01-11T18:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2472660991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 msgr2=0x562f6c78f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 0x562f6c78f090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f300009b000 tx=0x7f300009acb0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2472660991 shutdown_connections 2024-01-11T18:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78f5d0 0x562f6c795e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c78e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2472660991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78ec10 0x562f6c78f090 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.155+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2472660991 >> 172.21.15.7:0/2472660991 conn(0x562f6c703740 msgr2=0x562f6c788950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.156+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2472660991 shutdown_connections 2024-01-11T18:09:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.156+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2472660991 wait complete. 2024-01-11T18:09:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.157+0000 7f3028fe6b80 1 -- start start 2024-01-11T18:09:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.157+0000 7f3028fe6b80 1 Processor -- start 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.160+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c797a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c797f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c6e1cd0 con 0x562f6c78ec10 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c650340 con 0x562f6c78df20 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c64ffe0 con 0x562f6c78f5d0 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f3028fe6b80 10 client.0 _ll_register_callbacks cb 0x562f6c6826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.161+0000 7f300ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c797a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.162+0000 7f300ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c797a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37264/0 (socket says 172.21.15.7:37264) 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.162+0000 7f300ed9d700 1 -- 172.21.15.7:0/1109882237 learned_addr learned my addr 172.21.15.7:0/1109882237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.162+0000 7f300f59e700 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.163+0000 7f300ed9d700 1 -- 172.21.15.7:0/1109882237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 msgr2=0x562f6c79ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.163+0000 7f300ed9d700 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.163+0000 7f300ed9d700 1 -- 172.21.15.7:0/1109882237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 msgr2=0x562f6c797f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.163+0000 7f300ed9d700 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c797f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.163+0000 7f300ed9d700 1 -- 172.21.15.7:0/1109882237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc00a8f0 con 0x562f6c78df20 2024-01-11T18:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.163+0000 7f300ed9d700 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c797a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f300000c940 tx=0x7f3000004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698c2686ba3baae3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.164+0000 7f2ff27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.164+0000 7f2ff27fc700 1 -- 172.21.15.7:0/1109882237 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300009a560 con 0x562f6c78df20 2024-01-11T18:09:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.164+0000 7f2ff27fc700 1 -- 172.21.15.7:0/1109882237 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f300009a6c0 con 0x562f6c78df20 2024-01-11T18:09:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.164+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f6c780d60 con 0x562f6c78df20 2024-01-11T18:09:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.165+0000 7f2ff27fc700 1 -- 172.21.15.7:0/1109882237 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30000cfb00 con 0x562f6c78df20 2024-01-11T18:09:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.165+0000 7f3028fe6b80 10 client.26429 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.165+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f30101ae860 con 0x562f6c78df20 2024-01-11T18:09:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.166+0000 7f2ff27fc700 1 -- 172.21.15.7:0/1109882237 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30000b6030 con 0x562f6c78df20 2024-01-11T18:09:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f2ff27fc700 1 -- 172.21.15.7:0/1109882237 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3000099fd0 con 0x562f6c78df20 2024-01-11T18:09:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f3028fe6b80 10 client.26429 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f3028fe6b80 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f3028fe6b80 20 client.26429 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7a34b0 0x562f6c7a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f3028fe6b80 4 client.26429 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:09:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.167+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f6c6dab60 con 0x562f6c7a34b0 2024-01-11T18:09:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.168+0000 7f300e59c700 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7a34b0 0x562f6c7a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.168+0000 7f300e59c700 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7a34b0 0x562f6c7a58d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7b15c8cbbd9bcd server_cookie=1c6adf8ce74a248f in_seq=0 out_seq=0 2024-01-11T18:09:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.168+0000 7f2ff27fc700 10 client.26429 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.169+0000 7f2ff27fc700 1 -- 172.21.15.7:0/1109882237 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f6c6dab60 con 0x562f6c7a34b0 2024-01-11T18:09:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.169+0000 7f2ff27fc700 10 client.26429 handle_command_reply: tid=0 2024-01-11T18:09:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.169+0000 7f3028fe6b80 1 client.26429 shutdown 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 msgr2=0x562f6c797a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c797a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f300000c940 tx=0x7f3000004f00 comp rx=0 tx=0).stop 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 shutdown_connections 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7a34b0 0x562f6c7a58d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c797a10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/1109882237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c797f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 >> 172.21.15.7:0/1109882237 conn(0x562f6c703740 msgr2=0x562f6c787f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f2ff27fc700 0 client.26429 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 shutdown_connections 2024-01-11T18:09:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 1 -- 172.21.15.7:0/1109882237 wait complete. 2024-01-11T18:09:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.170+0000 7f3028fe6b80 20 client.26429 trim_cache size 0 max 16384 2024-01-11T18:09:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.204+0000 7f3028fe6b80 1 Processor -- start 2024-01-11T18:09:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.205+0000 7f3028fe6b80 1 -- start start 2024-01-11T18:09:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.206+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78df20 0x562f6c787f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.206+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.206+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78f5d0 0x562f6c7aa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.206+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c558640 con 0x562f6c78ec10 2024-01-11T18:09:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.206+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c79f460 con 0x562f6c78f5d0 2024-01-11T18:09:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.206+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c6dab60 con 0x562f6c78df20 2024-01-11T18:09:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.207+0000 7f300f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78f5d0 0x562f6c7aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.207+0000 7f300f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78f5d0 0x562f6c7aa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37280/0 (socket says 172.21.15.7:37280) 2024-01-11T18:09:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.207+0000 7f300f59e700 1 -- 172.21.15.7:0/2142527200 learned_addr learned my addr 172.21.15.7:0/2142527200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.207+0000 7f300e59c700 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.208+0000 7f300e59c700 1 -- 172.21.15.7:0/2142527200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78df20 msgr2=0x562f6c787f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.209+0000 7f300e59c700 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78df20 0x562f6c787f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.209+0000 7f300e59c700 1 -- 172.21.15.7:0/2142527200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78f5d0 msgr2=0x562f6c7aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.209+0000 7f300e59c700 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78f5d0 0x562f6c7aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.209+0000 7f300e59c700 1 -- 172.21.15.7:0/2142527200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6c502b00 con 0x562f6c78ec10 2024-01-11T18:09:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.211+0000 7f300e59c700 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79ef20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x562f6c789520 tx=0x7f2ffc00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c454bbca1cb00e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.212+0000 7f3004ff9700 1 -- 172.21.15.7:0/2142527200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc015040 con 0x562f6c78ec10 2024-01-11T18:09:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.212+0000 7f3004ff9700 1 -- 172.21.15.7:0/2142527200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ffc017040 con 0x562f6c78ec10 2024-01-11T18:09:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.212+0000 7f3004ff9700 1 -- 172.21.15.7:0/2142527200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004c60 con 0x562f6c78ec10 2024-01-11T18:09:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.213+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2142527200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 msgr2=0x562f6c79ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.213+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79ef20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x562f6c789520 tx=0x7f2ffc00af90 comp rx=0 tx=0).stop 2024-01-11T18:09:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.214+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2142527200 shutdown_connections 2024-01-11T18:09:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.214+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78df20 0x562f6c787f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.214+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78f5d0 0x562f6c7aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.214+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2142527200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79ef20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.214+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2142527200 >> 172.21.15.7:0/2142527200 conn(0x562f6c703740 msgr2=0x562f6c795c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.214+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2142527200 shutdown_connections 2024-01-11T18:09:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.216+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2142527200 wait complete. 2024-01-11T18:09:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.216+0000 7f3028fe6b80 1 -- start start 2024-01-11T18:09:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.217+0000 7f3028fe6b80 1 Processor -- start 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c79c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c79ddb0 con 0x562f6c78ec10 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c79df20 con 0x562f6c78df20 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6c79e0e0 con 0x562f6c78f5d0 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f3028fe6b80 10 client.0 _ll_register_callbacks cb 0x562f6c6826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:09:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.218+0000 7f300e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44322/0 (socket says 172.21.15.7:44322) 2024-01-11T18:09:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 -- 172.21.15.7:0/2149215440 learned_addr learned my addr 172.21.15.7:0/2149215440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:09:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c79c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 -- 172.21.15.7:0/2149215440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 msgr2=0x562f6c79d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 -- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 msgr2=0x562f6c79c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:09:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c79c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.219+0000 7f300e59c700 1 -- 172.21.15.7:0/2149215440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc00ae00 con 0x562f6c78ec10 2024-01-11T18:09:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.220+0000 7f2ff27fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:09:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.220+0000 7f300e59c700 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79c860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc012410 tx=0x7f2ffc012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5751ecbca9d485cd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:09:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.222+0000 7f2ff27fc700 1 -- 172.21.15.7:0/2149215440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc015050 con 0x562f6c78ec10 2024-01-11T18:09:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.222+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30000a2500 con 0x562f6c78ec10 2024-01-11T18:09:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.223+0000 7f2ff27fc700 1 -- 172.21.15.7:0/2149215440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2ffc004020 con 0x562f6c78ec10 2024-01-11T18:09:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.223+0000 7f2ff27fc700 1 -- 172.21.15.7:0/2149215440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ffc015050 con 0x562f6c78ec10 2024-01-11T18:09:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.223+0000 7f3028fe6b80 10 client.16221 fetch_fsmap learned FSMap version 45 2024-01-11T18:09:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.223+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x562f6c6a31d0 con 0x562f6c78ec10 2024-01-11T18:09:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.224+0000 7f2ff27fc700 1 -- 172.21.15.7:0/2149215440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30000a2500 con 0x562f6c78ec10 2024-01-11T18:09:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f2ff27fc700 1 -- 172.21.15.7:0/2149215440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2ffc02a070 con 0x562f6c78ec10 2024-01-11T18:09:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f3028fe6b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:09:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f3028fe6b80 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:09:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f3028fe6b80 20 client.16221 populate_metadata read hostname 'smithi007' 2024-01-11T18:09:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7fa030 0x562f6c7fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:09:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f3028fe6b80 4 client.16221 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:09:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.228+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f6c6da880 con 0x562f6c7fa030 2024-01-11T18:09:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.229+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7fa030 0x562f6c7fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:09:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.229+0000 7f2ff27fc700 10 client.16221 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.229+0000 7f300ed9d700 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7fa030 0x562f6c7fc450 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6047e7c2a3f4a1 server_cookie=434cfcc3599a24ff in_seq=0 out_seq=0 2024-01-11T18:09:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.308+0000 7f2ff27fc700 1 -- 172.21.15.7:0/2149215440 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f6c6da880 con 0x562f6c7fa030 2024-01-11T18:09:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.308+0000 7f2ff27fc700 10 client.16221 handle_command_reply: tid=0 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.309+0000 7f3028fe6b80 1 client.16221 shutdown 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.309+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 msgr2=0x562f6c79c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.309+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79c860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc012410 tx=0x7f2ffc012c60 comp rx=0 tx=0).stop 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 shutdown_connections 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x562f6c7fa030 0x562f6c7fc450 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562f6c78f5d0 0x562f6c79d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f6c78df20 0x562f6c79c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f2ff27fc700 0 client.16221 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 --2- 172.21.15.7:0/2149215440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562f6c78ec10 0x562f6c79c860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 >> 172.21.15.7:0/2149215440 conn(0x562f6c703740 msgr2=0x562f6c7aa230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.310+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 shutdown_connections 2024-01-11T18:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.311+0000 7f3028fe6b80 1 -- 172.21.15.7:0/2149215440 wait complete. 2024-01-11T18:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.311+0000 7f3028fe6b80 20 client.16221 trim_cache size 0 max 16384 2024-01-11T18:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2ff40a2020 msgr2=0x7f2ff40a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2ff40a2020 0x7f2ff40a44e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x562f6c78dcc0 tx=0x7f30080099e0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3010108610 msgr2=0x7f30101a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:09:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3010108610 0x7f30101a6270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f300000c970 tx=0x7f300000cc80 comp rx=0 tx=0).stop 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 shutdown_connections 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f2ff40a2020 0x7f2ff40a44e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3010108610 0x7f30101a6270 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f301010d950 0x7f30101ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 --2- 172.21.15.7:0/2866347375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301010af80 0x7f30101a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.312+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 >> 172.21.15.7:0/2866347375 conn(0x7f301006e500 msgr2=0x7f301010d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:09:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.313+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 shutdown_connections 2024-01-11T18:09:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:09:58.314+0000 7f30151ff700 1 -- 172.21.15.7:0/2866347375 wait complete. 2024-01-11T18:09:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 690.0 s, 516.6 tps, lat 200.048 ms stddev 243.331 2024-01-11T18:09:59.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:09:59 smithi016 ceph-mon[43195]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 393 op/s 2024-01-11T18:09:59.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:09:59 smithi038 ceph-mon[43582]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 393 op/s 2024-01-11T18:09:59.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:09:59 smithi007 ceph-mon[33361]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 393 op/s 2024-01-11T18:10:00.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:00 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T18:09:59.999+0000 7fec6b25b700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:10:01.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:00 smithi016 ceph-mon[43195]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:10:01.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:00 smithi038 ceph-mon[43582]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:10:01.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:00 smithi007 ceph-mon[33361]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-01-11T18:10:02.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:01 smithi016 ceph-mon[43195]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-01-11T18:10:02.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:01 smithi038 ceph-mon[43582]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-01-11T18:10:02.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:01 smithi007 ceph-mon[33361]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-01-11T18:10:03.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:03 smithi016 ceph-mon[43195]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.9 MiB/s wr, 466 op/s 2024-01-11T18:10:03.528 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:03 smithi038 ceph-mon[43582]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.9 MiB/s wr, 466 op/s 2024-01-11T18:10:03.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:03 smithi007 ceph-mon[33361]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.9 MiB/s wr, 466 op/s 2024-01-11T18:10:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 695.0 s, 458.5 tps, lat 212.164 ms stddev 284.791 2024-01-11T18:10:06.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:05 smithi016 ceph-mon[43195]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 460 op/s 2024-01-11T18:10:06.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:05 smithi038 ceph-mon[43582]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 460 op/s 2024-01-11T18:10:06.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:05 smithi007 ceph-mon[33361]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 460 op/s 2024-01-11T18:10:07.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:07 smithi016 ceph-mon[43195]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-01-11T18:10:07.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:07 smithi038 ceph-mon[43582]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-01-11T18:10:07.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:07 smithi007 ceph-mon[33361]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-01-11T18:10:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 700.0 s, 444.0 tps, lat 229.355 ms stddev 267.227 2024-01-11T18:10:09.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:09 smithi016 ceph-mon[43195]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.2 MiB/s wr, 421 op/s 2024-01-11T18:10:09.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:09 smithi038 ceph-mon[43582]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.2 MiB/s wr, 421 op/s 2024-01-11T18:10:09.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:09 smithi007 ceph-mon[33361]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.2 MiB/s wr, 421 op/s 2024-01-11T18:10:11.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:11 smithi016 ceph-mon[43195]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.7 MiB/s wr, 555 op/s 2024-01-11T18:10:11.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:11 smithi038 ceph-mon[43582]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.7 MiB/s wr, 555 op/s 2024-01-11T18:10:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.562+0000 7f8ed29ff700 1 -- 172.21.15.7:0/704746181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1015d0 msgr2=0x7f8ecc1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.562+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/704746181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1019f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009920 tx=0x7f8ebc009c30 comp rx=0 tx=0).stop 2024-01-11T18:10:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 -- 172.21.15.7:0/704746181 shutdown_connections 2024-01-11T18:10:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/704746181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 0x7f8ecc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/704746181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1019f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/704746181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1027c0 0x7f8ecc102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 -- 172.21.15.7:0/704746181 >> 172.21.15.7:0/704746181 conn(0x7f8ecc0fcb60 msgr2=0x7f8ecc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 -- 172.21.15.7:0/704746181 shutdown_connections 2024-01-11T18:10:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.565+0000 7f8ed29ff700 1 -- 172.21.15.7:0/704746181 wait complete. 2024-01-11T18:10:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.567+0000 7f8ed29ff700 1 Processor -- start 2024-01-11T18:10:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.567+0000 7f8ed29ff700 1 -- start start 2024-01-11T18:10:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed29ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed29ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1027c0 0x7f8ecc199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed29ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 0x7f8ecc1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed29ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1a1300 con 0x7f8ecc1027c0 2024-01-11T18:10:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed29ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1a1440 con 0x7f8ecc1015d0 2024-01-11T18:10:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed29ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1a15d0 con 0x7f8ecc103160 2024-01-11T18:10:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1027c0 0x7f8ecc199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51428/0 (socket says 172.21.15.7:51428) 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1345596657 learned_addr learned my addr 172.21.15.7:0/1345596657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.568+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 0x7f8ecc1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.569+0000 7f8ed0f9c700 1 -- 172.21.15.7:0/1345596657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1015d0 msgr2=0x7f8ecc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.569+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.569+0000 7f8ed0f9c700 1 -- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1027c0 msgr2=0x7f8ecc199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.569+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1027c0 0x7f8ecc199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.569+0000 7f8ed0f9c700 1 -- 172.21.15.7:0/1345596657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc009520 con 0x7f8ecc103160 2024-01-11T18:10:11.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:11 smithi007 ceph-mon[33361]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.7 MiB/s wr, 555 op/s 2024-01-11T18:10:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.575+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 0x7f8ecc1a0d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400efc0 tx=0x7f8ec400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.576+0000 7f8ec97fa700 1 -- 172.21.15.7:0/1345596657 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4018070 con 0x7f8ecc103160 2024-01-11T18:10:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.576+0000 7f8ec97fa700 1 -- 172.21.15.7:0/1345596657 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec400f040 con 0x7f8ecc103160 2024-01-11T18:10:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.576+0000 7f8ec97fa700 1 -- 172.21.15.7:0/1345596657 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400ccb0 con 0x7f8ecc103160 2024-01-11T18:10:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.576+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc1a1850 con 0x7f8ecc103160 2024-01-11T18:10:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.576+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc1a1d60 con 0x7f8ecc103160 2024-01-11T18:10:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.585+0000 7f8ee6738b80 1 Processor -- start 2024-01-11T18:10:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.585+0000 7f8ec97fa700 1 -- 172.21.15.7:0/1345596657 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f8ec4003a00 con 0x7f8ecc103160 2024-01-11T18:10:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.585+0000 7f8ee6738b80 1 -- start start 2024-01-11T18:10:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.587+0000 7f8ec97fa700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8eb40a20a0 0x7f8eb40a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1bbd030 con 0x55ecd1cceff0 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1c20ac0 con 0x55ecd1cce690 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1c1e3e0 con 0x55ecd1ccfad0 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.588+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51434/0 (socket says 172.21.15.7:51434) 2024-01-11T18:10:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1192606398 learned_addr learned my addr 172.21.15.7:0/1192606398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1192606398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1ccfad0 msgr2=0x55ecd1ccff50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1192606398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cceff0 msgr2=0x55ecd1ccf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ec97fa700 1 -- 172.21.15.7:0/1345596657 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8ec40cd640 con 0x7f8ecc103160 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1192606398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd1a8e350 con 0x55ecd1cce690 2024-01-11T18:10:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.589+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8eb40a20a0 0x7f8eb40a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.590+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cceab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0094e0 tx=0x7f8ebc009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eeb2f01490bc97d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.590+0000 7f8eb27fc700 1 -- 172.21.15.7:0/1192606398 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00f7e0 con 0x55ecd1cce690 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.590+0000 7f8eb27fc700 1 -- 172.21.15.7:0/1192606398 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ebc01d040 con 0x55ecd1cce690 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.590+0000 7f8eb27fc700 1 -- 172.21.15.7:0/1192606398 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00f7e0 con 0x55ecd1cce690 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.590+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8eb40a20a0 0x7f8eb40a4560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000b6c0 tx=0x7f8ec0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1192606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 msgr2=0x55ecd1cceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cceab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0094e0 tx=0x7f8ebc009f80 comp rx=0 tx=0).stop 2024-01-11T18:10:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1192606398 shutdown_connections 2024-01-11T18:10:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cceab0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1192606398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1192606398 >> 172.21.15.7:0/1192606398 conn(0x55ecd1c3b740 msgr2=0x55ecd1cc0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1192606398 shutdown_connections 2024-01-11T18:10:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.592+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1192606398 wait complete. 2024-01-11T18:10:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.593+0000 7f8ee6738b80 1 -- start start 2024-01-11T18:10:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.593+0000 7f8ee6738b80 1 Processor -- start 2024-01-11T18:10:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.597+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.597+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cc7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.597+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.597+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1c19cd0 con 0x55ecd1cce690 2024-01-11T18:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.597+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1b88340 con 0x55ecd1ccfad0 2024-01-11T18:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.597+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1b87fe0 con 0x55ecd1cceff0 2024-01-11T18:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.598+0000 7f8ee6738b80 10 client.0 _ll_register_callbacks cb 0x55ecd1bba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.598+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.598+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cc7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.598+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cc7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41316/0 (socket says 172.21.15.7:41316) 2024-01-11T18:10:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.598+0000 7f8ecb7fe700 1 -- 172.21.15.7:0/2317169434 learned_addr learned my addr 172.21.15.7:0/2317169434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8ecbfff700 1 -- 172.21.15.7:0/2317169434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 msgr2=0x55ecd1cc7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cc7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8ecbfff700 1 -- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 msgr2=0x55ecd1ccca50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8ecbfff700 1 -- 172.21.15.7:0/2317169434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc10a360 con 0x55ecd1cce690 2024-01-11T18:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc74e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0098f0 tx=0x7f8ebc0056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5b34b31457814a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.599+0000 7f8eab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:10:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.601+0000 7f8eab7fe700 1 -- 172.21.15.7:0/2317169434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00b970 con 0x55ecd1cce690 2024-01-11T18:10:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.601+0000 7f8eab7fe700 1 -- 172.21.15.7:0/2317169434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8ebc003680 con 0x55ecd1cce690 2024-01-11T18:10:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.601+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecd1cb8d60 con 0x55ecd1cce690 2024-01-11T18:10:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.605+0000 7f8eab7fe700 1 -- 172.21.15.7:0/2317169434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ebc00bad0 con 0x55ecd1cce690 2024-01-11T18:10:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.605+0000 7f8ee6738b80 10 client.16230 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.605+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ecd1a3ab00 con 0x55ecd1cce690 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.605+0000 7f8eab7fe700 1 -- 172.21.15.7:0/2317169434 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecd1cb8d60 con 0x55ecd1cce690 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8eab7fe700 1 -- 172.21.15.7:0/2317169434 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8ebc028070 con 0x55ecd1cce690 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ee6738b80 10 client.16230 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ee6738b80 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ee6738b80 20 client.16230 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1cec2e0 0x55ecd1cee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ee6738b80 4 client.16230 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:10:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecd1c12b60 con 0x55ecd1cec2e0 2024-01-11T18:10:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.606+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1cec2e0 0x55ecd1cee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.607+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1cec2e0 0x55ecd1cee700 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a90568e0a2c0d2 server_cookie=c9c3ef7d9633d23a in_seq=0 out_seq=0 2024-01-11T18:10:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.607+0000 7f8eab7fe700 10 client.16230 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.608+0000 7f8eab7fe700 1 -- 172.21.15.7:0/2317169434 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecd1c12b60 con 0x55ecd1cec2e0 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.608+0000 7f8eab7fe700 10 client.16230 handle_command_reply: tid=0 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.608+0000 7f8ee6738b80 1 client.16230 shutdown 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 msgr2=0x55ecd1cc74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc74e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0098f0 tx=0x7f8ebc0056c0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 shutdown_connections 2024-01-11T18:10:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1cec2e0 0x55ecd1cee700 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc74e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/2317169434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cc7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 >> 172.21.15.7:0/2317169434 conn(0x55ecd1c3b740 msgr2=0x55ecd1cbff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8eab7fe700 0 client.16230 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 shutdown_connections 2024-01-11T18:10:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.609+0000 7f8ee6738b80 1 -- 172.21.15.7:0/2317169434 wait complete. 2024-01-11T18:10:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.610+0000 7f8ee6738b80 20 client.16230 trim_cache size 0 max 16384 2024-01-11T18:10:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.644+0000 7f8ee6738b80 1 Processor -- start 2024-01-11T18:10:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.644+0000 7f8ee6738b80 1 -- start start 2024-01-11T18:10:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1a90640 con 0x55ecd1cce690 2024-01-11T18:10:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1cc1be0 con 0x55ecd1ccfad0 2024-01-11T18:10:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1cc2270 con 0x55ecd1cceff0 2024-01-11T18:10:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.645+0000 7f8ecb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41318/0 (socket says 172.21.15.7:41318) 2024-01-11T18:10:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecb7fe700 1 -- 172.21.15.7:0/433790754 learned_addr learned my addr 172.21.15.7:0/433790754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecb7fe700 1 -- 172.21.15.7:0/433790754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 msgr2=0x55ecd1ccd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecb7fe700 1 -- 172.21.15.7:0/433790754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 msgr2=0x55ecd1cc6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ecb7fe700 1 -- 172.21.15.7:0/433790754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd1bdb1d0 con 0x55ecd1cceff0 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.646+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.647+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccca10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000b810 tx=0x7f8ec0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd7b79ff7522d880 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.647+0000 7f8eb1ffb700 1 -- 172.21.15.7:0/433790754 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0024070 con 0x55ecd1cceff0 2024-01-11T18:10:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.647+0000 7f8eb1ffb700 1 -- 172.21.15.7:0/433790754 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec0016070 con 0x55ecd1cceff0 2024-01-11T18:10:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.647+0000 7f8eb1ffb700 1 -- 172.21.15.7:0/433790754 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0025270 con 0x55ecd1cceff0 2024-01-11T18:10:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.649+0000 7f8ee6738b80 1 -- 172.21.15.7:0/433790754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 msgr2=0x55ecd1ccca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.649+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccca10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000b810 tx=0x7f8ec0014000 comp rx=0 tx=0).stop 2024-01-11T18:10:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.650+0000 7f8ee6738b80 1 -- 172.21.15.7:0/433790754 shutdown_connections 2024-01-11T18:10:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.650+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1ccfad0 0x55ecd1ccd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.650+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1cce690 0x55ecd1cc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.650+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/433790754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1ccca10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.650+0000 7f8ee6738b80 1 -- 172.21.15.7:0/433790754 >> 172.21.15.7:0/433790754 conn(0x55ecd1c3b740 msgr2=0x55ecd1cbc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.651+0000 7f8ee6738b80 1 -- 172.21.15.7:0/433790754 shutdown_connections 2024-01-11T18:10:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.651+0000 7f8ee6738b80 1 -- 172.21.15.7:0/433790754 wait complete. 2024-01-11T18:10:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.651+0000 7f8ee6738b80 1 -- start start 2024-01-11T18:10:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.652+0000 7f8ee6738b80 1 Processor -- start 2024-01-11T18:10:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cfcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cfb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1ccfad0 0x55ecd1cfb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1cfbf10 con 0x55ecd1ccfad0 2024-01-11T18:10:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1d41280 con 0x55ecd1cce690 2024-01-11T18:10:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd1d413c0 con 0x55ecd1cceff0 2024-01-11T18:10:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ee6738b80 10 client.0 _ll_register_callbacks cb 0x55ecd1bba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cfcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cfcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51444/0 (socket says 172.21.15.7:51444) 2024-01-11T18:10:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.653+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1424575776 learned_addr learned my addr 172.21.15.7:0/1424575776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cfb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1ccfad0 0x55ecd1cfb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1424575776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 msgr2=0x55ecd1cfb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cfb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1424575776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1ccfad0 msgr2=0x55ecd1cfb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1ccfad0 0x55ecd1cfb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.654+0000 7f8ecbfff700 1 -- 172.21.15.7:0/1424575776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec0011bf0 con 0x55ecd1cce690 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.655+0000 7f8ecbfff700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cfcaf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005a80 tx=0x7f8ebc0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac734a4b585ada68 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.655+0000 7f8eab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.655+0000 7f8ed0f9c700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1ccfad0 0x55ecd1cfb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.655+0000 7f8eab7fe700 1 -- 172.21.15.7:0/1424575776 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc02b850 con 0x55ecd1cce690 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.655+0000 7f8eab7fe700 1 -- 172.21.15.7:0/1424575776 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8ebc000e40 con 0x55ecd1cce690 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.655+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ebc01d040 con 0x55ecd1cce690 2024-01-11T18:10:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8eab7fe700 1 -- 172.21.15.7:0/1424575776 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ebc00b770 con 0x55ecd1cce690 2024-01-11T18:10:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8ee6738b80 10 client.26450 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ecd1a92c70 con 0x55ecd1cce690 2024-01-11T18:10:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8eab7fe700 1 -- 172.21.15.7:0/1424575776 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ebc01d040 con 0x55ecd1cce690 2024-01-11T18:10:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8eab7fe700 1 -- 172.21.15.7:0/1424575776 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8ebc028070 con 0x55ecd1cce690 2024-01-11T18:10:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8ee6738b80 10 client.26450 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8ee6738b80 10 client.26450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8ee6738b80 20 client.26450 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.656+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1d42910 0x55ecd1d44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.657+0000 7f8ee6738b80 4 client.26450 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:10:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.657+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecd1c12880 con 0x55ecd1d42910 2024-01-11T18:10:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.657+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1d42910 0x55ecd1d44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.658+0000 7f8ecb7fe700 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1d42910 0x55ecd1d44d30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfe2b9f1bce5cbf server_cookie=6abfadd3f15d7106 in_seq=0 out_seq=0 2024-01-11T18:10:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.658+0000 7f8eab7fe700 10 client.26450 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.690+0000 7f8eab7fe700 1 -- 172.21.15.7:0/1424575776 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecd1c12880 con 0x55ecd1d42910 2024-01-11T18:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.690+0000 7f8eab7fe700 10 client.26450 handle_command_reply: tid=0 2024-01-11T18:10:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.690+0000 7f8ee6738b80 1 client.26450 shutdown 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 msgr2=0x55ecd1cfcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cfcaf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005a80 tx=0x7f8ebc0097f0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 shutdown_connections 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55ecd1d42910 0x55ecd1d44d30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ecd1ccfad0 0x55ecd1cfb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8eab7fe700 0 client.26450 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecd1cce690 0x55ecd1cfcaf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 --2- 172.21.15.7:0/1424575776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ecd1cceff0 0x55ecd1cfb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.691+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 >> 172.21.15.7:0/1424575776 conn(0x55ecd1c3b740 msgr2=0x55ecd1cbc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.692+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 shutdown_connections 2024-01-11T18:10:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.692+0000 7f8ee6738b80 1 -- 172.21.15.7:0/1424575776 wait complete. 2024-01-11T18:10:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.692+0000 7f8ee6738b80 20 client.26450 trim_cache size 0 max 16384 2024-01-11T18:10:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.693+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8eb40a20a0 msgr2=0x7f8eb40a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.693+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8eb40a20a0 0x7f8eb40a4560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000b6c0 tx=0x7f8ec0005d20 comp rx=0 tx=0).stop 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 msgr2=0x7f8ecc1a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 0x7f8ecc1a0d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400efc0 tx=0x7f8ec400c3a0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 shutdown_connections 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f8eb40a20a0 0x7f8eb40a4560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ecc103160 0x7f8ecc1a0d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ecc1015d0 0x7f8ecc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 --2- 172.21.15.7:0/1345596657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc1027c0 0x7f8ecc199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 >> 172.21.15.7:0/1345596657 conn(0x7f8ecc0fcb60 msgr2=0x7f8ecc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 shutdown_connections 2024-01-11T18:10:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:11.694+0000 7f8ed29ff700 1 -- 172.21.15.7:0/1345596657 wait complete. 2024-01-11T18:10:13.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:13 smithi016 ceph-mon[43195]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.9 MiB/s wr, 492 op/s 2024-01-11T18:10:13.523 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:13 smithi038 ceph-mon[43582]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.9 MiB/s wr, 492 op/s 2024-01-11T18:10:13.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:13 smithi007 ceph-mon[33361]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.9 MiB/s wr, 492 op/s 2024-01-11T18:10:14.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 705.0 s, 518.9 tps, lat 187.982 ms stddev 224.764 2024-01-11T18:10:16.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:15 smithi016 ceph-mon[43195]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-01-11T18:10:16.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:15 smithi038 ceph-mon[43582]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-01-11T18:10:16.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:15 smithi007 ceph-mon[33361]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-01-11T18:10:17.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:17 smithi016 ceph-mon[43195]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.3 MiB/s wr, 578 op/s 2024-01-11T18:10:17.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:17 smithi038 ceph-mon[43582]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.3 MiB/s wr, 578 op/s 2024-01-11T18:10:17.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:17 smithi007 ceph-mon[33361]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.3 MiB/s wr, 578 op/s 2024-01-11T18:10:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 710.0 s, 505.1 tps, lat 199.975 ms stddev 245.699 2024-01-11T18:10:19.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:19 smithi016 ceph-mon[43195]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-01-11T18:10:19.528 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:19 smithi038 ceph-mon[43582]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-01-11T18:10:19.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:19 smithi007 ceph-mon[33361]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-01-11T18:10:21.399 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:21 smithi016 ceph-mon[43195]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 553 op/s 2024-01-11T18:10:21.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:21 smithi038 ceph-mon[43582]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 553 op/s 2024-01-11T18:10:21.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:21 smithi007 ceph-mon[33361]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 553 op/s 2024-01-11T18:10:23.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:23 smithi016 ceph-mon[43195]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 529 op/s 2024-01-11T18:10:23.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:23 smithi038 ceph-mon[43582]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 529 op/s 2024-01-11T18:10:23.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:23 smithi007 ceph-mon[33361]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 529 op/s 2024-01-11T18:10:24.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 715.0 s, 480.6 tps, lat 205.563 ms stddev 230.598 2024-01-11T18:10:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.192+0000 7fae4ebff700 1 -- 172.21.15.7:0/2388210746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48109770 msgr2=0x7fae48110020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.192+0000 7fae4ebff700 1 --2- 172.21.15.7:0/2388210746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48109770 0x7fae48110020 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae40009920 tx=0x7fae40009c30 comp rx=0 tx=0).stop 2024-01-11T18:10:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.193+0000 7fae4ebff700 1 -- 172.21.15.7:0/2388210746 shutdown_connections 2024-01-11T18:10:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.193+0000 7fae4ebff700 1 --2- 172.21.15.7:0/2388210746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae48107be0 0x7fae48108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.193+0000 7fae4ebff700 1 --2- 172.21.15.7:0/2388210746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48109770 0x7fae48110020 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.193+0000 7fae4ebff700 1 --2- 172.21.15.7:0/2388210746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48108dd0 0x7fae48109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.193+0000 7fae4ebff700 1 -- 172.21.15.7:0/2388210746 >> 172.21.15.7:0/2388210746 conn(0x7fae48076b80 msgr2=0x7fae48078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.195+0000 7fae4ebff700 1 -- 172.21.15.7:0/2388210746 shutdown_connections 2024-01-11T18:10:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.195+0000 7fae4ebff700 1 -- 172.21.15.7:0/2388210746 wait complete. 2024-01-11T18:10:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.197+0000 7fae4ebff700 1 Processor -- start 2024-01-11T18:10:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.199+0000 7fae4ebff700 1 -- start start 2024-01-11T18:10:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.202+0000 7fae4ebff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48107be0 0x7fae4819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae4ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 0x7fae4819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae4ebff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae48109770 0x7fae481a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae4ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481a56d0 con 0x7fae48108dd0 2024-01-11T18:10:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae4ebff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481a5810 con 0x7fae48107be0 2024-01-11T18:10:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae4ebff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481a59a0 con 0x7fae48109770 2024-01-11T18:10:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 0x7fae4819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 0x7fae4819e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54944/0 (socket says 172.21.15.7:54944) 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae3ffff700 1 -- 172.21.15.7:0/3484283953 learned_addr learned my addr 172.21.15.7:0/3484283953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.203+0000 7fae4d19c700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae48109770 0x7fae481a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.204+0000 7fae3ffff700 1 -- 172.21.15.7:0/3484283953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae48109770 msgr2=0x7fae481a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.204+0000 7fae3ffff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae48109770 0x7fae481a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.204+0000 7fae3ffff700 1 -- 172.21.15.7:0/3484283953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48107be0 msgr2=0x7fae4819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.204+0000 7fae3ffff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48107be0 0x7fae4819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.204+0000 7fae3ffff700 1 -- 172.21.15.7:0/3484283953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40009520 con 0x7fae48108dd0 2024-01-11T18:10:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.204+0000 7fae3ffff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 0x7fae4819e090 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fae3000e9c0 tx=0x7fae3000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.205+0000 7fae467fc700 1 -- 172.21.15.7:0/3484283953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000cbd0 con 0x7fae48108dd0 2024-01-11T18:10:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.205+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae481a5c80 con 0x7fae48108dd0 2024-01-11T18:10:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.206+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae481a6210 con 0x7fae48108dd0 2024-01-11T18:10:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.206+0000 7fae467fc700 1 -- 172.21.15.7:0/3484283953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3000eeb0 con 0x7fae48108dd0 2024-01-11T18:10:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.206+0000 7fae467fc700 1 -- 172.21.15.7:0/3484283953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30005580 con 0x7fae48108dd0 2024-01-11T18:10:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.217+0000 7fae6293db80 1 Processor -- start 2024-01-11T18:10:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.218+0000 7fae6293db80 1 -- start start 2024-01-11T18:10:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae467fc700 1 -- 172.21.15.7:0/3484283953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fae30004750 con 0x7fae48108dd0 2024-01-11T18:10:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae6293db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbcf20 0x564684dbd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae6293db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684dbe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae6293db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbe5d0 0x564684dc4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae6293db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684cb4030 con 0x564684dbe5d0 2024-01-11T18:10:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae6293db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684d17ac0 con 0x564684dbdc10 2024-01-11T18:10:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae6293db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684d153e0 con 0x564684dbcf20 2024-01-11T18:10:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae4c99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684dbe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae4c99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684dbe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40862/0 (socket says 172.21.15.7:40862) 2024-01-11T18:10:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.219+0000 7fae4c99b700 1 -- 172.21.15.7:0/3479883736 learned_addr learned my addr 172.21.15.7:0/3479883736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4d19c700 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbcf20 0x564684dbd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4c99b700 1 -- 172.21.15.7:0/3479883736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbcf20 msgr2=0x564684dbd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4c99b700 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbcf20 0x564684dbd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4c99b700 1 -- 172.21.15.7:0/3479883736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbe5d0 msgr2=0x564684dc4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4c99b700 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbe5d0 0x564684dc4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4c99b700 1 -- 172.21.15.7:0/3479883736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564684b85350 con 0x564684dbdc10 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.220+0000 7fae4c99b700 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684dbe090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d50 tx=0x7fae3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f4e0992fb99047 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.221+0000 7fae3effd700 1 -- 172.21.15.7:0/3479883736 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800e070 con 0x564684dbdc10 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.221+0000 7fae3effd700 1 -- 172.21.15.7:0/3479883736 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae38009880 con 0x564684dbdc10 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.221+0000 7fae3effd700 1 -- 172.21.15.7:0/3479883736 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38011390 con 0x564684dbdc10 2024-01-11T18:10:25.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 -- 172.21.15.7:0/3479883736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 msgr2=0x564684dbe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684dbe090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d50 tx=0x7fae3800b330 comp rx=0 tx=0).stop 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 -- 172.21.15.7:0/3479883736 shutdown_connections 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbcf20 0x564684dbd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbe5d0 0x564684dc4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 --2- 172.21.15.7:0/3479883736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684dbe090 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 -- 172.21.15.7:0/3479883736 >> 172.21.15.7:0/3479883736 conn(0x564684d32740 msgr2=0x564684db7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 -- 172.21.15.7:0/3479883736 shutdown_connections 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.223+0000 7fae6293db80 1 -- 172.21.15.7:0/3479883736 wait complete. 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.224+0000 7fae6293db80 1 -- start start 2024-01-11T18:10:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.224+0000 7fae6293db80 1 Processor -- start 2024-01-11T18:10:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.225+0000 7fae467fc700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fae340a1ee0 0x7fae340a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.226+0000 7fae6293db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684dd79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.226+0000 7fae6293db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 0x564684dd7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.226+0000 7fae6293db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbe5d0 0x564684ddef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.226+0000 7fae6293db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684d10cd0 con 0x564684dbcf20 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.226+0000 7fae6293db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684c7f340 con 0x564684dbe5d0 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.226+0000 7fae6293db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684c7efe0 con 0x564684dbdc10 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.227+0000 7fae6293db80 10 client.0 _ll_register_callbacks cb 0x564684cb16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.227+0000 7fae467fc700 1 -- 172.21.15.7:0/3484283953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fae30014070 con 0x7fae48108dd0 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.227+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 0x564684dd7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.227+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 0x564684dd7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37458/0 (socket says 172.21.15.7:37458) 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.227+0000 7fae3ffff700 1 -- 172.21.15.7:0/4215642345 learned_addr learned my addr 172.21.15.7:0/4215642345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.228+0000 7fae3ffff700 1 -- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbe5d0 msgr2=0x564684ddef50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.228+0000 7fae3ffff700 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbe5d0 0x564684ddef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.228+0000 7fae3ffff700 1 -- 172.21.15.7:0/4215642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 msgr2=0x564684dd79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.228+0000 7fae3ffff700 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684dd79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.228+0000 7fae3ffff700 1 -- 172.21.15.7:0/4215642345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae38009a00 con 0x564684dbdc10 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.228+0000 7fae4d19c700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fae340a1ee0 0x7fae340a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae3ffff700 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 0x564684dd7f10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fae300a27b0 tx=0x7fae3009b100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00a051d038719bd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae27fff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:10:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae27fff700 1 -- 172.21.15.7:0/4215642345 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300b3200 con 0x564684dbdc10 2024-01-11T18:10:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae27fff700 1 -- 172.21.15.7:0/4215642345 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fae300a2ce0 con 0x564684dbdc10 2024-01-11T18:10:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564684d09b60 con 0x564684dbdc10 2024-01-11T18:10:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae27fff700 1 -- 172.21.15.7:0/4215642345 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3009abb0 con 0x564684dbdc10 2024-01-11T18:10:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae6293db80 10 client.26494 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.229+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x564684b31b00 con 0x564684dbdc10 2024-01-11T18:10:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.231+0000 7fae4d19c700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fae340a1ee0 0x7fae340a43a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae4000fb70 tx=0x7fae40009f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.232+0000 7fae27fff700 1 -- 172.21.15.7:0/4215642345 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564684d09b60 con 0x564684dbdc10 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.233+0000 7fae27fff700 1 -- 172.21.15.7:0/4215642345 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fae300a2850 con 0x564684dbdc10 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae6293db80 10 client.26494 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae6293db80 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae6293db80 20 client.26494 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae6293db80 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684de3940 0x564684de5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae6293db80 4 client.26494 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564684d09880 con 0x564684de3940 2024-01-11T18:10:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.234+0000 7fae4c99b700 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684de3940 0x564684de5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.235+0000 7fae27fff700 10 client.26494 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.235+0000 7fae4c99b700 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684de3940 0x564684de5d60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45e1bcd4b70f81b server_cookie=ca44e95ebb5a30c8 in_seq=0 out_seq=0 2024-01-11T18:10:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae27fff700 1 -- 172.21.15.7:0/4215642345 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564684d09880 con 0x564684de3940 2024-01-11T18:10:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae27fff700 10 client.26494 handle_command_reply: tid=0 2024-01-11T18:10:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 client.26494 shutdown 2024-01-11T18:10:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 msgr2=0x564684dd7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 0x564684dd7f10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fae300a27b0 tx=0x7fae3009b100 comp rx=0 tx=0).stop 2024-01-11T18:10:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 shutdown_connections 2024-01-11T18:10:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684de3940 0x564684de5d60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbe5d0 0x564684ddef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684dd79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 --2- 172.21.15.7:0/4215642345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbdc10 0x564684dd7f10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 >> 172.21.15.7:0/4215642345 conn(0x564684d32740 msgr2=0x564684db6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.236+0000 7fae27fff700 0 client.26494 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.237+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 shutdown_connections 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.237+0000 7fae6293db80 1 -- 172.21.15.7:0/4215642345 wait complete. 2024-01-11T18:10:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.237+0000 7fae6293db80 20 client.26494 trim_cache size 0 max 16384 2024-01-11T18:10:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 Processor -- start 2024-01-11T18:10:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 -- start start 2024-01-11T18:10:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684ddf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684ddee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dc4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684b87640 con 0x564684dbcf20 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.273+0000 7fae6293db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684ddf3b0 con 0x564684dbdc10 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.274+0000 7fae6293db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684db4560 con 0x564684dbe5d0 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.274+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684ddee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.274+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684ddee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40886/0 (socket says 172.21.15.7:40886) 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.274+0000 7fae3ffff700 1 -- 172.21.15.7:0/613077722 learned_addr learned my addr 172.21.15.7:0/613077722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.274+0000 7fae3ffff700 1 -- 172.21.15.7:0/613077722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 msgr2=0x564684dc4c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:10:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.274+0000 7fae4c99b700 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684ddf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.275+0000 7fae3ffff700 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dc4c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.276+0000 7fae3ffff700 1 -- 172.21.15.7:0/613077722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 msgr2=0x564684ddf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.276+0000 7fae3ffff700 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684ddf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.276+0000 7fae3ffff700 1 -- 172.21.15.7:0/613077722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564684cd21d0 con 0x564684dbdc10 2024-01-11T18:10:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.276+0000 7fae4c99b700 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684ddf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:10:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.276+0000 7fae3ffff700 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684ddee70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae30098ca0 tx=0x7fae30099fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f00ed8aefdade2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.278+0000 7fae3e7fc700 1 -- 172.21.15.7:0/613077722 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3009abb0 con 0x564684dbdc10 2024-01-11T18:10:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.278+0000 7fae3e7fc700 1 -- 172.21.15.7:0/613077722 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564684d09b60 con 0x564684dbdc10 2024-01-11T18:10:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.278+0000 7fae3e7fc700 1 -- 172.21.15.7:0/613077722 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30004a00 con 0x564684dbdc10 2024-01-11T18:10:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.279+0000 7fae6293db80 1 -- 172.21.15.7:0/613077722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 msgr2=0x564684ddee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.279+0000 7fae6293db80 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684ddee70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae30098ca0 tx=0x7fae30099fb0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.280+0000 7fae6293db80 1 -- 172.21.15.7:0/613077722 shutdown_connections 2024-01-11T18:10:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.280+0000 7fae6293db80 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dc4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.280+0000 7fae6293db80 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbcf20 0x564684ddf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.280+0000 7fae6293db80 1 --2- 172.21.15.7:0/613077722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbdc10 0x564684ddee70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.280+0000 7fae6293db80 1 -- 172.21.15.7:0/613077722 >> 172.21.15.7:0/613077722 conn(0x564684d32740 msgr2=0x564684dd6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.280+0000 7fae6293db80 1 -- 172.21.15.7:0/613077722 shutdown_connections 2024-01-11T18:10:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.281+0000 7fae6293db80 1 -- 172.21.15.7:0/613077722 wait complete. 2024-01-11T18:10:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.281+0000 7fae6293db80 1 -- start start 2024-01-11T18:10:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.284+0000 7fae6293db80 1 Processor -- start 2024-01-11T18:10:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbcf20 0x564684dda950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbdc10 0x564684ddae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684dde3f0 con 0x564684dbdc10 2024-01-11T18:10:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684dde560 con 0x564684dbcf20 2024-01-11T18:10:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564684dde720 con 0x564684dbe5d0 2024-01-11T18:10:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.286+0000 7fae6293db80 10 client.0 _ll_register_callbacks cb 0x564684cb16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.288+0000 7fae4d19c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.288+0000 7fae4c99b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbcf20 0x564684dda950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.288+0000 7fae4d19c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dddeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:37474/0 (socket says 172.21.15.7:37474) 2024-01-11T18:10:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.288+0000 7fae4d19c700 1 -- 172.21.15.7:0/2327727324 learned_addr learned my addr 172.21.15.7:0/2327727324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae3ffff700 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbdc10 0x564684ddae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae4c99b700 1 -- 172.21.15.7:0/2327727324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 msgr2=0x564684dddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae4c99b700 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae4c99b700 1 -- 172.21.15.7:0/2327727324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbdc10 msgr2=0x564684ddae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae4c99b700 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbdc10 0x564684ddae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae4c99b700 1 -- 172.21.15.7:0/2327727324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3009e8b0 con 0x564684dbcf20 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.290+0000 7fae4c99b700 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbcf20 0x564684dda950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d20 tx=0x7fae3800bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8233707cf5708a95 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.291+0000 7fae27fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.291+0000 7fae27fff700 1 -- 172.21.15.7:0/2327727324 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38027070 con 0x564684dbcf20 2024-01-11T18:10:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.291+0000 7fae27fff700 1 -- 172.21.15.7:0/2327727324 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fae38012640 con 0x564684dbcf20 2024-01-11T18:10:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.293+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564684dafd60 con 0x564684dbcf20 2024-01-11T18:10:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.293+0000 7fae27fff700 1 -- 172.21.15.7:0/2327727324 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae38009ea0 con 0x564684dbcf20 2024-01-11T18:10:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.293+0000 7fae6293db80 10 client.26462 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.293+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x564684b89c70 con 0x564684dbcf20 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.293+0000 7fae27fff700 1 -- 172.21.15.7:0/2327727324 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564684dafd60 con 0x564684dbcf20 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae27fff700 1 -- 172.21.15.7:0/2327727324 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fae3801dc00 con 0x564684dbcf20 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae6293db80 10 client.26462 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae6293db80 10 client.26462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae6293db80 20 client.26462 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae6293db80 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684e4ba30 0x564684e4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae6293db80 4 client.26462 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.294+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564684b52c90 con 0x564684e4ba30 2024-01-11T18:10:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.295+0000 7fae3ffff700 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684e4ba30 0x564684e4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.301+0000 7fae3ffff700 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684e4ba30 0x564684e4de50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6084be52d3c4e698 server_cookie=7fa3515e77daabfc in_seq=0 out_seq=0 2024-01-11T18:10:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.301+0000 7fae27fff700 10 client.26462 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.318+0000 7fae27fff700 1 -- 172.21.15.7:0/2327727324 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564684b52c90 con 0x564684e4ba30 2024-01-11T18:10:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.318+0000 7fae27fff700 10 client.26462 handle_command_reply: tid=0 2024-01-11T18:10:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.318+0000 7fae6293db80 1 client.26462 shutdown 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbcf20 msgr2=0x564684dda950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbcf20 0x564684dda950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fae38009d20 tx=0x7fae3800bbd0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 shutdown_connections 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x564684e4ba30 0x564684e4de50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564684dbe5d0 0x564684dddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae27fff700 0 client.26462 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564684dbcf20 0x564684dda950 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.322+0000 7fae6293db80 1 --2- 172.21.15.7:0/2327727324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564684dbdc10 0x564684ddae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.323+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 >> 172.21.15.7:0/2327727324 conn(0x564684d32740 msgr2=0x564684dc4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.323+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 shutdown_connections 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.323+0000 7fae6293db80 1 -- 172.21.15.7:0/2327727324 wait complete. 2024-01-11T18:10:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.323+0000 7fae6293db80 20 client.26462 trim_cache size 0 max 16384 2024-01-11T18:10:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:10:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:10:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:10:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:10:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.324+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fae340a1ee0 msgr2=0x7fae340a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.324+0000 7fae4ebff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fae340a1ee0 0x7fae340a43a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae4000fb70 tx=0x7fae40009f80 comp rx=0 tx=0).stop 2024-01-11T18:10:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.324+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 msgr2=0x7fae4819e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.324+0000 7fae4ebff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 0x7fae4819e090 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fae3000e9c0 tx=0x7fae3000ecd0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 shutdown_connections 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fae340a1ee0 0x7fae340a43a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae48109770 0x7fae481a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48107be0 0x7fae4819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 --2- 172.21.15.7:0/3484283953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae48108dd0 0x7fae4819e090 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 >> 172.21.15.7:0/3484283953 conn(0x7fae48076b80 msgr2=0x7fae4810c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 shutdown_connections 2024-01-11T18:10:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:25.325+0000 7fae4ebff700 1 -- 172.21.15.7:0/3484283953 wait complete. 2024-01-11T18:10:26.239 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:25 smithi016 ceph-mon[43195]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 521 op/s 2024-01-11T18:10:26.269 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:25 smithi038 ceph-mon[43582]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 521 op/s 2024-01-11T18:10:26.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:25 smithi007 ceph-mon[33361]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 521 op/s 2024-01-11T18:10:27.489 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:27 smithi016 ceph-mon[43195]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.0 MiB/s wr, 558 op/s 2024-01-11T18:10:27.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:27 smithi038 ceph-mon[43582]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.0 MiB/s wr, 558 op/s 2024-01-11T18:10:27.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:27 smithi007 ceph-mon[33361]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.0 MiB/s wr, 558 op/s 2024-01-11T18:10:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 720.0 s, 541.0 tps, lat 184.771 ms stddev 278.286 2024-01-11T18:10:29.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:29 smithi016 ceph-mon[43195]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.3 MiB/s wr, 426 op/s 2024-01-11T18:10:29.523 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:29 smithi038 ceph-mon[43582]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.3 MiB/s wr, 426 op/s 2024-01-11T18:10:29.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:29 smithi007 ceph-mon[33361]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.3 MiB/s wr, 426 op/s 2024-01-11T18:10:31.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:31 smithi016 ceph-mon[43195]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-01-11T18:10:31.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:31 smithi038 ceph-mon[43582]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-01-11T18:10:31.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:31 smithi007 ceph-mon[33361]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 489 op/s 2024-01-11T18:10:33.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:33 smithi016 ceph-mon[43195]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 6.0 MiB/s wr, 502 op/s 2024-01-11T18:10:33.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:33 smithi038 ceph-mon[43582]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 6.0 MiB/s wr, 502 op/s 2024-01-11T18:10:33.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:33 smithi007 ceph-mon[33361]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 6.0 MiB/s wr, 502 op/s 2024-01-11T18:10:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 725.0 s, 475.6 tps, lat 218.587 ms stddev 280.191 2024-01-11T18:10:36.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:35 smithi016 ceph-mon[43195]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.6 MiB/s wr, 446 op/s 2024-01-11T18:10:36.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:35 smithi038 ceph-mon[43582]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.6 MiB/s wr, 446 op/s 2024-01-11T18:10:36.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:35 smithi007 ceph-mon[33361]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.6 MiB/s wr, 446 op/s 2024-01-11T18:10:37.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:37 smithi016 ceph-mon[43195]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.2 MiB/s wr, 505 op/s 2024-01-11T18:10:37.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:37 smithi038 ceph-mon[43582]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.2 MiB/s wr, 505 op/s 2024-01-11T18:10:37.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:37 smithi007 ceph-mon[33361]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.2 MiB/s wr, 505 op/s 2024-01-11T18:10:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 -- 172.21.15.7:0/332232252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 msgr2=0x7f0c8c1044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 --2- 172.21.15.7:0/332232252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1044b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00faf0 tx=0x7f0c7c00fe00 comp rx=0 tx=0).stop 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 -- 172.21.15.7:0/332232252 shutdown_connections 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 --2- 172.21.15.7:0/332232252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1044b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 --2- 172.21.15.7:0/332232252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8c107320 0x7f0c8c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 --2- 172.21.15.7:0/332232252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c1049f0 0x7f0c8c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.596+0000 7f0c913ff700 1 -- 172.21.15.7:0/332232252 >> 172.21.15.7:0/332232252 conn(0x7f0c8c0fba30 msgr2=0x7f0c8c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.597+0000 7f0c913ff700 1 -- 172.21.15.7:0/332232252 shutdown_connections 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.597+0000 7f0c913ff700 1 -- 172.21.15.7:0/332232252 wait complete. 2024-01-11T18:10:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.598+0000 7f0c913ff700 1 Processor -- start 2024-01-11T18:10:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.599+0000 7f0c913ff700 1 -- start start 2024-01-11T18:10:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.599+0000 7f0c913ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.599+0000 7f0c913ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8c1049f0 0x7f0c8c195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c913ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c107320 0x7f0c8c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c913ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c19ce40 con 0x7f0c8c107320 2024-01-11T18:10:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c913ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c19cf80 con 0x7f0c8c1049f0 2024-01-11T18:10:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c913ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c19d110 con 0x7f0c8c1020c0 2024-01-11T18:10:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46844/0 (socket says 172.21.15.7:46844) 2024-01-11T18:10:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c8affd700 1 -- 172.21.15.7:0/2132821252 learned_addr learned my addr 172.21.15.7:0/2132821252 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8c1049f0 0x7f0c8c195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.600+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c107320 0x7f0c8c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8affd700 1 -- 172.21.15.7:0/2132821252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8c1049f0 msgr2=0x7f0c8c195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8affd700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8c1049f0 0x7f0c8c195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8affd700 1 -- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c107320 msgr2=0x7f0c8c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8affd700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c107320 0x7f0c8c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8affd700 1 -- 172.21.15.7:0/2132821252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c00f750 con 0x7f0c8c1020c0 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c107320 0x7f0c8c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:10:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.601+0000 7f0c8affd700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1952c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0036c0 tx=0x7f0c7c00f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.602+0000 7f0c73fff700 1 -- 172.21.15.7:0/2132821252 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c004070 con 0x7f0c8c1020c0 2024-01-11T18:10:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.602+0000 7f0c73fff700 1 -- 172.21.15.7:0/2132821252 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7c0041d0 con 0x7f0c8c1020c0 2024-01-11T18:10:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.602+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c19d390 con 0x7f0c8c1020c0 2024-01-11T18:10:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.602+0000 7f0c73fff700 1 -- 172.21.15.7:0/2132821252 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0216c0 con 0x7f0c8c1020c0 2024-01-11T18:10:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.602+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c19d860 con 0x7f0c8c1020c0 2024-01-11T18:10:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.609+0000 7f0c73fff700 1 -- 172.21.15.7:0/2132821252 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0c7c021820 con 0x7f0c8c1020c0 2024-01-11T18:10:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.611+0000 7f0ca50c6b80 1 Processor -- start 2024-01-11T18:10:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.612+0000 7f0c73fff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0c740a1f50 0x7f0c740a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.612+0000 7f0ca50c6b80 1 -- start start 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.612+0000 7f0c73fff700 1 -- 172.21.15.7:0/2132821252 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0c7c0d35c0 con 0x7f0c8c1020c0 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.612+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165715b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.612+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d1657168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d16571d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165604030 con 0x55d165716420 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165667ac0 con 0x55d165715730 2024-01-11T18:10:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1656653e0 con 0x55d165716de0 2024-01-11T18:10:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165715b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165715b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50818/0 (socket says 172.21.15.7:50818) 2024-01-11T18:10:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.613+0000 7f0c8affd700 1 -- 172.21.15.7:0/3230560335 learned_addr learned my addr 172.21.15.7:0/3230560335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8affd700 1 -- 172.21.15.7:0/3230560335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 msgr2=0x55d16571d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8affd700 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d16571d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8affd700 1 -- 172.21.15.7:0/3230560335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 msgr2=0x55d1657168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8affd700 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d1657168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8affd700 1 -- 172.21.15.7:0/3230560335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1654d5350 con 0x55d165715730 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8affd700 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165715b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c012040 tx=0x7f0c7c00f560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740f373ed5acf481 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0c740a1f50 0x7f0c740a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c717fa700 1 -- 172.21.15.7:0/3230560335 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0a0f60 con 0x55d165715730 2024-01-11T18:10:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c717fa700 1 -- 172.21.15.7:0/3230560335 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7c0a50b0 con 0x55d165715730 2024-01-11T18:10:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.614+0000 7f0c717fa700 1 -- 172.21.15.7:0/3230560335 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0a0f60 con 0x55d165715730 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.616+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/3230560335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 msgr2=0x55d165715b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.616+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165715b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c012040 tx=0x7f0c7c00f560 comp rx=0 tx=0).stop 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.618+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/3230560335 shutdown_connections 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.618+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d16571d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.618+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165715b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.618+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/3230560335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d1657168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.618+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/3230560335 >> 172.21.15.7:0/3230560335 conn(0x55d165682740 msgr2=0x55d165707880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.619+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0c740a1f50 0x7f0c740a4410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c840094f0 tx=0x7f0c84009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.621+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/3230560335 shutdown_connections 2024-01-11T18:10:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.621+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/3230560335 wait complete. 2024-01-11T18:10:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.622+0000 7f0ca50c6b80 1 -- start start 2024-01-11T18:10:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.623+0000 7f0ca50c6b80 1 Processor -- start 2024-01-11T18:10:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.624+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 0x55d16571ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.625+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d16571f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.625+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 0x55d16571ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.626+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 0x55d16571ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46852/0 (socket says 172.21.15.7:46852) 2024-01-11T18:10:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.626+0000 7f0c8affd700 1 -- 172.21.15.7:0/1521458503 learned_addr learned my addr 172.21.15.7:0/1521458503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:38.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.626+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716de0 0x55d1657263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.626+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165660cd0 con 0x55d165716420 2024-01-11T18:10:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.626+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1655cf340 con 0x55d165716de0 2024-01-11T18:10:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.626+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1655cefe0 con 0x55d165715730 2024-01-11T18:10:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716de0 0x55d1657263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0ca50c6b80 10 client.0 _ll_register_callbacks cb 0x55d1656016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d16571f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/1521458503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 msgr2=0x55d16571ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 0x55d16571ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/1521458503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 msgr2=0x55d16571f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d16571f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c19d860 con 0x55d165716de0 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d16571f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.627+0000 7f0c8affd700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 0x55d16571ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.628+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716de0 0x55d1657263d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c84018a70 tx=0x7f0c84004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd3a73e8dc405df server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.628+0000 7f0c6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.628+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1521458503 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401d070 con 0x55d165716de0 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.628+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1521458503 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0c840157c0 con 0x55d165716de0 2024-01-11T18:10:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.628+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1656ffd60 con 0x55d165716de0 2024-01-11T18:10:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.629+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1521458503 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c840182e0 con 0x55d165716de0 2024-01-11T18:10:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.629+0000 7f0ca50c6b80 10 client.26474 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.629+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55d165481b00 con 0x55d165716de0 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1521458503 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1656ffd60 con 0x55d165716de0 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1521458503 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c84029070 con 0x55d165716de0 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0ca50c6b80 10 client.26474 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0ca50c6b80 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0ca50c6b80 20 client.26474 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d16572aca0 0x55d16572d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0ca50c6b80 4 client.26474 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:10:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.631+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d165659b60 con 0x55d16572aca0 2024-01-11T18:10:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.632+0000 7f0c8affd700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d16572aca0 0x55d16572d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.633+0000 7f0c8affd700 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d16572aca0 0x55d16572d0c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418a4c6f37182ee9 server_cookie=58e5cc19c9dfaf4d in_seq=0 out_seq=0 2024-01-11T18:10:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.633+0000 7f0c6eff5700 10 client.26474 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.637+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1521458503 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d165659b60 con 0x55d16572aca0 2024-01-11T18:10:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.637+0000 7f0c6eff5700 10 client.26474 handle_command_reply: tid=0 2024-01-11T18:10:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.637+0000 7f0ca50c6b80 1 client.26474 shutdown 2024-01-11T18:10:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.637+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716de0 msgr2=0x55d1657263d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.637+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716de0 0x55d1657263d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c84018a70 tx=0x7f0c84004ad0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 shutdown_connections 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d16572aca0 0x55d16572d0c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165715730 0x55d16571ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716de0 0x55d1657263d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1521458503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716420 0x55d16571f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 >> 172.21.15.7:0/1521458503 conn(0x55d165682740 msgr2=0x55d165706f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0c6eff5700 0 client.26474 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 shutdown_connections 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1521458503 wait complete. 2024-01-11T18:10:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.638+0000 7f0ca50c6b80 20 client.26474 trim_cache size 0 max 16384 2024-01-11T18:10:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.672+0000 7f0ca50c6b80 1 Processor -- start 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 -- start start 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165715730 0x55d16571e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716420 0x55d165726390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d165719e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1654d7640 con 0x55d165715730 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16571e9c0 con 0x55d165716420 2024-01-11T18:10:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1657085c0 con 0x55d165716de0 2024-01-11T18:10:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.673+0000 7f0c8b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d165719e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d165719e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:46866/0 (socket says 172.21.15.7:46866) 2024-01-11T18:10:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/609801935 learned_addr learned my addr 172.21.15.7:0/609801935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716420 0x55d165726390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/609801935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716420 msgr2=0x55d165726390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716420 0x55d165726390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/609801935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165715730 msgr2=0x55d16571e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165715730 0x55d16571e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8b7fe700 1 -- 172.21.15.7:0/609801935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1656221d0 con 0x55d165716de0 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.674+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716420 0x55d165726390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.675+0000 7f0c8b7fe700 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d165719e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c840077e0 tx=0x7f0c84007ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314c7d70644d89e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.675+0000 7f0c70ff9700 1 -- 172.21.15.7:0/609801935 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401d070 con 0x55d165716de0 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.675+0000 7f0c70ff9700 1 -- 172.21.15.7:0/609801935 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c840168e0 con 0x55d165716de0 2024-01-11T18:10:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.676+0000 7f0c70ff9700 1 -- 172.21.15.7:0/609801935 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84013480 con 0x55d165716de0 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.676+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/609801935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 msgr2=0x55d165719e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d165719e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c840077e0 tx=0x7f0c84007ad0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/609801935 shutdown_connections 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716de0 0x55d165719e20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165715730 0x55d16571e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/609801935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165716420 0x55d165726390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/609801935 >> 172.21.15.7:0/609801935 conn(0x55d165682740 msgr2=0x55d165731a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/609801935 shutdown_connections 2024-01-11T18:10:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.677+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/609801935 wait complete. 2024-01-11T18:10:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.678+0000 7f0ca50c6b80 1 -- start start 2024-01-11T18:10:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.679+0000 7f0ca50c6b80 1 Processor -- start 2024-01-11T18:10:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165721940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716420 0x55d165721e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716de0 0x55d165724db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1657252f0 con 0x55d165716de0 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165725460 con 0x55d165715730 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165725620 con 0x55d165716420 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.680+0000 7f0ca50c6b80 10 client.0 _ll_register_callbacks cb 0x55d1656016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.681+0000 7f0c8a7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716420 0x55d165721e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.681+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165721940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.681+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165721940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50836/0 (socket says 172.21.15.7:50836) 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.681+0000 7f0c8affd700 1 -- 172.21.15.7:0/1702531703 learned_addr learned my addr 172.21.15.7:0/1702531703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c8affd700 1 -- 172.21.15.7:0/1702531703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716420 msgr2=0x55d165721e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c8affd700 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716420 0x55d165721e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c8affd700 1 -- 172.21.15.7:0/1702531703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716de0 msgr2=0x55d165724db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c8affd700 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716de0 0x55d165724db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c8affd700 1 -- 172.21.15.7:0/1702531703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d165481b00 con 0x55d165715730 2024-01-11T18:10:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c8affd700 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165721940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009870 tx=0x7f0c7c00fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee23b57b1c342cf6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.682+0000 7f0c6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.683+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1702531703 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0a0f60 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.683+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c7c0a50b0 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.683+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1702531703 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0c7c0d33c0 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.683+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1702531703 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c7c004340 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.683+0000 7f0ca50c6b80 10 client.26477 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.683+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55d1654d9c70 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1702531703 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7c083080 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1702531703 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c7c0d3070 con 0x55d165715730 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0ca50c6b80 10 client.26477 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0ca50c6b80 10 client.26477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0ca50c6b80 20 client.26477 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d165792560 0x55d165794980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0ca50c6b80 4 client.26477 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:10:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.684+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d165659880 con 0x55d165792560 2024-01-11T18:10:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.685+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d165792560 0x55d165794980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.685+0000 7f0c8a7fc700 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d165792560 0x55d165794980 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84585178733a89a1 server_cookie=eb1cbd5fab3a5f68 in_seq=0 out_seq=0 2024-01-11T18:10:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.685+0000 7f0c6eff5700 10 client.26477 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.755+0000 7f0c6eff5700 1 -- 172.21.15.7:0/1702531703 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d165659880 con 0x55d165792560 2024-01-11T18:10:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.755+0000 7f0c6eff5700 10 client.26477 handle_command_reply: tid=0 2024-01-11T18:10:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.755+0000 7f0ca50c6b80 1 client.26477 shutdown 2024-01-11T18:10:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.756+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 msgr2=0x55d165721940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.756+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165721940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009870 tx=0x7f0c7c00fe00 comp rx=0 tx=0).stop 2024-01-11T18:10:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 shutdown_connections 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d165792560 0x55d165794980 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d165716de0 0x55d165724db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d165715730 0x55d165721940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 --2- 172.21.15.7:0/1702531703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d165716420 0x55d165721e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 >> 172.21.15.7:0/1702531703 conn(0x55d165682740 msgr2=0x55d165719e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0c6eff5700 0 client.26477 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 shutdown_connections 2024-01-11T18:10:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.757+0000 7f0ca50c6b80 1 -- 172.21.15.7:0/1702531703 wait complete. 2024-01-11T18:10:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.758+0000 7f0ca50c6b80 20 client.26477 trim_cache size 0 max 16384 2024-01-11T18:10:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:10:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:10:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:10:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.759+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0c740a1f50 msgr2=0x7f0c740a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.759+0000 7f0c913ff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0c740a1f50 0x7f0c740a4410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c840094f0 tx=0x7f0c84009040 comp rx=0 tx=0).stop 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.759+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 msgr2=0x7f0c8c1952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.759+0000 7f0c913ff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1952c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0036c0 tx=0x7f0c7c00f280 comp rx=0 tx=0).stop 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 shutdown_connections 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0c740a1f50 0x7f0c740a4410 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c8c1020c0 0x7f0c8c1952c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c107320 0x7f0c8c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 --2- 172.21.15.7:0/2132821252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8c1049f0 0x7f0c8c195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 >> 172.21.15.7:0/2132821252 conn(0x7f0c8c0fba30 msgr2=0x7f0c8c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 shutdown_connections 2024-01-11T18:10:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:38.760+0000 7f0c913ff700 1 -- 172.21.15.7:0/2132821252 wait complete. 2024-01-11T18:10:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 730.0 s, 440.2 tps, lat 218.957 ms stddev 249.380 2024-01-11T18:10:39.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:39 smithi016 ceph-mon[43195]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.8 MiB/s wr, 414 op/s 2024-01-11T18:10:39.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:39 smithi038 ceph-mon[43582]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.8 MiB/s wr, 414 op/s 2024-01-11T18:10:39.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:39 smithi007 ceph-mon[33361]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.8 MiB/s wr, 414 op/s 2024-01-11T18:10:41.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:41 smithi016 ceph-mon[43195]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-01-11T18:10:41.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:41 smithi038 ceph-mon[43582]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-01-11T18:10:41.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:41 smithi007 ceph-mon[33361]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-01-11T18:10:43.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:43 smithi016 ceph-mon[43195]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-01-11T18:10:43.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:43 smithi038 ceph-mon[43582]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-01-11T18:10:43.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:43 smithi007 ceph-mon[33361]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-01-11T18:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 735.0 s, 538.0 tps, lat 185.706 ms stddev 217.052 2024-01-11T18:10:46.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:45 smithi016 ceph-mon[43195]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 454 op/s 2024-01-11T18:10:46.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:45 smithi038 ceph-mon[43582]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 454 op/s 2024-01-11T18:10:46.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:45 smithi007 ceph-mon[33361]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.3 MiB/s wr, 454 op/s 2024-01-11T18:10:47.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:47 smithi016 ceph-mon[43195]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 535 op/s 2024-01-11T18:10:47.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:47 smithi038 ceph-mon[43582]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 535 op/s 2024-01-11T18:10:47.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:47 smithi007 ceph-mon[33361]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 535 op/s 2024-01-11T18:10:48.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:48 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:10:48.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:48 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:10:48.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:48 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:10:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 740.0 s, 442.4 tps, lat 230.866 ms stddev 289.756 2024-01-11T18:10:50.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:50.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:49 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:50.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:49 smithi016 ceph-mon[43195]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-01-11T18:10:50.284 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:50.284 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:49 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:50.284 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:49 smithi038 ceph-mon[43582]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-01-11T18:10:50.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:50.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:49 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:50.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:49 smithi007 ceph-mon[33361]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-01-11T18:10:51.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:51 smithi016 ceph-mon[43195]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.9 MiB/s wr, 501 op/s 2024-01-11T18:10:51.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:51 smithi038 ceph-mon[43582]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.9 MiB/s wr, 501 op/s 2024-01-11T18:10:51.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:51 smithi007 ceph-mon[33361]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.9 MiB/s wr, 501 op/s 2024-01-11T18:10:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 -- 172.21.15.7:0/1616543325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610104950 msgr2=0x7fc610106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 --2- 172.21.15.7:0/1616543325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610104950 0x7fc610106d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc604009990 tx=0x7fc604009ca0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 -- 172.21.15.7:0/1616543325 shutdown_connections 2024-01-11T18:10:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 --2- 172.21.15.7:0/1616543325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc610107280 0x7fc6101096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 --2- 172.21.15.7:0/1616543325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc610102020 0x7fc610104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 --2- 172.21.15.7:0/1616543325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610104950 0x7fc610106d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.052+0000 7fc616dff700 1 -- 172.21.15.7:0/1616543325 >> 172.21.15.7:0/1616543325 conn(0x7fc6100fba30 msgr2=0x7fc6100fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.053+0000 7fc616dff700 1 -- 172.21.15.7:0/1616543325 shutdown_connections 2024-01-11T18:10:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.053+0000 7fc616dff700 1 -- 172.21.15.7:0/1616543325 wait complete. 2024-01-11T18:10:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.055+0000 7fc616dff700 1 Processor -- start 2024-01-11T18:10:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.055+0000 7fc616dff700 1 -- start start 2024-01-11T18:10:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.057+0000 7fc616dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 0x7fc610199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.057+0000 7fc614b9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 0x7fc610199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 0x7fc610199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33050/0 (socket says 172.21.15.7:33050) 2024-01-11T18:10:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 -- 172.21.15.7:0/800586392 learned_addr learned my addr 172.21.15.7:0/800586392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc610104950 0x7fc610199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc610107280 0x7fc6101a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101a1300 con 0x7fc610107280 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101a1440 con 0x7fc610104950 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101a15d0 con 0x7fc610102020 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc60ffff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc610104950 0x7fc610199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 -- 172.21.15.7:0/800586392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc610104950 msgr2=0x7fc610199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc610104950 0x7fc610199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 -- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc610107280 msgr2=0x7fc6101a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc610107280 0x7fc6101a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.058+0000 7fc614b9b700 1 -- 172.21.15.7:0/800586392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6040095f0 con 0x7fc610102020 2024-01-11T18:10:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.059+0000 7fc614b9b700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 0x7fc610199780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00b550 tx=0x7fc5fc00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.059+0000 7fc60dffb700 1 -- 172.21.15.7:0/800586392 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc011830 con 0x7fc610102020 2024-01-11T18:10:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.059+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101a18b0 con 0x7fc610102020 2024-01-11T18:10:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.060+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101a1e90 con 0x7fc610102020 2024-01-11T18:10:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.060+0000 7fc60dffb700 1 -- 172.21.15.7:0/800586392 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5fc011e50 con 0x7fc610102020 2024-01-11T18:10:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.060+0000 7fc60dffb700 1 -- 172.21.15.7:0/800586392 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc00f2e0 con 0x7fc610102020 2024-01-11T18:10:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.070+0000 7fc60dffb700 1 -- 172.21.15.7:0/800586392 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc5fc00f440 con 0x7fc610102020 2024-01-11T18:10:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.072+0000 7fc62ababb80 1 Processor -- start 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.072+0000 7fc62ababb80 1 -- start start 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.072+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2770ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.072+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc27714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.072+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc2771f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.072+0000 7fc62ababb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc265f030 con 0x55abc2771ad0 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.073+0000 7fc62ababb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc26c2ac0 con 0x55abc2770690 2024-01-11T18:10:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.073+0000 7fc62ababb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc26c03e0 con 0x55abc2770ff0 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.073+0000 7fc61539c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2770ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.073+0000 7fc61539c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2770ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37090/0 (socket says 172.21.15.7:37090) 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.073+0000 7fc61539c700 1 -- 172.21.15.7:0/2117581814 learned_addr learned my addr 172.21.15.7:0/2117581814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc614b9b700 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc27714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc60ffff700 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc2771f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc61539c700 1 -- 172.21.15.7:0/2117581814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 msgr2=0x55abc27714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc61539c700 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc27714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc61539c700 1 -- 172.21.15.7:0/2117581814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 msgr2=0x55abc2771f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc61539c700 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc2771f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.074+0000 7fc61539c700 1 -- 172.21.15.7:0/2117581814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc2530350 con 0x55abc2770690 2024-01-11T18:10:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.075+0000 7fc61539c700 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2770ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc608009c60 tx=0x7fc608009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc862cbfc7b25022 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.075+0000 7fc60dffb700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc6000a1fd0 0x7fc6000a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.076+0000 7fc60dffb700 1 -- 172.21.15.7:0/800586392 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc5fc0119f0 con 0x7fc610102020 2024-01-11T18:10:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.076+0000 7fc5faffd700 1 -- 172.21.15.7:0/2117581814 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608017070 con 0x55abc2770690 2024-01-11T18:10:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.076+0000 7fc5faffd700 1 -- 172.21.15.7:0/2117581814 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60800f040 con 0x55abc2770690 2024-01-11T18:10:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.076+0000 7fc5faffd700 1 -- 172.21.15.7:0/2117581814 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6080059e0 con 0x55abc2770690 2024-01-11T18:10:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.078+0000 7fc614b9b700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc6000a1fd0 0x7fc6000a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.078+0000 7fc62ababb80 1 -- 172.21.15.7:0/2117581814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 msgr2=0x55abc2770ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.078+0000 7fc62ababb80 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2770ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc608009c60 tx=0x7fc608009f70 comp rx=0 tx=0).stop 2024-01-11T18:10:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.079+0000 7fc62ababb80 1 -- 172.21.15.7:0/2117581814 shutdown_connections 2024-01-11T18:10:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.079+0000 7fc62ababb80 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc2771f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.079+0000 7fc62ababb80 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2770ab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.079+0000 7fc62ababb80 1 --2- 172.21.15.7:0/2117581814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc27714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.079+0000 7fc62ababb80 1 -- 172.21.15.7:0/2117581814 >> 172.21.15.7:0/2117581814 conn(0x55abc26dd740 msgr2=0x55abc2762950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.079+0000 7fc614b9b700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc6000a1fd0 0x7fc6000a4490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc010b00 tx=0x7fc5fc00e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.081+0000 7fc62ababb80 1 -- 172.21.15.7:0/2117581814 shutdown_connections 2024-01-11T18:10:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.081+0000 7fc62ababb80 1 -- 172.21.15.7:0/2117581814 wait complete. 2024-01-11T18:10:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.082+0000 7fc62ababb80 1 -- start start 2024-01-11T18:10:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.082+0000 7fc62ababb80 1 Processor -- start 2024-01-11T18:10:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.083+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2769540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc2769a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc61539c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2769540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc276ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc62ababb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc26bbcd0 con 0x55abc2771ad0 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc62ababb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc262a340 con 0x55abc2770690 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc62ababb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2629fe0 con 0x55abc2770ff0 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc61539c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2769540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37096/0 (socket says 172.21.15.7:37096) 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc61539c700 1 -- 172.21.15.7:0/565514940 learned_addr learned my addr 172.21.15.7:0/565514940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc62ababb80 10 client.0 _ll_register_callbacks cb 0x55abc265c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.084+0000 7fc60ffff700 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc2769a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc61539c700 1 -- 172.21.15.7:0/565514940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 msgr2=0x55abc2769a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc61539c700 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc2769a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc61539c700 1 -- 172.21.15.7:0/565514940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 msgr2=0x55abc276ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc61539c700 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc276ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc61539c700 1 -- 172.21.15.7:0/565514940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6080098c0 con 0x55abc2770690 2024-01-11T18:10:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc61539c700 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2769540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc6080093c0 tx=0x7fc608004e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e51f18df24c4045 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.085+0000 7fc5effff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:10:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.086+0000 7fc5effff700 1 -- 172.21.15.7:0/565514940 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608017040 con 0x55abc2770690 2024-01-11T18:10:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.086+0000 7fc5effff700 1 -- 172.21.15.7:0/565514940 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc608005920 con 0x55abc2770690 2024-01-11T18:10:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.089+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abc275ad60 con 0x55abc2770690 2024-01-11T18:10:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.089+0000 7fc5effff700 1 -- 172.21.15.7:0/565514940 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60801d600 con 0x55abc2770690 2024-01-11T18:10:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.089+0000 7fc62ababb80 10 client.26486 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.089+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55abc24dcb00 con 0x55abc2770690 2024-01-11T18:10:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc5effff700 1 -- 172.21.15.7:0/565514940 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abc275ad60 con 0x55abc2770690 2024-01-11T18:10:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc5effff700 1 -- 172.21.15.7:0/565514940 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc608009c90 con 0x55abc2770690 2024-01-11T18:10:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc62ababb80 10 client.26486 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc62ababb80 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc62ababb80 20 client.26486 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc62ababb80 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc278e5f0 0x55abc2790a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.090+0000 7fc62ababb80 4 client.26486 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.091+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abc26b4b60 con 0x55abc278e5f0 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.091+0000 7fc60ffff700 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc278e5f0 0x55abc2790a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.092+0000 7fc5effff700 10 client.26486 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.092+0000 7fc60ffff700 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc278e5f0 0x55abc2790a10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba774520eeff07f server_cookie=8f76553985d36dd2 in_seq=0 out_seq=0 2024-01-11T18:10:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.093+0000 7fc5effff700 1 -- 172.21.15.7:0/565514940 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abc26b4b60 con 0x55abc278e5f0 2024-01-11T18:10:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.093+0000 7fc5effff700 10 client.26486 handle_command_reply: tid=0 2024-01-11T18:10:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 client.26486 shutdown 2024-01-11T18:10:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 msgr2=0x55abc2769540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2769540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc6080093c0 tx=0x7fc608004e80 comp rx=0 tx=0).stop 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 shutdown_connections 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc278e5f0 0x55abc2790a10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc276ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc5effff700 0 client.26486 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc2769540 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 --2- 172.21.15.7:0/565514940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc2769a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.094+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 >> 172.21.15.7:0/565514940 conn(0x55abc26dd740 msgr2=0x55abc2761f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.095+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 shutdown_connections 2024-01-11T18:10:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.095+0000 7fc62ababb80 1 -- 172.21.15.7:0/565514940 wait complete. 2024-01-11T18:10:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.095+0000 7fc62ababb80 20 client.26486 trim_cache size 0 max 16384 2024-01-11T18:10:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.126+0000 7fc62ababb80 1 Processor -- start 2024-01-11T18:10:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.126+0000 7fc62ababb80 1 -- start start 2024-01-11T18:10:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.126+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770690 0x55abc276f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.126+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 0x55abc2771eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc275e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc62ababb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2532640 con 0x55abc2771ad0 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc62ababb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2763be0 con 0x55abc2770ff0 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc62ababb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc2770e30 con 0x55abc2770690 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 0x55abc2771eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 0x55abc2771eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37110/0 (socket says 172.21.15.7:37110) 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 -- 172.21.15.7:0/4104212670 learned_addr learned my addr 172.21.15.7:0/4104212670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc61539c700 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770690 0x55abc276f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 -- 172.21.15.7:0/4104212670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770690 msgr2=0x55abc276f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770690 0x55abc276f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 -- 172.21.15.7:0/4104212670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 msgr2=0x55abc275e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc275e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.127+0000 7fc60ffff700 1 -- 172.21.15.7:0/4104212670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc267d1d0 con 0x55abc2770ff0 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.128+0000 7fc60ffff700 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 0x55abc2771eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc604003790 tx=0x7fc604009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a678cdc6703ac6d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.128+0000 7fc5fa7fc700 1 -- 172.21.15.7:0/4104212670 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60400ba50 con 0x55abc2770ff0 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.128+0000 7fc5fa7fc700 1 -- 172.21.15.7:0/4104212670 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60401e040 con 0x55abc2770ff0 2024-01-11T18:10:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.128+0000 7fc5fa7fc700 1 -- 172.21.15.7:0/4104212670 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6040262b0 con 0x55abc2770ff0 2024-01-11T18:10:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 -- 172.21.15.7:0/4104212670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 msgr2=0x55abc2771eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 0x55abc2771eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc604003790 tx=0x7fc604009290 comp rx=0 tx=0).stop 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 -- 172.21.15.7:0/4104212670 shutdown_connections 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770690 0x55abc276f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc275e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 --2- 172.21.15.7:0/4104212670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770ff0 0x55abc2771eb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 -- 172.21.15.7:0/4104212670 >> 172.21.15.7:0/4104212670 conn(0x55abc26dd740 msgr2=0x55abc2762920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 -- 172.21.15.7:0/4104212670 shutdown_connections 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.129+0000 7fc62ababb80 1 -- 172.21.15.7:0/4104212670 wait complete. 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 -- start start 2024-01-11T18:10:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 Processor -- start 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc279e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc279e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc27e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc27e3e00 con 0x55abc2771ad0 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc27e3f70 con 0x55abc2770690 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc27f5140 con 0x55abc2770ff0 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.130+0000 7fc62ababb80 10 client.0 _ll_register_callbacks cb 0x55abc265c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc279e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc279e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33088/0 (socket says 172.21.15.7:33088) 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 -- 172.21.15.7:0/1728775272 learned_addr learned my addr 172.21.15.7:0/1728775272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc61539c700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc279e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc61539c700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc27e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 -- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 msgr2=0x55abc279e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc279e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 -- 172.21.15.7:0/1728775272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 msgr2=0x55abc27e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc27e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 -- 172.21.15.7:0/1728775272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61010a2e0 con 0x55abc2770ff0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc60ffff700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc279e6c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098e0 tx=0x7fc60400b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad18e655c747a5f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.131+0000 7fc5effff700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc5effff700 1 -- 172.21.15.7:0/1728775272 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60400bb20 con 0x55abc2770ff0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc60800f040 con 0x55abc2770ff0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc5effff700 1 -- 172.21.15.7:0/1728775272 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc604003960 con 0x55abc2770ff0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc5effff700 1 -- 172.21.15.7:0/1728775272 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc604003b20 con 0x55abc2770ff0 2024-01-11T18:10:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc62ababb80 10 client.26527 fetch_fsmap learned FSMap version 45 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55abc2534c70 con 0x55abc2770ff0 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.132+0000 7fc5effff700 1 -- 172.21.15.7:0/1728775272 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60800f040 con 0x55abc2770ff0 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc5effff700 1 -- 172.21.15.7:0/1728775272 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc60402f070 con 0x55abc2770ff0 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc62ababb80 10 client.26527 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc62ababb80 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc62ababb80 20 client.26527 populate_metadata read hostname 'smithi007' 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc62ababb80 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc27f6740 0x55abc27f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc62ababb80 4 client.26527 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55abc26b4880 con 0x55abc27f6740 2024-01-11T18:10:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.133+0000 7fc61539c700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc27f6740 0x55abc27f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:10:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.134+0000 7fc61539c700 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc27f6740 0x55abc27f8b60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e11b8acc2a256b1 server_cookie=1794d98afc595d05 in_seq=0 out_seq=0 2024-01-11T18:10:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.134+0000 7fc5effff700 10 client.26527 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc5effff700 1 -- 172.21.15.7:0/1728775272 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55abc26b4880 con 0x55abc27f6740 2024-01-11T18:10:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc5effff700 10 client.26527 handle_command_reply: tid=0 2024-01-11T18:10:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 client.26527 shutdown 2024-01-11T18:10:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 msgr2=0x55abc279e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc279e6c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098e0 tx=0x7fc60400b900 comp rx=0 tx=0).stop 2024-01-11T18:10:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 shutdown_connections 2024-01-11T18:10:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55abc27f6740 0x55abc27f8b60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abc2771ad0 0x55abc27e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abc2770690 0x55abc279e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 --2- 172.21.15.7:0/1728775272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abc2770ff0 0x55abc279e6c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc5effff700 0 client.26527 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 >> 172.21.15.7:0/1728775272 conn(0x55abc26dd740 msgr2=0x55abc2778010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 shutdown_connections 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 1 -- 172.21.15.7:0/1728775272 wait complete. 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.149+0000 7fc62ababb80 20 client.26527 trim_cache size 0 max 16384 2024-01-11T18:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:10:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:10:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:10:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:10:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.150+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc6000a1fd0 msgr2=0x7fc6000a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.150+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc6000a1fd0 0x7fc6000a4490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc010b00 tx=0x7fc5fc00e000 comp rx=0 tx=0).stop 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.150+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 msgr2=0x7fc610199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.150+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 0x7fc610199780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00b550 tx=0x7fc5fc00b860 comp rx=0 tx=0).stop 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 shutdown_connections 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc6000a1fd0 0x7fc6000a4490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc610102020 0x7fc610199780 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc610107280 0x7fc6101a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 --2- 172.21.15.7:0/800586392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc610104950 0x7fc610199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 >> 172.21.15.7:0/800586392 conn(0x7fc6100fba30 msgr2=0x7fc610101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 shutdown_connections 2024-01-11T18:10:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:10:52.151+0000 7fc616dff700 1 -- 172.21.15.7:0/800586392 wait complete. 2024-01-11T18:10:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:10:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:10:52.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:52.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:10:52.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:10:52.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:52.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:10:52.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:10:52.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:10:53.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:53 smithi016 ceph-mon[43195]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-01-11T18:10:53.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:53 smithi038 ceph-mon[43582]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-01-11T18:10:53.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:53 smithi007 ceph-mon[33361]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-01-11T18:10:54.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 745.0 s, 525.2 tps, lat 189.583 ms stddev 276.342 2024-01-11T18:10:56.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:55 smithi016 ceph-mon[43195]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-01-11T18:10:56.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:55 smithi038 ceph-mon[43582]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-01-11T18:10:56.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:55 smithi007 ceph-mon[33361]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-01-11T18:10:57.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:57 smithi016 ceph-mon[43195]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.0 MiB/s wr, 519 op/s 2024-01-11T18:10:57.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:57 smithi038 ceph-mon[43582]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.0 MiB/s wr, 519 op/s 2024-01-11T18:10:57.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:57 smithi007 ceph-mon[33361]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.0 MiB/s wr, 519 op/s 2024-01-11T18:10:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 750.0 s, 682.4 tps, lat 150.348 ms stddev 201.737 2024-01-11T18:10:59.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:10:59 smithi016 ceph-mon[43195]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 412 op/s 2024-01-11T18:10:59.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:10:59 smithi038 ceph-mon[43582]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 412 op/s 2024-01-11T18:10:59.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:10:59 smithi007 ceph-mon[33361]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 412 op/s 2024-01-11T18:11:01.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:01 smithi016 ceph-mon[43195]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 488 op/s 2024-01-11T18:11:01.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:01 smithi038 ceph-mon[43582]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 488 op/s 2024-01-11T18:11:01.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:01 smithi007 ceph-mon[33361]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 488 op/s 2024-01-11T18:11:03.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:03 smithi016 ceph-mon[43195]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2024-01-11T18:11:03.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:03 smithi038 ceph-mon[43582]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2024-01-11T18:11:03.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:03 smithi007 ceph-mon[33361]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2024-01-11T18:11:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 755.0 s, 516.6 tps, lat 185.503 ms stddev 224.090 2024-01-11T18:11:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.424+0000 7fa5919ff700 1 -- 172.21.15.7:0/1829498331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 msgr2=0x7fa58c104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.424+0000 7fa5919ff700 1 --2- 172.21.15.7:0/1829498331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 0x7fa58c104410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009990 tx=0x7fa57c009ca0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 -- 172.21.15.7:0/1829498331 shutdown_connections 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 --2- 172.21.15.7:0/1829498331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 0x7fa58c104410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 --2- 172.21.15.7:0/1829498331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c107280 0x7fa58c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 --2- 172.21.15.7:0/1829498331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c104950 0x7fa58c106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 -- 172.21.15.7:0/1829498331 >> 172.21.15.7:0/1829498331 conn(0x7fa58c0fba30 msgr2=0x7fa58c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 -- 172.21.15.7:0/1829498331 shutdown_connections 2024-01-11T18:11:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.425+0000 7fa5919ff700 1 -- 172.21.15.7:0/1829498331 wait complete. 2024-01-11T18:11:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.427+0000 7fa5919ff700 1 Processor -- start 2024-01-11T18:11:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.428+0000 7fa5919ff700 1 -- start start 2024-01-11T18:11:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.429+0000 7fa5919ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 0x7fa58c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.429+0000 7fa5919ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 0x7fa58c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.429+0000 7fa5919ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c107280 0x7fa58c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.429+0000 7fa5919ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c19f000 con 0x7fa58c107280 2024-01-11T18:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.429+0000 7fa5919ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c19f140 con 0x7fa58c104950 2024-01-11T18:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.429+0000 7fa5919ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0ffe10 con 0x7fa58c102020 2024-01-11T18:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 0x7fa58c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 0x7fa58c1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47358/0 (socket says 172.21.15.7:47358) 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 -- 172.21.15.7:0/3352851823 learned_addr learned my addr 172.21.15.7:0/3352851823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58affd700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 0x7fa58c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 -- 172.21.15.7:0/3352851823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 msgr2=0x7fa58c197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 0x7fa58c197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 -- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c107280 msgr2=0x7fa58c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c107280 0x7fa58c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.430+0000 7fa58a7fc700 1 -- 172.21.15.7:0/3352851823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009c40 con 0x7fa58c104950 2024-01-11T18:11:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.431+0000 7fa58a7fc700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 0x7fa58c1979c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e9a0 tx=0x7fa58000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.431+0000 7fa5909fd700 1 -- 172.21.15.7:0/3352851823 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580009790 con 0x7fa58c104950 2024-01-11T18:11:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.431+0000 7fa5909fd700 1 -- 172.21.15.7:0/3352851823 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa58000cbd0 con 0x7fa58c104950 2024-01-11T18:11:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.431+0000 7fa5909fd700 1 -- 172.21.15.7:0/3352851823 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580018680 con 0x7fa58c104950 2024-01-11T18:11:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.431+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0095f0 con 0x7fa58c104950 2024-01-11T18:11:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.432+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c100420 con 0x7fa58c104950 2024-01-11T18:11:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.437+0000 7fa5909fd700 1 -- 172.21.15.7:0/3352851823 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa5800187e0 con 0x7fa58c104950 2024-01-11T18:11:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.438+0000 7fa5a57e1b80 1 Processor -- start 2024-01-11T18:11:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.439+0000 7fa5a57e1b80 1 -- start start 2024-01-11T18:11:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6818ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e68194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5909fd700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5740a1f50 0x7fa5740a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5909fd700 1 -- 172.21.15.7:0/3352851823 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa580014070 con 0x7fa58c104950 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e6819f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6707030 con 0x5629e6818ff0 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e676aac0 con 0x5629e6819ad0 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e67683e0 con 0x5629e6818690 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.440+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6818ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.441+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6818ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39628/0 (socket says 172.21.15.7:39628) 2024-01-11T18:11:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.441+0000 7fa58b7fe700 1 -- 172.21.15.7:0/3224434037 learned_addr learned my addr 172.21.15.7:0/3224434037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.441+0000 7fa58affd700 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e68194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.441+0000 7fa58a7fc700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5740a1f50 0x7fa5740a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.441+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e6819f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58a7fc700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5740a1f50 0x7fa5740a4410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa58009a8d0 tx=0x7fa58009a480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58affd700 1 -- 172.21.15.7:0/3224434037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 msgr2=0x5629e6818ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58affd700 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6818ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58affd700 1 -- 172.21.15.7:0/3224434037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 msgr2=0x5629e6819f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58affd700 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e6819f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58affd700 1 -- 172.21.15.7:0/3224434037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e65d8350 con 0x5629e6818ff0 2024-01-11T18:11:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.442+0000 7fa58affd700 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e68194f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa57c000c00 tx=0x7fa57c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f3ba04daa7c16c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.443+0000 7fa571ffb700 1 -- 172.21.15.7:0/3224434037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00beb0 con 0x5629e6818ff0 2024-01-11T18:11:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.443+0000 7fa571ffb700 1 -- 172.21.15.7:0/3224434037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa57c01e040 con 0x5629e6818ff0 2024-01-11T18:11:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.443+0000 7fa571ffb700 1 -- 172.21.15.7:0/3224434037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c004ad0 con 0x5629e6818ff0 2024-01-11T18:11:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3224434037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 msgr2=0x5629e68194f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e68194f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa57c000c00 tx=0x7fa57c005ad0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3224434037 shutdown_connections 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6818ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e6819f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3224434037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e68194f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3224434037 >> 172.21.15.7:0/3224434037 conn(0x5629e6785740 msgr2=0x5629e680a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3224434037 shutdown_connections 2024-01-11T18:11:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.444+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3224434037 wait complete. 2024-01-11T18:11:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.445+0000 7fa5a57e1b80 1 -- start start 2024-01-11T18:11:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.445+0000 7fa5a57e1b80 1 Processor -- start 2024-01-11T18:11:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818690 0x5629e68220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818ff0 0x5629e6822620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6829660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6763cd0 con 0x5629e6818690 2024-01-11T18:11:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e66d2340 con 0x5629e6818ff0 2024-01-11T18:11:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e66d1fe0 con 0x5629e6819ad0 2024-01-11T18:11:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa5a57e1b80 10 client.0 _ll_register_callbacks cb 0x5629e67046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818690 0x5629e68220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818690 0x5629e68220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50222/0 (socket says 172.21.15.7:50222) 2024-01-11T18:11:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.446+0000 7fa58b7fe700 1 -- 172.21.15.7:0/1821324548 learned_addr learned my addr 172.21.15.7:0/1821324548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58affd700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818ff0 0x5629e6822620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6829660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58affd700 1 -- 172.21.15.7:0/1821324548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 msgr2=0x5629e6829660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58affd700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6829660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58affd700 1 -- 172.21.15.7:0/1821324548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818690 msgr2=0x5629e68220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58affd700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818690 0x5629e68220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.447+0000 7fa58affd700 1 -- 172.21.15.7:0/1821324548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c10a2e0 con 0x5629e6818ff0 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.448+0000 7fa58affd700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818ff0 0x5629e6822620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0098e0 tx=0x7fa57c005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e22af5375bfcfcb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.448+0000 7fa56affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:11:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.448+0000 7fa56affd700 1 -- 172.21.15.7:0/1821324548 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c005040 con 0x5629e6818ff0 2024-01-11T18:11:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.448+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629e6802d60 con 0x5629e6818ff0 2024-01-11T18:11:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.448+0000 7fa56affd700 1 -- 172.21.15.7:0/1821324548 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa57c0051a0 con 0x5629e6818ff0 2024-01-11T18:11:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.449+0000 7fa56affd700 1 -- 172.21.15.7:0/1821324548 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa57c005360 con 0x5629e6818ff0 2024-01-11T18:11:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.450+0000 7fa5a57e1b80 10 client.26504 fetch_fsmap learned FSMap version 45 2024-01-11T18:11:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.450+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5629e6584b00 con 0x5629e6818ff0 2024-01-11T18:11:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.452+0000 7fa56affd700 1 -- 172.21.15.7:0/1821324548 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629e6802d60 con 0x5629e6818ff0 2024-01-11T18:11:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.452+0000 7fa56affd700 1 -- 172.21.15.7:0/1821324548 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa57c00fc10 con 0x5629e6818ff0 2024-01-11T18:11:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.453+0000 7fa5a57e1b80 10 client.26504 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:11:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.453+0000 7fa5a57e1b80 10 client.26504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.453+0000 7fa5a57e1b80 20 client.26504 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.453+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e682daa0 0x5629e682fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.453+0000 7fa5a57e1b80 4 client.26504 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:11:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.453+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629e675cb60 con 0x5629e682daa0 2024-01-11T18:11:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.454+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e682daa0 0x5629e682fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.454+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e682daa0 0x5629e682fe90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c10d9493c0fca30 server_cookie=23e01b0e0ca0a425 in_seq=0 out_seq=0 2024-01-11T18:11:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.454+0000 7fa56affd700 10 client.26504 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa56affd700 1 -- 172.21.15.7:0/1821324548 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629e675cb60 con 0x5629e682daa0 2024-01-11T18:11:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa56affd700 10 client.26504 handle_command_reply: tid=0 2024-01-11T18:11:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 client.26504 shutdown 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818ff0 msgr2=0x5629e6822620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818ff0 0x5629e6822620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0098e0 tx=0x7fa57c005a00 comp rx=0 tx=0).stop 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 shutdown_connections 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e682daa0 0x5629e682fe90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6829660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818690 0x5629e68220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/1821324548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818ff0 0x5629e6822620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa56affd700 0 client.26504 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.455+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 >> 172.21.15.7:0/1821324548 conn(0x5629e6785740 msgr2=0x5629e6809f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.456+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 shutdown_connections 2024-01-11T18:11:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.456+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/1821324548 wait complete. 2024-01-11T18:11:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.456+0000 7fa5a57e1b80 20 client.26504 trim_cache size 0 max 16384 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 Processor -- start 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 -- start start 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818690 0x5629e6821660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6829620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6819eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e65da640 con 0x5629e6818ff0 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e675cb60 con 0x5629e6818690 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.496+0000 7fa5a57e1b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e682a7e0 con 0x5629e6819ad0 2024-01-11T18:11:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.497+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6819eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.497+0000 7fa58affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6829620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.498+0000 7fa58affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6829620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50232/0 (socket says 172.21.15.7:50232) 2024-01-11T18:11:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.498+0000 7fa58affd700 1 -- 172.21.15.7:0/2486051494 learned_addr learned my addr 172.21.15.7:0/2486051494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.498+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818690 0x5629e6821660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.499+0000 7fa58affd700 1 -- 172.21.15.7:0/2486051494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 msgr2=0x5629e6819eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.499+0000 7fa58affd700 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6819eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.499+0000 7fa58affd700 1 -- 172.21.15.7:0/2486051494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818690 msgr2=0x5629e6821660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.499+0000 7fa58affd700 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818690 0x5629e6821660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.499+0000 7fa58affd700 1 -- 172.21.15.7:0/2486051494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e67251d0 con 0x5629e6818ff0 2024-01-11T18:11:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.500+0000 7fa58affd700 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6829620 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00ffd0 tx=0x7fa57c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f056aaa3d277cda server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.503+0000 7fa5717fa700 1 -- 172.21.15.7:0/2486051494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c02b7e0 con 0x5629e6818ff0 2024-01-11T18:11:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.503+0000 7fa5717fa700 1 -- 172.21.15.7:0/2486051494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa57c009290 con 0x5629e6818ff0 2024-01-11T18:11:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.503+0000 7fa5717fa700 1 -- 172.21.15.7:0/2486051494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00fc30 con 0x5629e6818ff0 2024-01-11T18:11:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.504+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/2486051494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 msgr2=0x5629e6829620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.504+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6829620 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00ffd0 tx=0x7fa57c005a90 comp rx=0 tx=0).stop 2024-01-11T18:11:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/2486051494 shutdown_connections 2024-01-11T18:11:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6819ad0 0x5629e6819eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6818690 0x5629e6821660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/2486051494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6829620 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/2486051494 >> 172.21.15.7:0/2486051494 conn(0x5629e6785740 msgr2=0x5629e68200c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/2486051494 shutdown_connections 2024-01-11T18:11:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.506+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/2486051494 wait complete. 2024-01-11T18:11:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.507+0000 7fa5a57e1b80 1 -- start start 2024-01-11T18:11:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.507+0000 7fa5a57e1b80 1 Processor -- start 2024-01-11T18:11:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.510+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6826be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.510+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6827120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.510+0000 7fa5a57e1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e68280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.510+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6826be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.511+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6826be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:39650/0 (socket says 172.21.15.7:39650) 2024-01-11T18:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.511+0000 7fa58affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6827120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.511+0000 7fa58affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6827120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50242/0 (socket says 172.21.15.7:50242) 2024-01-11T18:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.513+0000 7fa58b7fe700 1 -- 172.21.15.7:0/3973913667 learned_addr learned my addr 172.21.15.7:0/3973913667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.513+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6828620 con 0x5629e6818ff0 2024-01-11T18:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.513+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6828790 con 0x5629e6819ad0 2024-01-11T18:11:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.513+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6828950 con 0x5629e6818690 2024-01-11T18:11:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.513+0000 7fa5a57e1b80 10 client.0 _ll_register_callbacks cb 0x5629e67046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.513+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e68280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.514+0000 7fa58b7fe700 1 -- 172.21.15.7:0/3973913667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 msgr2=0x5629e6826be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.514+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6826be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.514+0000 7fa58b7fe700 1 -- 172.21.15.7:0/3973913667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 msgr2=0x5629e6827120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.514+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6827120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.515+0000 7fa58b7fe700 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629e6584b00 con 0x5629e6819ad0 2024-01-11T18:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.515+0000 7fa58b7fe700 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e68280e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5629e6830740 tx=0x7fa57c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc245b729d51687c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.515+0000 7fa56affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.515+0000 7fa56affd700 1 -- 172.21.15.7:0/3973913667 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400e040 con 0x5629e6819ad0 2024-01-11T18:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.516+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa57c01e040 con 0x5629e6819ad0 2024-01-11T18:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.516+0000 7fa56affd700 1 -- 172.21.15.7:0/3973913667 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa584004bb0 con 0x5629e6819ad0 2024-01-11T18:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.516+0000 7fa56affd700 1 -- 172.21.15.7:0/3973913667 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa584003e50 con 0x5629e6819ad0 2024-01-11T18:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.516+0000 7fa5a57e1b80 10 client.26510 fetch_fsmap learned FSMap version 45 2024-01-11T18:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.516+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5629e65dcc70 con 0x5629e6819ad0 2024-01-11T18:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.516+0000 7fa56affd700 1 -- 172.21.15.7:0/3973913667 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa57c01e040 con 0x5629e6819ad0 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa56affd700 1 -- 172.21.15.7:0/3973913667 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa58402b9d0 con 0x5629e6819ad0 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa5a57e1b80 10 client.26510 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa5a57e1b80 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa5a57e1b80 20 client.26510 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e6895740 0x5629e6897b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa5a57e1b80 4 client.26510 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629e675c880 con 0x5629e6895740 2024-01-11T18:11:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.517+0000 7fa58affd700 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e6895740 0x5629e6897b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.518+0000 7fa58affd700 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e6895740 0x5629e6897b60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4beb81c1f57b1c8d server_cookie=8df992d6c04a92ad in_seq=0 out_seq=0 2024-01-11T18:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.518+0000 7fa56affd700 10 client.26510 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.558+0000 7fa56affd700 1 -- 172.21.15.7:0/3973913667 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629e675c880 con 0x5629e6895740 2024-01-11T18:11:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.558+0000 7fa56affd700 10 client.26510 handle_command_reply: tid=0 2024-01-11T18:11:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.558+0000 7fa5a57e1b80 1 client.26510 shutdown 2024-01-11T18:11:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 msgr2=0x5629e68280e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e68280e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5629e6830740 tx=0x7fa57c005c00 comp rx=0 tx=0).stop 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 shutdown_connections 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5629e6895740 0x5629e6897b60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa56affd700 0 client.26510 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629e6818690 0x5629e6826be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629e6819ad0 0x5629e68280e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 --2- 172.21.15.7:0/3973913667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629e6818ff0 0x5629e6827120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 >> 172.21.15.7:0/3973913667 conn(0x5629e6785740 msgr2=0x5629e681fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.559+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 shutdown_connections 2024-01-11T18:11:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.560+0000 7fa5a57e1b80 1 -- 172.21.15.7:0/3973913667 wait complete. 2024-01-11T18:11:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.561+0000 7fa5a57e1b80 20 client.26510 trim_cache size 0 max 16384 2024-01-11T18:11:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:11:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:11:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:11:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:11:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5740a1f50 msgr2=0x7fa5740a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5740a1f50 0x7fa5740a4410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa58009a8d0 tx=0x7fa58009a480 comp rx=0 tx=0).stop 2024-01-11T18:11:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 msgr2=0x7fa58c1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 0x7fa58c1979c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e9a0 tx=0x7fa58000ed60 comp rx=0 tx=0).stop 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 shutdown_connections 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa5740a1f50 0x7fa5740a4410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa58c102020 0x7fa58c197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c107280 0x7fa58c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 --2- 172.21.15.7:0/3352851823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c104950 0x7fa58c1979c0 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e9a0 tx=0x7fa58000ed60 comp rx=0 tx=0).stop 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.562+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 >> 172.21.15.7:0/3352851823 conn(0x7fa58c0fba30 msgr2=0x7fa58c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.563+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 shutdown_connections 2024-01-11T18:11:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:05.563+0000 7fa5919ff700 1 -- 172.21.15.7:0/3352851823 wait complete. 2024-01-11T18:11:06.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:05 smithi016 ceph-mon[43195]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.5 MiB/s wr, 467 op/s 2024-01-11T18:11:06.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:05 smithi038 ceph-mon[43582]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.5 MiB/s wr, 467 op/s 2024-01-11T18:11:06.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:05 smithi007 ceph-mon[33361]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.5 MiB/s wr, 467 op/s 2024-01-11T18:11:07.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:07 smithi016 ceph-mon[43195]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2024-01-11T18:11:07.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:07 smithi038 ceph-mon[43582]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2024-01-11T18:11:07.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:07 smithi007 ceph-mon[33361]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2024-01-11T18:11:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 760.0 s, 472.4 tps, lat 215.624 ms stddev 258.275 2024-01-11T18:11:09.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:09 smithi016 ceph-mon[43195]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 422 op/s 2024-01-11T18:11:09.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:09 smithi038 ceph-mon[43582]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 422 op/s 2024-01-11T18:11:09.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:09 smithi007 ceph-mon[33361]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 422 op/s 2024-01-11T18:11:11.490 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:11 smithi016 ceph-mon[43195]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-01-11T18:11:11.521 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:11 smithi038 ceph-mon[43582]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-01-11T18:11:11.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:11 smithi007 ceph-mon[33361]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-01-11T18:11:13.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:13 smithi016 ceph-mon[43195]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2024-01-11T18:11:13.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:13 smithi038 ceph-mon[43582]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2024-01-11T18:11:13.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:13 smithi007 ceph-mon[33361]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2024-01-11T18:11:14.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 765.0 s, 633.3 tps, lat 153.531 ms stddev 214.305 2024-01-11T18:11:16.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:15 smithi016 ceph-mon[43195]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 478 op/s 2024-01-11T18:11:16.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:15 smithi038 ceph-mon[43582]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 478 op/s 2024-01-11T18:11:16.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:15 smithi007 ceph-mon[33361]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 478 op/s 2024-01-11T18:11:17.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:17 smithi016 ceph-mon[43195]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 622 op/s 2024-01-11T18:11:17.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:17 smithi038 ceph-mon[43582]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 622 op/s 2024-01-11T18:11:17.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:17 smithi007 ceph-mon[33361]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 622 op/s 2024-01-11T18:11:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.931+0000 7f821b7ff700 1 -- 172.21.15.7:0/3006740343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 msgr2=0x7f821406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.931+0000 7f821b7ff700 1 --2- 172.21.15.7:0/3006740343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f821406ea80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82040098f0 tx=0x7f8204009c00 comp rx=0 tx=0).stop 2024-01-11T18:11:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 -- 172.21.15.7:0/3006740343 shutdown_connections 2024-01-11T18:11:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 --2- 172.21.15.7:0/3006740343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406efc0 0x7f821406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 --2- 172.21.15.7:0/3006740343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821406d3c0 0x7f821406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 --2- 172.21.15.7:0/3006740343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f821406ea80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 -- 172.21.15.7:0/3006740343 >> 172.21.15.7:0/3006740343 conn(0x7f821406bf50 msgr2=0x7f821406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 -- 172.21.15.7:0/3006740343 shutdown_connections 2024-01-11T18:11:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.932+0000 7f821b7ff700 1 -- 172.21.15.7:0/3006740343 wait complete. 2024-01-11T18:11:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.933+0000 7f821b7ff700 1 Processor -- start 2024-01-11T18:11:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 -- start start 2024-01-11T18:11:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406d3c0 0x7f82141a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f82141a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821406efc0 0x7f82141ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141aded0 con 0x7f821406efc0 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141ae010 con 0x7f821406e600 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.934+0000 7f821b7ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141ae1a0 con 0x7f821406d3c0 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f821959b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406d3c0 0x7f82141a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f82141a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f821959b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406d3c0 0x7f82141a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:36158/0 (socket says 172.21.15.7:36158) 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f821959b700 1 -- 172.21.15.7:0/2781506488 learned_addr learned my addr 172.21.15.7:0/2781506488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8219d9c700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821406efc0 0x7f82141ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 -- 172.21.15.7:0/2781506488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406d3c0 msgr2=0x7f82141a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406d3c0 0x7f82141a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 -- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821406efc0 msgr2=0x7f82141ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821406efc0 0x7f82141ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 -- 172.21.15.7:0/2781506488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009520 con 0x7f821406e600 2024-01-11T18:11:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.935+0000 7f8218d9a700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f82141a6890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8204000c00 tx=0x7f8204005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.936+0000 7f820a7fc700 1 -- 172.21.15.7:0/2781506488 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820401c070 con 0x7f821406e600 2024-01-11T18:11:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.936+0000 7f820a7fc700 1 -- 172.21.15.7:0/2781506488 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820400bb20 con 0x7f821406e600 2024-01-11T18:11:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.936+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82141ae420 con 0x7f821406e600 2024-01-11T18:11:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.936+0000 7f820a7fc700 1 -- 172.21.15.7:0/2781506488 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040038e0 con 0x7f821406e600 2024-01-11T18:11:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.936+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141ae8a0 con 0x7f821406e600 2024-01-11T18:11:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.939+0000 7f820a7fc700 1 -- 172.21.15.7:0/2781506488 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f8204020030 con 0x7f821406e600 2024-01-11T18:11:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.940+0000 7f822f50bb80 1 Processor -- start 2024-01-11T18:11:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.940+0000 7f822f50bb80 1 -- start start 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053bdf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f820a7fc700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82000a1f50 0x7f82000a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdfc10 0x558053be0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053be6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f822f50bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053ad6030 con 0x558053bdef20 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f822f50bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053b39ac0 con 0x558053be05d0 2024-01-11T18:11:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f822f50bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053b373e0 con 0x558053bdfc10 2024-01-11T18:11:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f820a7fc700 1 -- 172.21.15.7:0/2781506488 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f82040ceb50 con 0x7f821406e600 2024-01-11T18:11:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053bdf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f8219d9c700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82000a1f50 0x7f82000a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053bdf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49378/0 (socket says 172.21.15.7:49378) 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 -- 172.21.15.7:0/2450429490 learned_addr learned my addr 172.21.15.7:0/2450429490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f8218d9a700 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdfc10 0x558053be0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f8219d9c700 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053be6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 -- 172.21.15.7:0/2450429490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdfc10 msgr2=0x558053be0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdfc10 0x558053be0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 -- 172.21.15.7:0/2450429490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 msgr2=0x558053be6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053be6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.941+0000 7f821959b700 1 -- 172.21.15.7:0/2450429490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580539a7350 con 0x558053bdef20 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.942+0000 7f821959b700 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053bdf340 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8210009e50 tx=0x7f8210011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c927de91901ea1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.942+0000 7f8219d9c700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82000a1f50 0x7f82000a4410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f820c016b10 tx=0x7f820c0099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.942+0000 7f81ff7fe700 1 -- 172.21.15.7:0/2450429490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210004a30 con 0x558053bdef20 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.942+0000 7f81ff7fe700 1 -- 172.21.15.7:0/2450429490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82100054e0 con 0x558053bdef20 2024-01-11T18:11:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.942+0000 7f81ff7fe700 1 -- 172.21.15.7:0/2450429490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82100198f0 con 0x558053bdef20 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 -- 172.21.15.7:0/2450429490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 msgr2=0x558053bdf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053bdf340 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8210009e50 tx=0x7f8210011720 comp rx=0 tx=0).stop 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 -- 172.21.15.7:0/2450429490 shutdown_connections 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053be6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053bdf340 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 --2- 172.21.15.7:0/2450429490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdfc10 0x558053be0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 -- 172.21.15.7:0/2450429490 >> 172.21.15.7:0/2450429490 conn(0x558053b54740 msgr2=0x558053bd9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 -- 172.21.15.7:0/2450429490 shutdown_connections 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 -- 172.21.15.7:0/2450429490 wait complete. 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.943+0000 7f822f50bb80 1 -- start start 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 Processor -- start 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdfc10 0x558053be8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053befe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053b32cd0 con 0x558053bdfc10 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053aa1340 con 0x558053be05d0 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053aa0fe0 con 0x558053bdef20 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f822f50bb80 10 client.0 _ll_register_callbacks cb 0x558053ad36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.944+0000 7f821959b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f8218d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdfc10 0x558053be8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:36178/0 (socket says 172.21.15.7:36178) 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 -- 172.21.15.7:0/3854896946 learned_addr learned my addr 172.21.15.7:0/3854896946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f8219d9c700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053befe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 -- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 msgr2=0x558053befe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053befe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 -- 172.21.15.7:0/3854896946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdfc10 msgr2=0x558053be8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdfc10 0x558053be8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 -- 172.21.15.7:0/3854896946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8210009a80 con 0x558053bdef20 2024-01-11T18:11:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f8218d9a700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdfc10 0x558053be8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f821959b700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be88a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f821001b040 tx=0x7f8210004c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a8f934f25e4ad91 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f81fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f81fcff9700 1 -- 172.21.15.7:0/3854896946 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210005320 con 0x558053bdef20 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.945+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558053bd1d60 con 0x558053bdef20 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.946+0000 7f81fcff9700 1 -- 172.21.15.7:0/3854896946 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8210004040 con 0x558053bdef20 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.946+0000 7f81fcff9700 1 -- 172.21.15.7:0/3854896946 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8210004200 con 0x558053bdef20 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.946+0000 7f822f50bb80 10 client.26557 fetch_fsmap learned FSMap version 45 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.946+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x558053953b00 con 0x558053bdef20 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.946+0000 7f81fcff9700 1 -- 172.21.15.7:0/3854896946 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558053bd1d60 con 0x558053bdef20 2024-01-11T18:11:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f81fcff9700 1 -- 172.21.15.7:0/3854896946 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8210021040 con 0x558053bdef20 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f822f50bb80 10 client.26557 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f822f50bb80 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f822f50bb80 20 client.26557 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f822f50bb80 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053bf4580 0x558053bf69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f822f50bb80 4 client.26557 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558053b2bb60 con 0x558053bf4580 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.947+0000 7f8218d9a700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053bf4580 0x558053bf69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f8218d9a700 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053bf4580 0x558053bf69a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17bcf8d88ff71215 server_cookie=ec54db6f3cd4ffa3 in_seq=0 out_seq=0 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f81fcff9700 10 client.26557 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f81fcff9700 1 -- 172.21.15.7:0/3854896946 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558053b2bb60 con 0x558053bf4580 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f81fcff9700 10 client.26557 handle_command_reply: tid=0 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f822f50bb80 1 client.26557 shutdown 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 msgr2=0x558053be88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.948+0000 7f822f50bb80 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be88a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f821001b040 tx=0x7f8210004c90 comp rx=0 tx=0).stop 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 shutdown_connections 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053bf4580 0x558053bf69a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be88a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053be05d0 0x558053befe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 --2- 172.21.15.7:0/3854896946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdfc10 0x558053be8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 >> 172.21.15.7:0/3854896946 conn(0x558053b54740 msgr2=0x558053bd8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f81fcff9700 0 client.26557 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 shutdown_connections 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 1 -- 172.21.15.7:0/3854896946 wait complete. 2024-01-11T18:11:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.949+0000 7f822f50bb80 20 client.26557 trim_cache size 0 max 16384 2024-01-11T18:11:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.964+0000 7f822f50bb80 1 Processor -- start 2024-01-11T18:11:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 -- start start 2024-01-11T18:11:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053befde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053be05d0 0x558053bfb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580539a9640 con 0x558053be05d0 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053bf4120 con 0x558053bdfc10 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f822f50bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053bdb020 con 0x558053bdef20 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f821959b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f821959b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:36186/0 (socket says 172.21.15.7:36186) 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f821959b700 1 -- 172.21.15.7:0/1238882825 learned_addr learned my addr 172.21.15.7:0/1238882825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f8219d9c700 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053be05d0 0x558053bfb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.965+0000 7f8218d9a700 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053befde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f821959b700 1 -- 172.21.15.7:0/1238882825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 msgr2=0x558053befde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f821959b700 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053befde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f821959b700 1 -- 172.21.15.7:0/1238882825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053be05d0 msgr2=0x558053bfb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f821959b700 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053be05d0 0x558053bfb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f821959b700 1 -- 172.21.15.7:0/1238882825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558053af41d0 con 0x558053bdef20 2024-01-11T18:11:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f8218d9a700 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053befde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:11:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f821959b700 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be6b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8210005a80 tx=0x7f8210009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=face1f3d799553ad server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f81feffd700 1 -- 172.21.15.7:0/1238882825 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210005120 con 0x558053bdef20 2024-01-11T18:11:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f81feffd700 1 -- 172.21.15.7:0/1238882825 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8210005280 con 0x558053bdef20 2024-01-11T18:11:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.966+0000 7f81feffd700 1 -- 172.21.15.7:0/1238882825 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210019e90 con 0x558053bdef20 2024-01-11T18:11:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 -- 172.21.15.7:0/1238882825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 msgr2=0x558053be6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be6b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8210005a80 tx=0x7f8210009e20 comp rx=0 tx=0).stop 2024-01-11T18:11:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 -- 172.21.15.7:0/1238882825 shutdown_connections 2024-01-11T18:11:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053bdef20 0x558053be6b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053be05d0 0x558053bfb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 --2- 172.21.15.7:0/1238882825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053befde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 -- 172.21.15.7:0/1238882825 >> 172.21.15.7:0/1238882825 conn(0x558053b54740 msgr2=0x558053be7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 -- 172.21.15.7:0/1238882825 shutdown_connections 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 -- 172.21.15.7:0/1238882825 wait complete. 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 -- start start 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.967+0000 7f822f50bb80 1 Processor -- start 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053becc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053bed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053be05d0 0x558053c48aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053bed820 con 0x558053bdef20 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053c48fe0 con 0x558053bdfc10 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558053c49120 con 0x558053be05d0 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f822f50bb80 10 client.0 _ll_register_callbacks cb 0x558053ad36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f8218d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053bed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f8219d9c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053be05d0 0x558053c48aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f8218d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053bed140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35838/0 (socket says 172.21.15.7:35838) 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f8218d9a700 1 -- 172.21.15.7:0/838404769 learned_addr learned my addr 172.21.15.7:0/838404769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.968+0000 7f821959b700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053becc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f8218d9a700 1 -- 172.21.15.7:0/838404769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053be05d0 msgr2=0x558053c48aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f8218d9a700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053be05d0 0x558053c48aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f8218d9a700 1 -- 172.21.15.7:0/838404769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 msgr2=0x558053becc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f8218d9a700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053becc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f8218d9a700 1 -- 172.21.15.7:0/838404769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141ae8a0 con 0x558053bdfc10 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f821959b700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053becc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f8218d9a700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053bed140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f82040cc2c0 tx=0x7f8204005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f93ad7ad27e3a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f81fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f81fcff9700 1 -- 172.21.15.7:0/838404769 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204014920 con 0x558053bdfc10 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82100054e0 con 0x558053bdfc10 2024-01-11T18:11:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f81fcff9700 1 -- 172.21.15.7:0/838404769 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f820409b5b0 con 0x558053bdfc10 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f81fcff9700 1 -- 172.21.15.7:0/838404769 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820409f700 con 0x558053bdfc10 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f822f50bb80 10 client.26528 fetch_fsmap learned FSMap version 45 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x558053953b00 con 0x558053bdfc10 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.969+0000 7f81fcff9700 1 -- 172.21.15.7:0/838404769 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82100054e0 con 0x558053bdfc10 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f81fcff9700 1 -- 172.21.15.7:0/838404769 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f82040051d0 con 0x558053bdfc10 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f822f50bb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 45 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f822f50bb80 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f822f50bb80 20 client.26528 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f822f50bb80 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053c4a8d0 0x558053c4ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f822f50bb80 4 client.26528 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558053b2b880 con 0x558053c4a8d0 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f821959b700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053c4a8d0 0x558053c4ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f821959b700 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053c4a8d0 0x558053c4ccf0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e23ff85e22a31c6 server_cookie=5a3a90cb0d071ed4 in_seq=0 out_seq=0 2024-01-11T18:11:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:18.970+0000 7f81fcff9700 10 client.26528 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f81fcff9700 1 -- 172.21.15.7:0/838404769 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558053b2b880 con 0x558053c4a8d0 2024-01-11T18:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f81fcff9700 10 client.26528 handle_command_reply: tid=0 2024-01-11T18:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 client.26528 shutdown 2024-01-11T18:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 msgr2=0x558053bed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053bed140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f82040cc2c0 tx=0x7f8204005570 comp rx=0 tx=0).stop 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 shutdown_connections 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x558053c4a8d0 0x558053c4ccf0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558053be05d0 0x558053c48aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558053bdef20 0x558053becc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 --2- 172.21.15.7:0/838404769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558053bdfc10 0x558053bed140 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 >> 172.21.15.7:0/838404769 conn(0x558053b54740 msgr2=0x558053be6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.025+0000 7f81fcff9700 0 client.26528 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.026+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 shutdown_connections 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.026+0000 7f822f50bb80 1 -- 172.21.15.7:0/838404769 wait complete. 2024-01-11T18:11:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.026+0000 7f822f50bb80 20 client.26528 trim_cache size 0 max 16384 2024-01-11T18:11:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:11:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:11:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:11:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82000a1f50 msgr2=0x7f82000a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82000a1f50 0x7f82000a4410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f820c016b10 tx=0x7f820c0099e0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 msgr2=0x7f82141a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f82141a6890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8204000c00 tx=0x7f8204005d90 comp rx=0 tx=0).stop 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 shutdown_connections 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f82000a1f50 0x7f82000a4410 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f821406d3c0 0x7f82141a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821406efc0 0x7f82141ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 --2- 172.21.15.7:0/2781506488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f821406e600 0x7f82141a6890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.027+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 >> 172.21.15.7:0/2781506488 conn(0x7f821406bf50 msgr2=0x7f821410a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.028+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 shutdown_connections 2024-01-11T18:11:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:19.028+0000 7f821b7ff700 1 -- 172.21.15.7:0/2781506488 wait complete. 2024-01-11T18:11:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 770.0 s, 442.3 tps, lat 231.637 ms stddev 313.880 2024-01-11T18:11:19.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:19 smithi016 ceph-mon[43195]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 571 op/s 2024-01-11T18:11:19.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:19 smithi038 ceph-mon[43582]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 571 op/s 2024-01-11T18:11:19.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:19 smithi007 ceph-mon[33361]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 571 op/s 2024-01-11T18:11:21.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:21 smithi016 ceph-mon[43195]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.7 MiB/s wr, 636 op/s 2024-01-11T18:11:21.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:21 smithi038 ceph-mon[43582]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.7 MiB/s wr, 636 op/s 2024-01-11T18:11:21.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:21 smithi007 ceph-mon[33361]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.7 MiB/s wr, 636 op/s 2024-01-11T18:11:23.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:23 smithi016 ceph-mon[43195]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 654 op/s 2024-01-11T18:11:23.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:23 smithi038 ceph-mon[43582]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 654 op/s 2024-01-11T18:11:23.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:23 smithi007 ceph-mon[33361]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 654 op/s 2024-01-11T18:11:24.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 775.0 s, 469.8 tps, lat 213.759 ms stddev 255.297 2024-01-11T18:11:26.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:25 smithi016 ceph-mon[43195]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-01-11T18:11:26.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:25 smithi038 ceph-mon[43582]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-01-11T18:11:26.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:25 smithi007 ceph-mon[33361]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-01-11T18:11:27.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:27 smithi016 ceph-mon[43195]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 606 op/s 2024-01-11T18:11:27.522 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:27 smithi038 ceph-mon[43582]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 606 op/s 2024-01-11T18:11:27.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:27 smithi007 ceph-mon[33361]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 606 op/s 2024-01-11T18:11:29.113 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 780.0 s, 450.0 tps, lat 219.696 ms stddev 254.996 2024-01-11T18:11:29.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:29 smithi016 ceph-mon[43195]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-01-11T18:11:29.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:29 smithi038 ceph-mon[43582]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-01-11T18:11:29.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:29 smithi007 ceph-mon[33361]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-01-11T18:11:32.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:31 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:11:32.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:31 smithi016 ceph-mon[43195]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 -- 172.21.15.7:0/1404138715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 msgr2=0x7fa3681002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 --2- 172.21.15.7:0/1404138715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa3681002b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa358009990 tx=0x7fa358009ca0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 -- 172.21.15.7:0/1404138715 shutdown_connections 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 --2- 172.21.15.7:0/1404138715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa3681002b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 --2- 172.21.15.7:0/1404138715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3681011b0 0x7fa368101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 --2- 172.21.15.7:0/1404138715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681007f0 0x7fa368100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 -- 172.21.15.7:0/1404138715 >> 172.21.15.7:0/1404138715 conn(0x7fa3680fba30 msgr2=0x7fa3680fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.251+0000 7fa36ddff700 1 -- 172.21.15.7:0/1404138715 shutdown_connections 2024-01-11T18:11:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.252+0000 7fa36ddff700 1 -- 172.21.15.7:0/1404138715 wait complete. 2024-01-11T18:11:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 Processor -- start 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 -- start start 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa368199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3681007f0 0x7fa368199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681011b0 0x7fa3681a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681a11a0 con 0x7fa3681011b0 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36806b030 con 0x7fa3681007f0 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.253+0000 7fa36ddff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36806b1c0 con 0x7fa3680ffe90 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa368199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa368199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33646/0 (socket says 172.21.15.7:33646) 2024-01-11T18:11:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 -- 172.21.15.7:0/2858785528 learned_addr learned my addr 172.21.15.7:0/2858785528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 -- 172.21.15.7:0/2858785528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3681007f0 msgr2=0x7fa368199b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3681007f0 0x7fa368199b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 -- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681011b0 msgr2=0x7fa3681a0bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681011b0 0x7fa3681a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.254+0000 7fa3677fe700 1 -- 172.21.15.7:0/2858785528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580095f0 con 0x7fa3680ffe90 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.255+0000 7fa3677fe700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa368199620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa358004730 tx=0x7fa358005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.255+0000 7fa364ff9700 1 -- 172.21.15.7:0/2858785528 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35801d070 con 0x7fa3680ffe90 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.255+0000 7fa364ff9700 1 -- 172.21.15.7:0/2858785528 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa358004cd0 con 0x7fa3680ffe90 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.255+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36806b440 con 0x7fa3680ffe90 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.255+0000 7fa364ff9700 1 -- 172.21.15.7:0/2858785528 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358005670 con 0x7fa3680ffe90 2024-01-11T18:11:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.256+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36806b8f0 con 0x7fa3680ffe90 2024-01-11T18:11:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.259+0000 7fa364ff9700 1 -- 172.21.15.7:0/2858785528 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa35800fc70 con 0x7fa3680ffe90 2024-01-11T18:11:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.262+0000 7fa381b97b80 1 Processor -- start 2024-01-11T18:11:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.262+0000 7fa381b97b80 1 -- start start 2024-01-11T18:11:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa364ff9700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3500a2020 0x7fa3500a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa364ff9700 1 -- 172.21.15.7:0/2858785528 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa3580ceef0 con 0x7fa3680ffe90 2024-01-11T18:11:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 0x55fcd9f29340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa381b97b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9e20030 con 0x55fcd9f29c10 2024-01-11T18:11:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa381b97b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9e83ac0 con 0x55fcd9f28f20 2024-01-11T18:11:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa381b97b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9e813e0 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa367fff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3500a2020 0x7fa3500a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.263+0000 7fa3677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 0x55fcd9f29340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 0x55fcd9f29340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39674/0 (socket says 172.21.15.7:39674) 2024-01-11T18:11:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 -- 172.21.15.7:0/3679180525 learned_addr learned my addr 172.21.15.7:0/3679180525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa366ffd700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 -- 172.21.15.7:0/3679180525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 msgr2=0x55fcd9f30e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa367fff700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f30e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 -- 172.21.15.7:0/3679180525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 msgr2=0x55fcd9f2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 -- 172.21.15.7:0/3679180525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd9cf1350 con 0x55fcd9f28f20 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa366ffd700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa367fff700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.264+0000 7fa3677fe700 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 0x55fcd9f29340 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3580a3720 tx=0x7fa358005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca78e24c62dcdce server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.265+0000 7fa34dffb700 1 -- 172.21.15.7:0/3679180525 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3580b3070 con 0x55fcd9f28f20 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.265+0000 7fa34dffb700 1 -- 172.21.15.7:0/3679180525 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3580a3c50 con 0x55fcd9f28f20 2024-01-11T18:11:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.265+0000 7fa34dffb700 1 -- 172.21.15.7:0/3679180525 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35809bbb0 con 0x55fcd9f28f20 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.266+0000 7fa381b97b80 1 -- 172.21.15.7:0/3679180525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 msgr2=0x55fcd9f29340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.266+0000 7fa381b97b80 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 0x55fcd9f29340 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3580a3720 tx=0x7fa358005a90 comp rx=0 tx=0).stop 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.266+0000 7fa367fff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3500a2020 0x7fa3500a44e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa3600060b0 tx=0x7fa360006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.267+0000 7fa381b97b80 1 -- 172.21.15.7:0/3679180525 shutdown_connections 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.267+0000 7fa381b97b80 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.267+0000 7fa381b97b80 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f28f20 0x55fcd9f29340 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.267+0000 7fa381b97b80 1 --2- 172.21.15.7:0/3679180525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.267+0000 7fa381b97b80 1 -- 172.21.15.7:0/3679180525 >> 172.21.15.7:0/3679180525 conn(0x55fcd9e9e740 msgr2=0x55fcd9f23950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.267+0000 7fa381b97b80 1 -- 172.21.15.7:0/3679180525 shutdown_connections 2024-01-11T18:11:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.268+0000 7fa381b97b80 1 -- 172.21.15.7:0/3679180525 wait complete. 2024-01-11T18:11:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.269+0000 7fa381b97b80 1 -- start start 2024-01-11T18:11:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.269+0000 7fa381b97b80 1 Processor -- start 2024-01-11T18:11:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f32500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f32a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f39a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9e7ccd0 con 0x55fcd9f28f20 2024-01-11T18:11:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9deb340 con 0x55fcd9f29c10 2024-01-11T18:11:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9deafe0 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.271+0000 7fa381b97b80 10 client.0 _ll_register_callbacks cb 0x55fcd9e1d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa367fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f39a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa367fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f39a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33664/0 (socket says 172.21.15.7:33664) 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa367fff700 1 -- 172.21.15.7:0/966933035 learned_addr learned my addr 172.21.15.7:0/966933035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa3677fe700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f32500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa366ffd700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f32a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa367fff700 1 -- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 msgr2=0x55fcd9f32a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa367fff700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f32a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.272+0000 7fa367fff700 1 -- 172.21.15.7:0/966933035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 msgr2=0x55fcd9f32500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.273+0000 7fa367fff700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f32500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.273+0000 7fa367fff700 1 -- 172.21.15.7:0/966933035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36806b8f0 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.273+0000 7fa3677fe700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f32500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.273+0000 7fa346ffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:11:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.273+0000 7fa367fff700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f39a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa360007fd0 tx=0x7fa360004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe48f81a58111e21 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.274+0000 7fa346ffd700 1 -- 172.21.15.7:0/966933035 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36001d070 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.274+0000 7fa346ffd700 1 -- 172.21.15.7:0/966933035 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa360018c00 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.274+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcd9f1bd60 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.275+0000 7fa346ffd700 1 -- 172.21.15.7:0/966933035 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa36000e070 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.276+0000 7fa381b97b80 10 client.26572 fetch_fsmap learned FSMap version 46 2024-01-11T18:11:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.276+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fcd9c9db00 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.277+0000 7fa346ffd700 1 -- 172.21.15.7:0/966933035 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcd9f1bd60 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.277+0000 7fa346ffd700 1 -- 172.21.15.7:0/966933035 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 46) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa360029070 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.278+0000 7fa381b97b80 10 client.26572 fetch_fsmap finished waiting for FSMap version 46 2024-01-11T18:11:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.278+0000 7fa381b97b80 10 client.26572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.278+0000 7fa381b97b80 20 client.26572 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.278+0000 7fa381b97b80 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f3e240 0x55fcd9f40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.278+0000 7fa381b97b80 4 client.26572 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:11:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.279+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcd9e75b60 con 0x55fcd9f3e240 2024-01-11T18:11:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.280+0000 7fa3677fe700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f3e240 0x55fcd9f40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:31 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:11:32.272 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:31 smithi038 ceph-mon[43582]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-01-11T18:11:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.293+0000 7fa346ffd700 10 client.26572 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.293+0000 7fa3677fe700 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f3e240 0x55fcd9f40660 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142d05dedf0525cf server_cookie=bba6dfec22d58d8a in_seq=0 out_seq=0 2024-01-11T18:11:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.297+0000 7fa346ffd700 1 -- 172.21.15.7:0/966933035 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcd9e75b60 con 0x55fcd9f3e240 2024-01-11T18:11:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.297+0000 7fa346ffd700 10 client.26572 handle_command_reply: tid=0 2024-01-11T18:11:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.297+0000 7fa381b97b80 1 client.26572 shutdown 2024-01-11T18:11:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 msgr2=0x55fcd9f39a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f39a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa360007fd0 tx=0x7fa360004ad0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 shutdown_connections 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f3e240 0x55fcd9f40660 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f39a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f32500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 --2- 172.21.15.7:0/966933035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f32a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 >> 172.21.15.7:0/966933035 conn(0x55fcd9e9e740 msgr2=0x55fcd9f22f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.298+0000 7fa346ffd700 0 client.26572 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.299+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 shutdown_connections 2024-01-11T18:11:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.299+0000 7fa381b97b80 1 -- 172.21.15.7:0/966933035 wait complete. 2024-01-11T18:11:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.299+0000 7fa381b97b80 20 client.26572 trim_cache size 0 max 16384 2024-01-11T18:11:32.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:31 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:11:32.313 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:31 smithi007 ceph-mon[33361]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-01-11T18:11:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.344+0000 7fa381b97b80 1 Processor -- start 2024-01-11T18:11:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.344+0000 7fa381b97b80 1 -- start start 2024-01-11T18:11:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f1b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f39a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f2d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa381b97b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9cf3640 con 0x55fcd9f28f20 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa381b97b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9f24cd0 con 0x55fcd9f29c10 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa381b97b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9f25690 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.345+0000 7fa366ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f39a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.346+0000 7fa367fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f2d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.346+0000 7fa367fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f2d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33672/0 (socket says 172.21.15.7:33672) 2024-01-11T18:11:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.346+0000 7fa367fff700 1 -- 172.21.15.7:0/1972602053 learned_addr learned my addr 172.21.15.7:0/1972602053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.347+0000 7fa367fff700 1 -- 172.21.15.7:0/1972602053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 msgr2=0x55fcd9f39a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.347+0000 7fa367fff700 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f39a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.347+0000 7fa367fff700 1 -- 172.21.15.7:0/1972602053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 msgr2=0x55fcd9f1b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.347+0000 7fa367fff700 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f1b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.347+0000 7fa367fff700 1 -- 172.21.15.7:0/1972602053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd9e3e1d0 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.348+0000 7fa367fff700 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f2d490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa3600092e0 tx=0x7fa360009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5f58d4d20883a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.351+0000 7fa34d7fa700 1 -- 172.21.15.7:0/1972602053 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36001d030 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.351+0000 7fa34d7fa700 1 -- 172.21.15.7:0/1972602053 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa360009c90 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.351+0000 7fa34d7fa700 1 -- 172.21.15.7:0/1972602053 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360016ea0 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 -- 172.21.15.7:0/1972602053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 msgr2=0x55fcd9f2d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f2d490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa3600092e0 tx=0x7fa360009490 comp rx=0 tx=0).stop 2024-01-11T18:11:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 -- 172.21.15.7:0/1972602053 shutdown_connections 2024-01-11T18:11:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f2d490 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f28f20 0x55fcd9f1b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 --2- 172.21.15.7:0/1972602053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f29c10 0x55fcd9f39a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.352+0000 7fa381b97b80 1 -- 172.21.15.7:0/1972602053 >> 172.21.15.7:0/1972602053 conn(0x55fcd9e9e740 msgr2=0x55fcd9f44fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.353+0000 7fa381b97b80 1 -- 172.21.15.7:0/1972602053 shutdown_connections 2024-01-11T18:11:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.353+0000 7fa381b97b80 1 -- 172.21.15.7:0/1972602053 wait complete. 2024-01-11T18:11:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.353+0000 7fa381b97b80 1 -- start start 2024-01-11T18:11:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.354+0000 7fa381b97b80 1 Processor -- start 2024-01-11T18:11:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f28f20 0x55fcd9f36c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f38150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9f38690 con 0x55fcd9f29c10 2024-01-11T18:11:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9f38800 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd9f389c0 con 0x55fcd9f28f20 2024-01-11T18:11:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.355+0000 7fa381b97b80 10 client.0 _ll_register_callbacks cb 0x55fcd9e1d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.356+0000 7fa3677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f28f20 0x55fcd9f36c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.356+0000 7fa3677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f28f20 0x55fcd9f36c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:33680/0 (socket says 172.21.15.7:33680) 2024-01-11T18:11:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.356+0000 7fa3677fe700 1 -- 172.21.15.7:0/2713551303 learned_addr learned my addr 172.21.15.7:0/2713551303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.357+0000 7fa367fff700 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f38150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa367fff700 1 -- 172.21.15.7:0/2713551303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f28f20 msgr2=0x55fcd9f36c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa367fff700 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f28f20 0x55fcd9f36c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa367fff700 1 -- 172.21.15.7:0/2713551303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 msgr2=0x55fcd9f37140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa367fff700 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa367fff700 1 -- 172.21.15.7:0/2713551303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcd9c9db00 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa367fff700 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f38150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa3600164c0 tx=0x7fa3600168e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def8f65d5a0c84bb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.358+0000 7fa346ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.359+0000 7fa346ffd700 1 -- 172.21.15.7:0/2713551303 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000e070 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.359+0000 7fa346ffd700 1 -- 172.21.15.7:0/2713551303 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa36001e070 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.359+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3580a3c50 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.359+0000 7fa346ffd700 1 -- 172.21.15.7:0/2713551303 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa36001d040 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.359+0000 7fa381b97b80 10 client.26540 fetch_fsmap learned FSMap version 46 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.359+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fcd9cf5c70 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa346ffd700 1 -- 172.21.15.7:0/2713551303 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3580a3c50 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa346ffd700 1 -- 172.21.15.7:0/2713551303 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa36003c050 con 0x55fcd9f2a5d0 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa381b97b80 10 client.26540 fetch_fsmap finished waiting for FSMap version 46 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa381b97b80 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa381b97b80 20 client.26540 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa381b97b80 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f9d020 0x55fcd9f9f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa381b97b80 4 client.26540 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:11:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.360+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fcd9e75880 con 0x55fcd9f9d020 2024-01-11T18:11:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.363+0000 7fa3677fe700 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f9d020 0x55fcd9f9f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.365+0000 7fa3677fe700 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f9d020 0x55fcd9f9f440 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd80894d0d6d287a server_cookie=19f56cc0179be49 in_seq=0 out_seq=0 2024-01-11T18:11:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.365+0000 7fa346ffd700 10 client.26540 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.497+0000 7fa346ffd700 1 -- 172.21.15.7:0/2713551303 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fcd9e75880 con 0x55fcd9f9d020 2024-01-11T18:11:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.497+0000 7fa346ffd700 10 client.26540 handle_command_reply: tid=0 2024-01-11T18:11:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa381b97b80 1 client.26540 shutdown 2024-01-11T18:11:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f2a5d0 msgr2=0x55fcd9f38150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa381b97b80 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f38150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa3600164c0 tx=0x7fa3600168e0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 shutdown_connections 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa381b97b80 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55fcd9f9d020 0x55fcd9f9f440 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa346ffd700 0 client.26540 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.498+0000 7fa381b97b80 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fcd9f28f20 0x55fcd9f36c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.499+0000 7fa381b97b80 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fcd9f2a5d0 0x55fcd9f38150 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.499+0000 7fa381b97b80 1 --2- 172.21.15.7:0/2713551303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fcd9f29c10 0x55fcd9f37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.499+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 >> 172.21.15.7:0/2713551303 conn(0x55fcd9e9e740 msgr2=0x55fcd9f2d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.499+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 shutdown_connections 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.499+0000 7fa381b97b80 1 -- 172.21.15.7:0/2713551303 wait complete. 2024-01-11T18:11:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.499+0000 7fa381b97b80 20 client.26540 trim_cache size 0 max 16384 2024-01-11T18:11:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:11:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:11:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:11:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.500+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3500a2020 msgr2=0x7fa3500a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.500+0000 7fa36ddff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3500a2020 0x7fa3500a44e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa3600060b0 tx=0x7fa360006040 comp rx=0 tx=0).stop 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.500+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 msgr2=0x7fa368199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.500+0000 7fa36ddff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa368199620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa358004730 tx=0x7fa358005ef0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 shutdown_connections 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa3500a2020 0x7fa3500a44e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3680ffe90 0x7fa368199620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681011b0 0x7fa3681a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 --2- 172.21.15.7:0/2858785528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3681007f0 0x7fa368199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 >> 172.21.15.7:0/2858785528 conn(0x7fa3680fba30 msgr2=0x7fa3680fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 shutdown_connections 2024-01-11T18:11:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:32.501+0000 7fa36ddff700 1 -- 172.21.15.7:0/2858785528 wait complete. 2024-01-11T18:11:33.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:33 smithi016 ceph-mon[43195]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 492 op/s 2024-01-11T18:11:33.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:33 smithi038 ceph-mon[43582]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 492 op/s 2024-01-11T18:11:33.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:33 smithi007 ceph-mon[33361]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 492 op/s 2024-01-11T18:11:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 785.0 s, 496.2 tps, lat 203.062 ms stddev 233.421 2024-01-11T18:11:36.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:35 smithi016 ceph-mon[43195]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 MiB/s wr, 409 op/s 2024-01-11T18:11:36.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:35 smithi038 ceph-mon[43582]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 MiB/s wr, 409 op/s 2024-01-11T18:11:36.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:35 smithi007 ceph-mon[33361]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.1 MiB/s wr, 409 op/s 2024-01-11T18:11:37.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:36 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:11:37.268 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:36 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:11:37.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:36 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:11:38.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:37 smithi016 ceph-mon[43195]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-01-11T18:11:38.267 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:37 smithi038 ceph-mon[43582]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-01-11T18:11:38.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:37 smithi007 ceph-mon[33361]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-01-11T18:11:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 790.0 s, 483.0 tps, lat 202.365 ms stddev 245.781 2024-01-11T18:11:39.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:39 smithi016 ceph-mon[43195]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-01-11T18:11:39.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:39 smithi038 ceph-mon[43582]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-01-11T18:11:39.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:39 smithi007 ceph-mon[33361]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-01-11T18:11:41.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:41 smithi016 ceph-mon[43195]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.1 MiB/s wr, 506 op/s 2024-01-11T18:11:41.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:41 smithi038 ceph-mon[43582]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.1 MiB/s wr, 506 op/s 2024-01-11T18:11:41.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:41 smithi007 ceph-mon[33361]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.1 MiB/s wr, 506 op/s 2024-01-11T18:11:43.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:43 smithi016 ceph-mon[43195]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.4 MiB/s wr, 526 op/s 2024-01-11T18:11:43.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:43 smithi038 ceph-mon[43582]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.4 MiB/s wr, 526 op/s 2024-01-11T18:11:43.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:43 smithi007 ceph-mon[33361]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.4 MiB/s wr, 526 op/s 2024-01-11T18:11:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 795.0 s, 514.0 tps, lat 192.028 ms stddev 236.032 2024-01-11T18:11:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 -- 172.21.15.7:0/4262024041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c0ffbb0 msgr2=0x7fa09c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/4262024041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c0ffbb0 0x7fa09c106db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084009ca0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 -- 172.21.15.7:0/4262024041 shutdown_connections 2024-01-11T18:11:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/4262024041 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/4262024041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c1072f0 0x7fa09c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/4262024041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c0ffbb0 0x7fa09c106db0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 -- 172.21.15.7:0/4262024041 >> 172.21.15.7:0/4262024041 conn(0x7fa09c0fad60 msgr2=0x7fa09c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 -- 172.21.15.7:0/4262024041 shutdown_connections 2024-01-11T18:11:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.750+0000 7fa0a1fff700 1 -- 172.21.15.7:0/4262024041 wait complete. 2024-01-11T18:11:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.751+0000 7fa0a1fff700 1 Processor -- start 2024-01-11T18:11:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 -- start start 2024-01-11T18:11:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0ffbb0 0x7fa09c100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c1072f0 0x7fa09c101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c101a20 con 0x7fa09c1072f0 2024-01-11T18:11:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c101b90 con 0x7fa09c0ff1e0 2024-01-11T18:11:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.752+0000 7fa0a1fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c1a14f0 con 0x7fa09c0ffbb0 2024-01-11T18:11:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c102700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38082/0 (socket says 172.21.15.7:38082) 2024-01-11T18:11:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09b7fe700 1 -- 172.21.15.7:0/1412915245 learned_addr learned my addr 172.21.15.7:0/1412915245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09bfff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c1072f0 0x7fa09c101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09affd700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0ffbb0 0x7fa09c100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09bfff700 1 -- 172.21.15.7:0/1412915245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0ffbb0 msgr2=0x7fa09c100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09bfff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0ffbb0 0x7fa09c100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09bfff700 1 -- 172.21.15.7:0/1412915245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 msgr2=0x7fa09c102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09bfff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.753+0000 7fa09bfff700 1 -- 172.21.15.7:0/1412915245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840095f0 con 0x7fa09c1072f0 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.754+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.754+0000 7fa09bfff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c1072f0 0x7fa09c101280 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa09400e940 tx=0x7fa09400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.755+0000 7fa098ff9700 1 -- 172.21.15.7:0/1412915245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094004220 con 0x7fa09c1072f0 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.755+0000 7fa098ff9700 1 -- 172.21.15.7:0/1412915245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa094004cf0 con 0x7fa09c1072f0 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.755+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c1a17d0 con 0x7fa09c1072f0 2024-01-11T18:11:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.755+0000 7fa098ff9700 1 -- 172.21.15.7:0/1412915245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094005650 con 0x7fa09c1072f0 2024-01-11T18:11:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.755+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c1a1ce0 con 0x7fa09c1072f0 2024-01-11T18:11:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.758+0000 7fa098ff9700 1 -- 172.21.15.7:0/1412915245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fa094004750 con 0x7fa09c1072f0 2024-01-11T18:11:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.760+0000 7fa098ff9700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0880a1f50 0x7fa0880a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.760+0000 7fa098ff9700 1 -- 172.21.15.7:0/1412915245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa094014070 con 0x7fa09c1072f0 2024-01-11T18:11:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.760+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0880a1f50 0x7fa0880a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.760+0000 7fa0b5ce6b80 1 Processor -- start 2024-01-11T18:11:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 -- start start 2024-01-11T18:11:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0880a1f50 0x7fa0880a4410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009c10 tx=0x7fa08c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b8ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014a79030 con 0x55f014b8bde0 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014adcac0 con 0x55f014b8b420 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014ada3e0 con 0x55f014b8a730 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa09affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa09affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38090/0 (socket says 172.21.15.7:38090) 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa09affd700 1 -- 172.21.15.7:0/3947231811 learned_addr learned my addr 172.21.15.7:0/3947231811 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.761+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b8ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09bfff700 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09affd700 1 -- 172.21.15.7:0/3947231811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 msgr2=0x55f014b8ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09affd700 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b8ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09affd700 1 -- 172.21.15.7:0/3947231811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 msgr2=0x55f014b92670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09affd700 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b92670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09affd700 1 -- 172.21.15.7:0/3947231811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f01494a350 con 0x55f014b8b420 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09bfff700 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b92670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa09affd700 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b8a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f183c5583e2eb6a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa091ffb700 1 -- 172.21.15.7:0/3947231811 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08401d070 con 0x55f014b8b420 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa091ffb700 1 -- 172.21.15.7:0/3947231811 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08401e040 con 0x55f014b8b420 2024-01-11T18:11:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.762+0000 7fa091ffb700 1 -- 172.21.15.7:0/3947231811 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0840049c0 con 0x55f014b8b420 2024-01-11T18:11:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.765+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/3947231811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 msgr2=0x55f014b8b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.765+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b8a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084005e80 comp rx=0 tx=0).stop 2024-01-11T18:11:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.766+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/3947231811 shutdown_connections 2024-01-11T18:11:45.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.766+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b8ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.766+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b92670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.766+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/3947231811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b8a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.766+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/3947231811 >> 172.21.15.7:0/3947231811 conn(0x55f014af7740 msgr2=0x55f014b7c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.767+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/3947231811 shutdown_connections 2024-01-11T18:11:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.767+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/3947231811 wait complete. 2024-01-11T18:11:45.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.768+0000 7fa0b5ce6b80 1 -- start start 2024-01-11T18:11:45.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.768+0000 7fa0b5ce6b80 1 Processor -- start 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b9b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014ad5cd0 con 0x55f014b8bde0 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014a44340 con 0x55f014b8b420 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014a43fe0 con 0x55f014b8a730 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.769+0000 7fa0b5ce6b80 10 client.0 _ll_register_callbacks cb 0x55f014a766c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54650/0 (socket says 172.21.15.7:54650) 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09b7fe700 1 -- 172.21.15.7:0/2664672269 learned_addr learned my addr 172.21.15.7:0/2664672269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09affd700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09bfff700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b9b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09b7fe700 1 -- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 msgr2=0x55f014b945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.770+0000 7fa09b7fe700 1 -- 172.21.15.7:0/2664672269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 msgr2=0x55f014b9b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b9b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa09b7fe700 1 -- 172.21.15.7:0/2664672269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c10a320 con 0x55f014b8a730 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa09bfff700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b9b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b940b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa08c015700 tx=0x7fa08c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22836c91ba8c7d42 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa07affd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa07affd700 1 -- 172.21.15.7:0/2664672269 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c017850 con 0x55f014b8a730 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa07affd700 1 -- 172.21.15.7:0/2664672269 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa08c0179b0 con 0x55f014b8a730 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.771+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f014b74d60 con 0x55f014b8a730 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa07affd700 1 -- 172.21.15.7:0/2664672269 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa08c00ee80 con 0x55f014b8a730 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 10 client.26590 fetch_fsmap learned FSMap version 47 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55f0148f6b00 con 0x55f014b8a730 2024-01-11T18:11:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa07affd700 1 -- 172.21.15.7:0/2664672269 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f014b74d60 con 0x55f014b8a730 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa07affd700 1 -- 172.21.15.7:0/2664672269 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa08c023c90 con 0x55f014b8a730 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 10 client.26590 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 10 client.26590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 20 client.26590 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014b9ffa0 0x55f014ba23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 4 client.26590 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:11:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.772+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f014aceb60 con 0x55f014b9ffa0 2024-01-11T18:11:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.773+0000 7fa09affd700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014b9ffa0 0x55f014ba23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.774+0000 7fa09affd700 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014b9ffa0 0x55f014ba23c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b554f49ee60c27 server_cookie=e9f80ac10f37a0eb in_seq=0 out_seq=0 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.774+0000 7fa07affd700 10 client.26590 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.774+0000 7fa07affd700 1 -- 172.21.15.7:0/2664672269 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f014aceb60 con 0x55f014b9ffa0 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.774+0000 7fa07affd700 10 client.26590 handle_command_reply: tid=0 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.774+0000 7fa0b5ce6b80 1 client.26590 shutdown 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.775+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 msgr2=0x55f014b940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.775+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b940b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa08c015700 tx=0x7fa08c004ad0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 shutdown_connections 2024-01-11T18:11:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014b9ffa0 0x55f014ba23c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b940b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8bde0 0x55f014b9b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/2664672269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa07affd700 0 client.26590 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 >> 172.21.15.7:0/2664672269 conn(0x55f014af7740 msgr2=0x55f014b7c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 shutdown_connections 2024-01-11T18:11:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/2664672269 wait complete. 2024-01-11T18:11:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.776+0000 7fa0b5ce6b80 20 client.26590 trim_cache size 0 max 16384 2024-01-11T18:11:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.800+0000 7fa0b5ce6b80 1 Processor -- start 2024-01-11T18:11:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.801+0000 7fa0b5ce6b80 1 -- start start 2024-01-11T18:11:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.802+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 0x55f014b9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.802+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.802+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8bde0 0x55f014b9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.802+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01494c640 con 0x55f014b8a730 2024-01-11T18:11:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.802+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014b79560 con 0x55f014b8b420 2024-01-11T18:11:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.802+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014ba7640 con 0x55f014b8bde0 2024-01-11T18:11:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.803+0000 7fa09bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8bde0 0x55f014b9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.803+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 0x55f014b9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.803+0000 7fa09affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.803+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 0x55f014b9bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52774/0 (socket says 172.21.15.7:52774) 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.803+0000 7fa09b7fe700 1 -- 172.21.15.7:0/156074819 learned_addr learned my addr 172.21.15.7:0/156074819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.804+0000 7fa09b7fe700 1 -- 172.21.15.7:0/156074819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8bde0 msgr2=0x55f014b9b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.804+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8bde0 0x55f014b9b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.804+0000 7fa09b7fe700 1 -- 172.21.15.7:0/156074819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 msgr2=0x55f014b8b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.804+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.804+0000 7fa09b7fe700 1 -- 172.21.15.7:0/156074819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f014a971d0 con 0x55f014b8a730 2024-01-11T18:11:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.804+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 0x55f014b9bce0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0098a0 tx=0x7fa08c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab16e9ce2e5a399 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.808+0000 7fa0917fa700 1 -- 172.21.15.7:0/156074819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00ee60 con 0x55f014b8a730 2024-01-11T18:11:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.808+0000 7fa0917fa700 1 -- 172.21.15.7:0/156074819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08c00f040 con 0x55f014b8a730 2024-01-11T18:11:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.808+0000 7fa0917fa700 1 -- 172.21.15.7:0/156074819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c017dc0 con 0x55f014b8a730 2024-01-11T18:11:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/156074819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 msgr2=0x55f014b9bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 0x55f014b9bce0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0098a0 tx=0x7fa08c0098d0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/156074819 shutdown_connections 2024-01-11T18:11:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8bde0 0x55f014b9b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8a730 0x55f014b9bce0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/156074819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8b420 0x55f014b8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.809+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/156074819 >> 172.21.15.7:0/156074819 conn(0x55f014af7740 msgr2=0x55f014b789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.810+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/156074819 shutdown_connections 2024-01-11T18:11:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.810+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/156074819 wait complete. 2024-01-11T18:11:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.810+0000 7fa0b5ce6b80 1 -- start start 2024-01-11T18:11:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.811+0000 7fa0b5ce6b80 1 Processor -- start 2024-01-11T18:11:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b98eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8b420 0x55f014b993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b98eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8bde0 0x55f014b9a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014b9a8f0 con 0x55f014b8b420 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014b9aa60 con 0x55f014b8bde0 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f014b9ac20 con 0x55f014b8a730 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.812+0000 7fa0b5ce6b80 10 client.0 _ll_register_callbacks cb 0x55f014a766c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.813+0000 7fa09b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b98eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:54672/0 (socket says 172.21.15.7:54672) 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.813+0000 7fa09b7fe700 1 -- 172.21.15.7:0/684085184 learned_addr learned my addr 172.21.15.7:0/684085184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.813+0000 7fa09affd700 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8b420 0x55f014b993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.814+0000 7fa09affd700 1 -- 172.21.15.7:0/684085184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 msgr2=0x55f014b98eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.814+0000 7fa09affd700 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b98eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.814+0000 7fa09affd700 1 -- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8bde0 msgr2=0x55f014b9a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.814+0000 7fa09affd700 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8bde0 0x55f014b9a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.814+0000 7fa09affd700 1 -- 172.21.15.7:0/684085184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0148f6b00 con 0x55f014b8b420 2024-01-11T18:11:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.817+0000 7fa09affd700 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8b420 0x55f014b993f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa08400b680 tx=0x7fa08400bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3584fc2c057c22ea server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.818+0000 7fa07affd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:11:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.818+0000 7fa07affd700 1 -- 172.21.15.7:0/684085184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08401d040 con 0x55f014b8b420 2024-01-11T18:11:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.818+0000 7fa07affd700 1 -- 172.21.15.7:0/684085184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa084005450 con 0x55f014b8b420 2024-01-11T18:11:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.818+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa08401e040 con 0x55f014b8b420 2024-01-11T18:11:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.818+0000 7fa07affd700 1 -- 172.21.15.7:0/684085184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa084004500 con 0x55f014b8b420 2024-01-11T18:11:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.819+0000 7fa0b5ce6b80 10 client.16311 fetch_fsmap learned FSMap version 47 2024-01-11T18:11:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.819+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55f01494ec70 con 0x55f014b8b420 2024-01-11T18:11:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.819+0000 7fa07affd700 1 -- 172.21.15.7:0/684085184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08401e040 con 0x55f014b8b420 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa07affd700 1 -- 172.21.15.7:0/684085184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa0840099c0 con 0x55f014b8b420 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa0b5ce6b80 10 client.16311 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa0b5ce6b80 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa0b5ce6b80 20 client.16311 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014bf6a80 0x55f014bf8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa0b5ce6b80 4 client.16311 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.820+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f014ace880 con 0x55f014bf6a80 2024-01-11T18:11:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.821+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014bf6a80 0x55f014bf8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.842+0000 7fa07affd700 10 client.16311 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.842+0000 7fa09b7fe700 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014bf6a80 0x55f014bf8e70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4681b7e2c2cd6c9 server_cookie=264e8468c0746a4c in_seq=0 out_seq=0 2024-01-11T18:11:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.969+0000 7fa07affd700 1 -- 172.21.15.7:0/684085184 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f014ace880 con 0x55f014bf6a80 2024-01-11T18:11:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.969+0000 7fa07affd700 10 client.16311 handle_command_reply: tid=0 2024-01-11T18:11:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.969+0000 7fa0b5ce6b80 1 client.16311 shutdown 2024-01-11T18:11:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8b420 msgr2=0x55f014b993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8b420 0x55f014b993f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa08400b680 tx=0x7fa08400bc40 comp rx=0 tx=0).stop 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 shutdown_connections 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55f014bf6a80 0x55f014bf8e70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa07affd700 0 client.16311 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f014b8a730 0x55f014b98eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f014b8bde0 0x55f014b9a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 --2- 172.21.15.7:0/684085184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f014b8b420 0x55f014b993f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.970+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 >> 172.21.15.7:0/684085184 conn(0x55f014af7740 msgr2=0x55f014b92380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.971+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 shutdown_connections 2024-01-11T18:11:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.971+0000 7fa0b5ce6b80 1 -- 172.21.15.7:0/684085184 wait complete. 2024-01-11T18:11:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.971+0000 7fa0b5ce6b80 20 client.16311 trim_cache size 0 max 16384 2024-01-11T18:11:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:11:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:11:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:11:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:11:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0880a1f50 msgr2=0x7fa0880a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0880a1f50 0x7fa0880a4410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009c10 tx=0x7fa08c00b040 comp rx=0 tx=0).stop 2024-01-11T18:11:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c1072f0 msgr2=0x7fa09c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c1072f0 0x7fa09c101280 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa09400e940 tx=0x7fa09400ec50 comp rx=0 tx=0).stop 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 shutdown_connections 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fa0880a1f50 0x7fa0880a4410 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009c10 tx=0x7fa08c00b040 comp rx=0 tx=0).stop 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa09c1072f0 0x7fa09c101280 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09c0ff1e0 0x7fa09c102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 --2- 172.21.15.7:0/1412915245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa09c0ffbb0 0x7fa09c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.972+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 >> 172.21.15.7:0/1412915245 conn(0x7fa09c0fad60 msgr2=0x7fa09c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.973+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 shutdown_connections 2024-01-11T18:11:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:45.973+0000 7fa0a1fff700 1 -- 172.21.15.7:0/1412915245 wait complete. 2024-01-11T18:11:46.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:45 smithi016 ceph-mon[43195]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.2 MiB/s wr, 427 op/s 2024-01-11T18:11:46.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:45 smithi038 ceph-mon[43582]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.2 MiB/s wr, 427 op/s 2024-01-11T18:11:46.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:45 smithi007 ceph-mon[33361]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.2 MiB/s wr, 427 op/s 2024-01-11T18:11:47.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:47 smithi016 ceph-mon[43195]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.8 MiB/s wr, 574 op/s 2024-01-11T18:11:47.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:47 smithi038 ceph-mon[43582]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.8 MiB/s wr, 574 op/s 2024-01-11T18:11:47.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:47 smithi007 ceph-mon[33361]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.8 MiB/s wr, 574 op/s 2024-01-11T18:11:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 800.0 s, 495.6 tps, lat 208.072 ms stddev 284.100 2024-01-11T18:11:49.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:49 smithi016 ceph-mon[43195]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-01-11T18:11:49.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:49 smithi038 ceph-mon[43582]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-01-11T18:11:49.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:49 smithi007 ceph-mon[33361]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-01-11T18:11:51.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:51 smithi038 ceph-mon[43582]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-01-11T18:11:51.526 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:51 smithi016 ceph-mon[43195]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-01-11T18:11:51.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:51 smithi007 ceph-mon[33361]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-01-11T18:11:52.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:52 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:11:52.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:52 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:11:52.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:52 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:11:53.985 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:53.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:53.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:53.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:53 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:53.986 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:53 smithi016 ceph-mon[43195]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.1 MiB/s wr, 510 op/s 2024-01-11T18:11:54.016 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.016 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.016 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.016 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:53 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.016 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:53 smithi038 ceph-mon[43582]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.1 MiB/s wr, 510 op/s 2024-01-11T18:11:54.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:53 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:54.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:53 smithi007 ceph-mon[33361]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.1 MiB/s wr, 510 op/s 2024-01-11T18:11:54.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 805.0 s, 499.4 tps, lat 199.640 ms stddev 215.574 2024-01-11T18:11:55.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:11:55.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:11:55.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:55.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:11:55.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:11:55.061 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:55.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:11:55.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:11:55.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:11:56.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:55 smithi016 ceph-mon[43195]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.3 MiB/s wr, 421 op/s 2024-01-11T18:11:56.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:55 smithi038 ceph-mon[43582]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.3 MiB/s wr, 421 op/s 2024-01-11T18:11:56.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:55 smithi007 ceph-mon[33361]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.3 MiB/s wr, 421 op/s 2024-01-11T18:11:57.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:57 smithi016 ceph-mon[43195]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-01-11T18:11:57.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:57 smithi038 ceph-mon[43582]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-01-11T18:11:57.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:57 smithi007 ceph-mon[33361]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-01-11T18:11:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 810.0 s, 434.2 tps, lat 221.365 ms stddev 252.895 2024-01-11T18:11:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 -- 172.21.15.7:0/1250248977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f840999c0 msgr2=0x7f6f84099e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 --2- 172.21.15.7:0/1250248977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f840999c0 0x7f6f84099e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0679f0 tx=0x7f6f8c073b20 comp rx=0 tx=0).stop 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 -- 172.21.15.7:0/1250248977 shutdown_connections 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 --2- 172.21.15.7:0/1250248977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f840999c0 0x7f6f84099e40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 --2- 172.21.15.7:0/1250248977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 0x7f6f84096ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 --2- 172.21.15.7:0/1250248977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84096ff0 0x7f6f84099480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 -- 172.21.15.7:0/1250248977 >> 172.21.15.7:0/1250248977 conn(0x7f6f84090260 msgr2=0x7f6f840926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 -- 172.21.15.7:0/1250248977 shutdown_connections 2024-01-11T18:11:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.228+0000 7f6f9239d700 1 -- 172.21.15.7:0/1250248977 wait complete. 2024-01-11T18:11:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.230+0000 7f6f9239d700 1 Processor -- start 2024-01-11T18:11:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.230+0000 7f6f9239d700 1 -- start start 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9239d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 0x7f6f84129a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9239d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84096ff0 0x7f6f84129fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9239d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f840999c0 0x7f6f84131010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9239d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841315e0 con 0x7f6f84094680 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9239d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84142730 con 0x7f6f840999c0 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9239d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841428c0 con 0x7f6f84096ff0 2024-01-11T18:11:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f90b9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84096ff0 0x7f6f84129fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f90b9a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84096ff0 0x7f6f84129fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44120/0 (socket says 172.21.15.7:44120) 2024-01-11T18:11:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f90b9a700 1 -- 172.21.15.7:0/2074912836 learned_addr learned my addr 172.21.15.7:0/2074912836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.231+0000 7f6f9139b700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 0x7f6f84129a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.232+0000 7f6f9139b700 1 -- 172.21.15.7:0/2074912836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84096ff0 msgr2=0x7f6f84129fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.232+0000 7f6f9139b700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84096ff0 0x7f6f84129fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.232+0000 7f6f9139b700 1 -- 172.21.15.7:0/2074912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f840999c0 msgr2=0x7f6f84131010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.233+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f840999c0 0x7f6f84131010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.233+0000 7f6f9139b700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f840999c0 0x7f6f84131010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.233+0000 7f6f9139b700 1 -- 172.21.15.7:0/2074912836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c067b50 con 0x7f6f84094680 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.233+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f840999c0 0x7f6f84131010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.233+0000 7f6f9139b700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 0x7f6f84129a60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6f88006fd0 tx=0x7f6f8800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.234+0000 7f6f827fc700 1 -- 172.21.15.7:0/2074912836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88011830 con 0x7f6f84094680 2024-01-11T18:11:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.234+0000 7f6f827fc700 1 -- 172.21.15.7:0/2074912836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f88011e50 con 0x7f6f84094680 2024-01-11T18:11:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.234+0000 7f6f827fc700 1 -- 172.21.15.7:0/2074912836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88010e80 con 0x7f6f84094680 2024-01-11T18:11:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.234+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f84142ba0 con 0x7f6f84094680 2024-01-11T18:11:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.235+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f841430b0 con 0x7f6f84094680 2024-01-11T18:11:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.240+0000 7f6f827fc700 1 -- 172.21.15.7:0/2074912836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6f88010380 con 0x7f6f84094680 2024-01-11T18:11:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.241+0000 7f6fa731cb80 1 Processor -- start 2024-01-11T18:11:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.243+0000 7f6f827fc700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f780a2060 0x7f6f780a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.244+0000 7f6f827fc700 1 -- 172.21.15.7:0/2074912836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6f880cd760 con 0x7f6f84094680 2024-01-11T18:11:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.245+0000 7f6fa731cb80 1 -- start start 2024-01-11T18:11:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260d9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9eb0 0x5654260da330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x565425fd0030 con 0x5654260da870 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565426033ac0 con 0x5654260d9eb0 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654260313e0 con 0x5654260d9120 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.246+0000 7f6f90b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9eb0 0x5654260da330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.247+0000 7f6f9139b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260d9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.247+0000 7f6f90b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9eb0 0x5654260da330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45520/0 (socket says 172.21.15.7:45520) 2024-01-11T18:11:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.247+0000 7f6f90b9a700 1 -- 172.21.15.7:0/671143480 learned_addr learned my addr 172.21.15.7:0/671143480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.248+0000 7f6f9139b700 1 -- 172.21.15.7:0/671143480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9eb0 msgr2=0x5654260da330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.248+0000 7f6f9139b700 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9eb0 0x5654260da330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.248+0000 7f6f9139b700 1 -- 172.21.15.7:0/671143480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 msgr2=0x5654260e1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.248+0000 7f6f9139b700 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.248+0000 7f6f9139b700 1 -- 172.21.15.7:0/671143480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565425ea1350 con 0x5654260d9120 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.248+0000 7f6f9139b700 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260d9540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f880caf30 tx=0x7f6f880108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe282256012fc5b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.249+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f780a2060 0x7f6f780a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.249+0000 7f6f777fe700 1 -- 172.21.15.7:0/671143480 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8809a190 con 0x5654260d9120 2024-01-11T18:11:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.249+0000 7f6f777fe700 1 -- 172.21.15.7:0/671143480 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8809e2e0 con 0x5654260d9120 2024-01-11T18:11:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.249+0000 7f6f777fe700 1 -- 172.21.15.7:0/671143480 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880a2090 con 0x5654260d9120 2024-01-11T18:11:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.251+0000 7f6fa731cb80 1 -- 172.21.15.7:0/671143480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 msgr2=0x5654260d9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.251+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260d9540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f880caf30 tx=0x7f6f880108a0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.252+0000 7f6fa731cb80 1 -- 172.21.15.7:0/671143480 shutdown_connections 2024-01-11T18:11:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.252+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260d9540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.252+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.252+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/671143480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9eb0 0x5654260da330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.252+0000 7f6fa731cb80 1 -- 172.21.15.7:0/671143480 >> 172.21.15.7:0/671143480 conn(0x56542604e740 msgr2=0x5654260d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.254+0000 7f6fa731cb80 1 -- 172.21.15.7:0/671143480 shutdown_connections 2024-01-11T18:11:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.254+0000 7f6fa731cb80 1 -- 172.21.15.7:0/671143480 wait complete. 2024-01-11T18:11:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.255+0000 7f6fa731cb80 1 -- start start 2024-01-11T18:11:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.255+0000 7f6fa731cb80 1 Processor -- start 2024-01-11T18:11:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542602ccd0 con 0x5654260d9eb0 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565425f9b340 con 0x5654260da870 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x565425f9afe0 con 0x5654260d9120 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.257+0000 7f6fa731cb80 10 client.0 _ll_register_callbacks cb 0x565425fcd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.256+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f780a2060 0x7f6f780a4520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c04fd50 tx=0x7f6f8c067800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.257+0000 7f6f90b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.257+0000 7f6f91b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.257+0000 7f6f91b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260ea0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45528/0 (socket says 172.21.15.7:45528) 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.257+0000 7f6f91b9c700 1 -- 172.21.15.7:0/3769437001 learned_addr learned my addr 172.21.15.7:0/3769437001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.257+0000 7f6f9139b700 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.258+0000 7f6f90b9a700 1 -- 172.21.15.7:0/3769437001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 msgr2=0x5654260e2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.258+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260e2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.258+0000 7f6f90b9a700 1 -- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 msgr2=0x5654260ea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.258+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260ea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.258+0000 7f6f90b9a700 1 -- 172.21.15.7:0/3769437001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f841430b0 con 0x5654260d9eb0 2024-01-11T18:11:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.259+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260e30b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c01d010 tx=0x7f6f7c0227a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9075667fd59a12b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.259+0000 7f6f74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.260+0000 7f6f74ff9700 1 -- 172.21.15.7:0/3769437001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c004260 con 0x5654260d9eb0 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.260+0000 7f6f74ff9700 1 -- 172.21.15.7:0/3769437001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6f7c0043c0 con 0x5654260d9eb0 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.260+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654260cbfb0 con 0x5654260d9eb0 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.261+0000 7f6f74ff9700 1 -- 172.21.15.7:0/3769437001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f7c004580 con 0x5654260d9eb0 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.261+0000 7f6fa731cb80 10 client.16317 fetch_fsmap learned FSMap version 47 2024-01-11T18:11:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.261+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x565425e4db00 con 0x5654260d9eb0 2024-01-11T18:11:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6f74ff9700 1 -- 172.21.15.7:0/3769437001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654260cbfb0 con 0x5654260d9eb0 2024-01-11T18:11:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6f74ff9700 1 -- 172.21.15.7:0/3769437001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6f7c02b9b0 con 0x5654260d9eb0 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6fa731cb80 10 client.16317 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6fa731cb80 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6fa731cb80 20 client.16317 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5654260eeab0 0x5654260f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6fa731cb80 4 client.16317 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.262+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565426025b60 con 0x5654260eeab0 2024-01-11T18:11:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.263+0000 7f6f9139b700 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5654260eeab0 0x5654260f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.270+0000 7f6f9139b700 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5654260eeab0 0x5654260f0ed0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68de7e502b2935a server_cookie=cde17ab1e4f9f120 in_seq=0 out_seq=0 2024-01-11T18:11:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.270+0000 7f6f74ff9700 10 client.16317 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.272+0000 7f6f74ff9700 1 -- 172.21.15.7:0/3769437001 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565426025b60 con 0x5654260eeab0 2024-01-11T18:11:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.272+0000 7f6f74ff9700 10 client.16317 handle_command_reply: tid=0 2024-01-11T18:11:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.272+0000 7f6fa731cb80 1 client.16317 shutdown 2024-01-11T18:11:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.272+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 msgr2=0x5654260e30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260e30b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c01d010 tx=0x7f6f7c0227a0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 shutdown_connections 2024-01-11T18:11:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x5654260eeab0 0x5654260f0ed0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260ea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6f74ff9700 0 client.16317 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/3769437001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260e30b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 >> 172.21.15.7:0/3769437001 conn(0x56542604e740 msgr2=0x5654260d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.273+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 shutdown_connections 2024-01-11T18:11:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.274+0000 7f6fa731cb80 1 -- 172.21.15.7:0/3769437001 wait complete. 2024-01-11T18:11:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.274+0000 7f6fa731cb80 20 client.16317 trim_cache size 0 max 16384 2024-01-11T18:11:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.322+0000 7f6fa731cb80 1 Processor -- start 2024-01-11T18:11:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.322+0000 7f6fa731cb80 1 -- start start 2024-01-11T18:11:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.322+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260da290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.322+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.322+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260f5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x565425ea3640 con 0x5654260d9eb0 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654260e26e0 con 0x5654260da870 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654260d07e0 con 0x5654260d9120 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6f91b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6f91b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45532/0 (socket says 172.21.15.7:45532) 2024-01-11T18:11:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6f91b9c700 1 -- 172.21.15.7:0/741413425 learned_addr learned my addr 172.21.15.7:0/741413425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260ea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.323+0000 7f6f91b9c700 1 -- 172.21.15.7:0/741413425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 msgr2=0x5654260da290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f9139b700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260da290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f91b9c700 1 -- 172.21.15.7:0/741413425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 msgr2=0x5654260ea0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260ea0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f91b9c700 1 -- 172.21.15.7:0/741413425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565425fee1d0 con 0x5654260da870 2024-01-11T18:11:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260ea0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:11:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.324+0000 7f6f9139b700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260da290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:11:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.325+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260f5830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c04fa10 tx=0x7f6f8c072000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2a312aff5b7317c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.325+0000 7f6f76ffd700 1 -- 172.21.15.7:0/741413425 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c08e070 con 0x5654260da870 2024-01-11T18:11:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.326+0000 7f6f76ffd700 1 -- 172.21.15.7:0/741413425 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8c08f040 con 0x5654260da870 2024-01-11T18:11:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.326+0000 7f6f76ffd700 1 -- 172.21.15.7:0/741413425 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c08e070 con 0x5654260da870 2024-01-11T18:11:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 -- 172.21.15.7:0/741413425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 msgr2=0x5654260f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260f5830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c04fa10 tx=0x7f6f8c072000 comp rx=0 tx=0).stop 2024-01-11T18:11:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 -- 172.21.15.7:0/741413425 shutdown_connections 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9120 0x5654260da290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260da870 0x5654260f5830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/741413425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260d9eb0 0x5654260ea0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.327+0000 7f6fa731cb80 1 -- 172.21.15.7:0/741413425 >> 172.21.15.7:0/741413425 conn(0x56542604e740 msgr2=0x5654260e0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.328+0000 7f6fa731cb80 1 -- 172.21.15.7:0/741413425 shutdown_connections 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.328+0000 7f6fa731cb80 1 -- 172.21.15.7:0/741413425 wait complete. 2024-01-11T18:11:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.328+0000 7f6fa731cb80 1 -- start start 2024-01-11T18:11:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.329+0000 7f6fa731cb80 1 Processor -- start 2024-01-11T18:11:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 0x5654260e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9eb0 0x5654260e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654260e9760 con 0x5654260da870 2024-01-11T18:11:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654260e98d0 con 0x5654260d9120 2024-01-11T18:11:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654260e9a90 con 0x5654260d9eb0 2024-01-11T18:11:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.331+0000 7f6fa731cb80 10 client.0 _ll_register_callbacks cb 0x565425fcd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:11:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.332+0000 7f6f9139b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 0x5654260e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.332+0000 7f6f9139b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 0x5654260e7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45534/0 (socket says 172.21.15.7:45534) 2024-01-11T18:11:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.332+0000 7f6f9139b700 1 -- 172.21.15.7:0/260366902 learned_addr learned my addr 172.21.15.7:0/260366902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:11:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.332+0000 7f6f91b9c700 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.333+0000 7f6f9139b700 1 -- 172.21.15.7:0/260366902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9eb0 msgr2=0x5654260e8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.333+0000 7f6f9139b700 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9eb0 0x5654260e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.333+0000 7f6f9139b700 1 -- 172.21.15.7:0/260366902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 msgr2=0x5654260e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:11:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.333+0000 7f6f9139b700 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.333+0000 7f6f9139b700 1 -- 172.21.15.7:0/260366902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8409a970 con 0x5654260d9120 2024-01-11T18:11:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.334+0000 7f6f9139b700 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 0x5654260e7cd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6f88096640 tx=0x7f6f88099bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150ff55e412ca013 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:11:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.334+0000 7f6f74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:11:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.336+0000 7f6f74ff9700 1 -- 172.21.15.7:0/260366902 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880b8040 con 0x5654260d9120 2024-01-11T18:11:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.336+0000 7f6f74ff9700 1 -- 172.21.15.7:0/260366902 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6f88099680 con 0x5654260d9120 2024-01-11T18:11:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.336+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f8809e2e0 con 0x5654260d9120 2024-01-11T18:11:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.337+0000 7f6f74ff9700 1 -- 172.21.15.7:0/260366902 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f8809a210 con 0x5654260d9120 2024-01-11T18:11:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.338+0000 7f6fa731cb80 10 client.26567 fetch_fsmap learned FSMap version 47 2024-01-11T18:11:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.338+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x565425ea5c70 con 0x5654260d9120 2024-01-11T18:11:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.338+0000 7f6f74ff9700 1 -- 172.21.15.7:0/260366902 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8809e2e0 con 0x5654260d9120 2024-01-11T18:11:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.338+0000 7f6f74ff9700 1 -- 172.21.15.7:0/260366902 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6f88018ba0 con 0x5654260d9120 2024-01-11T18:11:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.339+0000 7f6fa731cb80 10 client.26567 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:11:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.339+0000 7f6fa731cb80 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:11:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.339+0000 7f6fa731cb80 20 client.26567 populate_metadata read hostname 'smithi007' 2024-01-11T18:11:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.339+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565426145a50 0x565426147e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:11:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.339+0000 7f6fa731cb80 4 client.26567 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:11:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.339+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565426025880 con 0x565426145a50 2024-01-11T18:11:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.341+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565426145a50 0x565426147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:11:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.347+0000 7f6f90b9a700 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565426145a50 0x565426147e70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1316cc202f37906 server_cookie=36f5838985481deb in_seq=0 out_seq=0 2024-01-11T18:11:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.347+0000 7f6f74ff9700 10 client.26567 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.473+0000 7f6f74ff9700 1 -- 172.21.15.7:0/260366902 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565426025880 con 0x565426145a50 2024-01-11T18:11:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.473+0000 7f6f74ff9700 10 client.26567 handle_command_reply: tid=0 2024-01-11T18:11:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.473+0000 7f6fa731cb80 1 client.26567 shutdown 2024-01-11T18:11:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 msgr2=0x5654260e7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 0x5654260e7cd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6f88096640 tx=0x7f6f88099bf0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 shutdown_connections 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x565426145a50 0x565426147e70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5654260da870 0x5654260e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654260d9120 0x5654260e7cd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 --2- 172.21.15.7:0/260366902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654260d9eb0 0x5654260e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 >> 172.21.15.7:0/260366902 conn(0x56542604e740 msgr2=0x5654260dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.474+0000 7f6f74ff9700 0 client.26567 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.475+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 shutdown_connections 2024-01-11T18:11:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.475+0000 7f6fa731cb80 1 -- 172.21.15.7:0/260366902 wait complete. 2024-01-11T18:11:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.475+0000 7f6fa731cb80 20 client.26567 trim_cache size 0 max 16384 2024-01-11T18:11:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:11:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:11:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:11:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:11:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.476+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f780a2060 msgr2=0x7f6f780a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.476+0000 7f6f9239d700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f780a2060 0x7f6f780a4520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c04fd50 tx=0x7f6f8c067800 comp rx=0 tx=0).stop 2024-01-11T18:11:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.476+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 msgr2=0x7f6f84129a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:11:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.476+0000 7f6f9239d700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 0x7f6f84129a60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6f88006fd0 tx=0x7f6f8800b7e0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 shutdown_connections 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f780a2060 0x7f6f780a4520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f840999c0 0x7f6f84131010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84094680 0x7f6f84129a60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 --2- 172.21.15.7:0/2074912836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6f84096ff0 0x7f6f84129fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 >> 172.21.15.7:0/2074912836 conn(0x7f6f84090260 msgr2=0x7f6f84092650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 shutdown_connections 2024-01-11T18:11:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:11:59.477+0000 7f6f9239d700 1 -- 172.21.15.7:0/2074912836 wait complete. 2024-01-11T18:11:59.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:11:59 smithi016 ceph-mon[43195]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-01-11T18:11:59.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:11:59 smithi038 ceph-mon[43582]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-01-11T18:11:59.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:11:59 smithi007 ceph-mon[33361]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-01-11T18:12:01.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:01 smithi016 ceph-mon[43195]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-01-11T18:12:01.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:01 smithi038 ceph-mon[43582]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-01-11T18:12:01.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:01 smithi007 ceph-mon[33361]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-01-11T18:12:03.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:03 smithi016 ceph-mon[43195]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-01-11T18:12:03.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:03 smithi038 ceph-mon[43582]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-01-11T18:12:03.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:03 smithi007 ceph-mon[33361]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-01-11T18:12:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 815.0 s, 454.8 tps, lat 230.426 ms stddev 266.957 2024-01-11T18:12:06.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:05 smithi016 ceph-mon[43195]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-01-11T18:12:06.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:05 smithi038 ceph-mon[43582]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-01-11T18:12:06.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:05 smithi007 ceph-mon[33361]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-01-11T18:12:07.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:07 smithi016 ceph-mon[43195]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.9 MiB/s wr, 511 op/s 2024-01-11T18:12:07.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:07 smithi038 ceph-mon[43582]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.9 MiB/s wr, 511 op/s 2024-01-11T18:12:07.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:07 smithi007 ceph-mon[33361]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.9 MiB/s wr, 511 op/s 2024-01-11T18:12:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 820.0 s, 444.2 tps, lat 217.968 ms stddev 259.590 2024-01-11T18:12:09.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:09 smithi016 ceph-mon[43195]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-01-11T18:12:09.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:09 smithi038 ceph-mon[43582]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-01-11T18:12:09.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:09 smithi007 ceph-mon[33361]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-01-11T18:12:11.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:11 smithi016 ceph-mon[43195]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.2 MiB/s wr, 457 op/s 2024-01-11T18:12:11.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:11 smithi038 ceph-mon[43582]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.2 MiB/s wr, 457 op/s 2024-01-11T18:12:11.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:11 smithi007 ceph-mon[33361]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.2 MiB/s wr, 457 op/s 2024-01-11T18:12:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 -- 172.21.15.7:0/4021900243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40501015b0 msgr2=0x7f40501019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 --2- 172.21.15.7:0/4021900243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40501015b0 0x7f40501019d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4044009920 tx=0x7f4044009c30 comp rx=0 tx=0).stop 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 -- 172.21.15.7:0/4021900243 shutdown_connections 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 --2- 172.21.15.7:0/4021900243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4050103140 0x7f40501099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 --2- 172.21.15.7:0/4021900243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40501015b0 0x7f40501019d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 --2- 172.21.15.7:0/4021900243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501027a0 0x7f4050102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.743+0000 7f40561ff700 1 -- 172.21.15.7:0/4021900243 >> 172.21.15.7:0/4021900243 conn(0x7f40500fcb00 msgr2=0x7f40500fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.744+0000 7f40561ff700 1 -- 172.21.15.7:0/4021900243 shutdown_connections 2024-01-11T18:12:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.744+0000 7f40561ff700 1 -- 172.21.15.7:0/4021900243 wait complete. 2024-01-11T18:12:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.745+0000 7f40561ff700 1 Processor -- start 2024-01-11T18:12:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.747+0000 7f40561ff700 1 -- start start 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f40561ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 0x7f4050197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f40561ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40501027a0 0x7f4050197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f40561ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4050103140 0x7f405019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f40561ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019f130 con 0x7f4050103140 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f40561ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019f270 con 0x7f40501015b0 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f40561ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019f400 con 0x7f40501027a0 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f404f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 0x7f4050197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f404f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 0x7f4050197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48056/0 (socket says 172.21.15.7:48056) 2024-01-11T18:12:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.749+0000 7f404f7fe700 1 -- 172.21.15.7:0/1940154523 learned_addr learned my addr 172.21.15.7:0/1940154523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.750+0000 7f404effd700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40501027a0 0x7f4050197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404effd700 1 -- 172.21.15.7:0/1940154523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 msgr2=0x7f4050197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404effd700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 0x7f4050197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404effd700 1 -- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4050103140 msgr2=0x7f405019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404effd700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4050103140 0x7f405019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404effd700 1 -- 172.21.15.7:0/1940154523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4044009520 con 0x7f40501027a0 2024-01-11T18:12:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404f7fe700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 0x7f4050197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:12:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404effd700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40501027a0 0x7f4050197b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f404000e9c0 tx=0x7f404000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404cff9700 1 -- 172.21.15.7:0/1940154523 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000c9c0 con 0x7f40501027a0 2024-01-11T18:12:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.751+0000 7f404cff9700 1 -- 172.21.15.7:0/1940154523 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4040010430 con 0x7f40501027a0 2024-01-11T18:12:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.752+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4050073960 con 0x7f40501027a0 2024-01-11T18:12:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.752+0000 7f404cff9700 1 -- 172.21.15.7:0/1940154523 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040018680 con 0x7f40501027a0 2024-01-11T18:12:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.752+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4050073ec0 con 0x7f40501027a0 2024-01-11T18:12:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.761+0000 7f404cff9700 1 -- 172.21.15.7:0/1940154523 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4040010860 con 0x7f40501027a0 2024-01-11T18:12:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.763+0000 7f4069ef4b80 1 Processor -- start 2024-01-11T18:12:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.765+0000 7f4069ef4b80 1 -- start start 2024-01-11T18:12:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.765+0000 7f404cff9700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40380a1f50 0x7f40380a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f404cff9700 1 -- 172.21.15.7:0/1940154523 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4040014070 con 0x7f40501027a0 2024-01-11T18:12:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f404ffff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40380a1f50 0x7f40380a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb054f20 0x55bdfb055340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb056090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb0565d0 0x55bdfb05ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfaf4c030 con 0x55bdfb0565d0 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfafafac0 con 0x55bdfb054f20 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.766+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfafad3e0 con 0x55bdfb055c10 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.767+0000 7f404effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb056090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.767+0000 7f404effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb056090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:41332/0 (socket says 172.21.15.7:41332) 2024-01-11T18:12:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.767+0000 7f404effd700 1 -- 172.21.15.7:0/942469491 learned_addr learned my addr 172.21.15.7:0/942469491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.768+0000 7f404f7fe700 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb054f20 0x55bdfb055340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.768+0000 7f404effd700 1 -- 172.21.15.7:0/942469491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb054f20 msgr2=0x55bdfb055340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.768+0000 7f404effd700 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb054f20 0x55bdfb055340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.768+0000 7f404effd700 1 -- 172.21.15.7:0/942469491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb0565d0 msgr2=0x55bdfb05ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.768+0000 7f404effd700 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb0565d0 0x55bdfb05ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.768+0000 7f404effd700 1 -- 172.21.15.7:0/942469491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfae1d350 con 0x55bdfb055c10 2024-01-11T18:12:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.769+0000 7f404effd700 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb056090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f404009a650 tx=0x7f404009a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=485f114e91577d06 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.769+0000 7f4035ffb700 1 -- 172.21.15.7:0/942469491 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400ba0b0 con 0x55bdfb055c10 2024-01-11T18:12:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.769+0000 7f4035ffb700 1 -- 172.21.15.7:0/942469491 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404009adc0 con 0x55bdfb055c10 2024-01-11T18:12:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.769+0000 7f4035ffb700 1 -- 172.21.15.7:0/942469491 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040099bb0 con 0x55bdfb055c10 2024-01-11T18:12:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 -- 172.21.15.7:0/942469491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 msgr2=0x55bdfb056090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f404ffff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40380a1f50 0x7f40380a4410 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4048010490 tx=0x7f4048009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb056090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f404009a650 tx=0x7f404009a250 comp rx=0 tx=0).stop 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 -- 172.21.15.7:0/942469491 shutdown_connections 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb0565d0 0x55bdfb05ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb054f20 0x55bdfb055340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/942469491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb056090 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 -- 172.21.15.7:0/942469491 >> 172.21.15.7:0/942469491 conn(0x55bdfafca740 msgr2=0x55bdfb04f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 -- 172.21.15.7:0/942469491 shutdown_connections 2024-01-11T18:12:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.771+0000 7f4069ef4b80 1 -- 172.21.15.7:0/942469491 wait complete. 2024-01-11T18:12:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.772+0000 7f4069ef4b80 1 -- start start 2024-01-11T18:12:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.772+0000 7f4069ef4b80 1 Processor -- start 2024-01-11T18:12:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.773+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.773+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb05ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.773+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb065f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f404effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb05ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f404effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb05ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48078/0 (socket says 172.21.15.7:48078) 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f404effd700 1 -- 172.21.15.7:0/1777191884 learned_addr learned my addr 172.21.15.7:0/1777191884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfafa8cd0 con 0x55bdfb054f20 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfaf17340 con 0x55bdfb055c10 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfaf16fe0 con 0x55bdfb0565d0 2024-01-11T18:12:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.775+0000 7f4069ef4b80 10 client.0 _ll_register_callbacks cb 0x55bdfaf496c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f404effd700 1 -- 172.21.15.7:0/1777191884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 msgr2=0x55bdfb065f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f404effd700 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb065f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f404effd700 1 -- 172.21.15.7:0/1777191884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 msgr2=0x55bdfb05e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f404effd700 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f404effd700 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4050073ec0 con 0x55bdfb055c10 2024-01-11T18:12:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f404effd700 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb05ef10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f404000c5b0 tx=0x7f404009af30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f12e1bba4f8d5df server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.776+0000 7f402effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:12:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.777+0000 7f402effd700 1 -- 172.21.15.7:0/1777191884 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400a27c0 con 0x55bdfb055c10 2024-01-11T18:12:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.777+0000 7f402effd700 1 -- 172.21.15.7:0/1777191884 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f404009a6c0 con 0x55bdfb055c10 2024-01-11T18:12:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.777+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdfb047d60 con 0x55bdfb055c10 2024-01-11T18:12:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.777+0000 7f402effd700 1 -- 172.21.15.7:0/1777191884 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40400ba0b0 con 0x55bdfb055c10 2024-01-11T18:12:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.777+0000 7f4069ef4b80 10 client.26579 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.777+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55bdfadc9b00 con 0x55bdfb055c10 2024-01-11T18:12:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f402effd700 1 -- 172.21.15.7:0/1777191884 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdfb047d60 con 0x55bdfb055c10 2024-01-11T18:12:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f402effd700 1 -- 172.21.15.7:0/1777191884 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4040083ab0 con 0x55bdfb055c10 2024-01-11T18:12:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f4069ef4b80 10 client.26579 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f4069ef4b80 10 client.26579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f4069ef4b80 20 client.26579 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb072d70 0x55bdfb075210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f4069ef4b80 4 client.26579 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:12:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.779+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdfafa1b60 con 0x55bdfb072d70 2024-01-11T18:12:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.780+0000 7f404f7fe700 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb072d70 0x55bdfb075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.783+0000 7f402effd700 10 client.26579 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.783+0000 7f404f7fe700 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb072d70 0x55bdfb075210 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1adac99fb8a1a15 server_cookie=2c060e46530ee456 in_seq=0 out_seq=0 2024-01-11T18:12:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f402effd700 1 -- 172.21.15.7:0/1777191884 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdfafa1b60 con 0x55bdfb072d70 2024-01-11T18:12:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f402effd700 10 client.26579 handle_command_reply: tid=0 2024-01-11T18:12:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f4069ef4b80 1 client.26579 shutdown 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 msgr2=0x55bdfb05ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb05ef10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f404000c5b0 tx=0x7f404009af30 comp rx=0 tx=0).stop 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 shutdown_connections 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.784+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb072d70 0x55bdfb075210 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f402effd700 0 client.26579 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/1777191884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb05ef10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 >> 172.21.15.7:0/1777191884 conn(0x55bdfafca740 msgr2=0x55bdfb04ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 shutdown_connections 2024-01-11T18:12:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 1 -- 172.21.15.7:0/1777191884 wait complete. 2024-01-11T18:12:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.785+0000 7f4069ef4b80 20 client.26579 trim_cache size 0 max 16384 2024-01-11T18:12:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.827+0000 7f4069ef4b80 1 Processor -- start 2024-01-11T18:12:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.827+0000 7f4069ef4b80 1 -- start start 2024-01-11T18:12:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb065f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb05cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfae1f640 con 0x55bdfb054f20 2024-01-11T18:12:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb05f8d0 con 0x55bdfb055c10 2024-01-11T18:12:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb0512e0 con 0x55bdfb0565d0 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f404effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f404effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb065f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48080/0 (socket says 172.21.15.7:48080) 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f404effd700 1 -- 172.21.15.7:0/3914821600 learned_addr learned my addr 172.21.15.7:0/3914821600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.828+0000 7f404ffff700 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb05cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.829+0000 7f404f7fe700 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.829+0000 7f404effd700 1 -- 172.21.15.7:0/3914821600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 msgr2=0x55bdfb05cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.829+0000 7f404effd700 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb05cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.829+0000 7f404effd700 1 -- 172.21.15.7:0/3914821600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 msgr2=0x55bdfb05df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.829+0000 7f404effd700 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.829+0000 7f404effd700 1 -- 172.21.15.7:0/3914821600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfaf6a1d0 con 0x55bdfb055c10 2024-01-11T18:12:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.830+0000 7f404effd700 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb065f10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4040096b70 tx=0x7f4040009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a33ab9fe3c4d58e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.830+0000 7f40357fa700 1 -- 172.21.15.7:0/3914821600 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400ba0b0 con 0x55bdfb055c10 2024-01-11T18:12:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.830+0000 7f40357fa700 1 -- 172.21.15.7:0/3914821600 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40400b83c0 con 0x55bdfb055c10 2024-01-11T18:12:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.830+0000 7f40357fa700 1 -- 172.21.15.7:0/3914821600 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400a2ca0 con 0x55bdfb055c10 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.831+0000 7f4069ef4b80 1 -- 172.21.15.7:0/3914821600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 msgr2=0x55bdfb065f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.831+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb065f10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4040096b70 tx=0x7f4040009f90 comp rx=0 tx=0).stop 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 -- 172.21.15.7:0/3914821600 shutdown_connections 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb0565d0 0x55bdfb05cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb05df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/3914821600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb055c10 0x55bdfb065f10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 -- 172.21.15.7:0/3914821600 >> 172.21.15.7:0/3914821600 conn(0x55bdfafca740 msgr2=0x55bdfb04b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 -- 172.21.15.7:0/3914821600 shutdown_connections 2024-01-11T18:12:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.832+0000 7f4069ef4b80 1 -- 172.21.15.7:0/3914821600 wait complete. 2024-01-11T18:12:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.833+0000 7f4069ef4b80 1 -- start start 2024-01-11T18:12:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.833+0000 7f4069ef4b80 1 Processor -- start 2024-01-11T18:12:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.834+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb0636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.834+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.834+0000 7f4069ef4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 0x55bdfb064c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb065160 con 0x55bdfb054f20 2024-01-11T18:12:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb0652d0 con 0x55bdfb0565d0 2024-01-11T18:12:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f4069ef4b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb065490 con 0x55bdfb055c10 2024-01-11T18:12:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f4069ef4b80 10 client.0 _ll_register_callbacks cb 0x55bdfaf496c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f404ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 0x55bdfb064c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f404ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 0x55bdfb064c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48090/0 (socket says 172.21.15.7:48090) 2024-01-11T18:12:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.835+0000 7f404ffff700 1 -- 172.21.15.7:0/723297457 learned_addr learned my addr 172.21.15.7:0/723297457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:12.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404ffff700 1 -- 172.21.15.7:0/723297457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 msgr2=0x55bdfb063c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404ffff700 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb063c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404ffff700 1 -- 172.21.15.7:0/723297457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 msgr2=0x55bdfb0636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404f7fe700 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb0636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404ffff700 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb0636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404ffff700 1 -- 172.21.15.7:0/723297457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdfadc9b00 con 0x55bdfb0565d0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f404ffff700 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 0x55bdfb064c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40480159b0 tx=0x7f4048004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0e5b24d2d0d0d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.836+0000 7f402effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.837+0000 7f402effd700 1 -- 172.21.15.7:0/723297457 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800e030 con 0x55bdfb0565d0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.837+0000 7f402effd700 1 -- 172.21.15.7:0/723297457 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f404801a040 con 0x55bdfb0565d0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.837+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f404009adc0 con 0x55bdfb0565d0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.837+0000 7f402effd700 1 -- 172.21.15.7:0/723297457 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4048015e20 con 0x55bdfb0565d0 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.838+0000 7f4069ef4b80 10 client.26585 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.838+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55bdfae21c70 con 0x55bdfb0565d0 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.838+0000 7f402effd700 1 -- 172.21.15.7:0/723297457 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404009adc0 con 0x55bdfb0565d0 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.838+0000 7f402effd700 1 -- 172.21.15.7:0/723297457 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f404802da70 con 0x55bdfb0565d0 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f4069ef4b80 10 client.26585 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f4069ef4b80 10 client.26585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f4069ef4b80 20 client.26585 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb06e740 0x55bdfb06eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f4069ef4b80 4 client.26585 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdfafa1880 con 0x55bdfb06e740 2024-01-11T18:12:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.839+0000 7f404f7fe700 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb06e740 0x55bdfb06eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.840+0000 7f404f7fe700 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb06e740 0x55bdfb06eb50 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504a49aeb92ef0e5 server_cookie=806c2dcffab8bbc0 in_seq=0 out_seq=0 2024-01-11T18:12:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.840+0000 7f402effd700 10 client.26585 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.853+0000 7f402effd700 1 -- 172.21.15.7:0/723297457 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdfafa1880 con 0x55bdfb06e740 2024-01-11T18:12:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.853+0000 7f402effd700 10 client.26585 handle_command_reply: tid=0 2024-01-11T18:12:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.853+0000 7f4069ef4b80 1 client.26585 shutdown 2024-01-11T18:12:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.853+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 msgr2=0x55bdfb064c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.853+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 0x55bdfb064c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40480159b0 tx=0x7f4048004ad0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 shutdown_connections 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bdfb06e740 0x55bdfb06eb50 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bdfb0565d0 0x55bdfb064c20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bdfb054f20 0x55bdfb0636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f402effd700 0 client.26585 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 --2- 172.21.15.7:0/723297457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bdfb055c10 0x55bdfb063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 >> 172.21.15.7:0/723297457 conn(0x55bdfafca740 msgr2=0x55bdfb05df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.854+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 shutdown_connections 2024-01-11T18:12:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.855+0000 7f4069ef4b80 1 -- 172.21.15.7:0/723297457 wait complete. 2024-01-11T18:12:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.855+0000 7f4069ef4b80 20 client.26585 trim_cache size 0 max 16384 2024-01-11T18:12:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:12:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:12:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:12:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40380a1f50 msgr2=0x7f40380a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40380a1f50 0x7f40380a4410 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4048010490 tx=0x7f4048009d90 comp rx=0 tx=0).stop 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40501027a0 msgr2=0x7f4050197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40501027a0 0x7f4050197b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f404000e9c0 tx=0x7f404000ecd0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 shutdown_connections 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f40380a1f50 0x7f40380a4410 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4050103140 0x7f405019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40501015b0 0x7f4050197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 --2- 172.21.15.7:0/1940154523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40501027a0 0x7f4050197b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 >> 172.21.15.7:0/1940154523 conn(0x7f40500fcb00 msgr2=0x7f4050105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 shutdown_connections 2024-01-11T18:12:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:12.858+0000 7f40561ff700 1 -- 172.21.15.7:0/1940154523 wait complete. 2024-01-11T18:12:13.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:13 smithi016 ceph-mon[43195]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 469 op/s 2024-01-11T18:12:13.519 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:13 smithi038 ceph-mon[43582]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 469 op/s 2024-01-11T18:12:13.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:13 smithi007 ceph-mon[33361]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 469 op/s 2024-01-11T18:12:14.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 825.0 s, 520.2 tps, lat 197.539 ms stddev 253.038 2024-01-11T18:12:16.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:15 smithi016 ceph-mon[43195]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-01-11T18:12:16.270 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:15 smithi038 ceph-mon[43582]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-01-11T18:12:16.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:15 smithi007 ceph-mon[33361]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-01-11T18:12:17.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:17 smithi016 ceph-mon[43195]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 523 op/s 2024-01-11T18:12:17.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:17 smithi038 ceph-mon[43582]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 523 op/s 2024-01-11T18:12:17.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:17 smithi007 ceph-mon[33361]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 523 op/s 2024-01-11T18:12:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 830.0 s, 462.4 tps, lat 217.256 ms stddev 256.668 2024-01-11T18:12:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:19 smithi016 ceph-mon[43195]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.1 MiB/s wr, 446 op/s 2024-01-11T18:12:19.523 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:19 smithi038 ceph-mon[43582]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.1 MiB/s wr, 446 op/s 2024-01-11T18:12:19.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:19 smithi007 ceph-mon[33361]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.1 MiB/s wr, 446 op/s 2024-01-11T18:12:21.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:21 smithi016 ceph-mon[43195]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.8 MiB/s wr, 498 op/s 2024-01-11T18:12:21.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:21 smithi038 ceph-mon[43582]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.8 MiB/s wr, 498 op/s 2024-01-11T18:12:21.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:21 smithi007 ceph-mon[33361]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.8 MiB/s wr, 498 op/s 2024-01-11T18:12:23.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:23 smithi016 ceph-mon[43195]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.8 MiB/s wr, 494 op/s 2024-01-11T18:12:23.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:23 smithi038 ceph-mon[43582]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.8 MiB/s wr, 494 op/s 2024-01-11T18:12:23.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:23 smithi007 ceph-mon[33361]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.8 MiB/s wr, 494 op/s 2024-01-11T18:12:24.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 835.0 s, 454.2 tps, lat 215.989 ms stddev 246.836 2024-01-11T18:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 -- 172.21.15.7:0/2517434272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 msgr2=0x7feacc106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 --2- 172.21.15.7:0/2517434272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc106de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feac00098f0 tx=0x7feac0009c00 comp rx=0 tx=0).stop 2024-01-11T18:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 -- 172.21.15.7:0/2517434272 shutdown_connections 2024-01-11T18:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 --2- 172.21.15.7:0/2517434272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feacc107320 0x7feacc109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 --2- 172.21.15.7:0/2517434272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feacc1020c0 0x7feacc1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 --2- 172.21.15.7:0/2517434272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc106de0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.101+0000 7fead1fff700 1 -- 172.21.15.7:0/2517434272 >> 172.21.15.7:0/2517434272 conn(0x7feacc0fba30 msgr2=0x7feacc0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.102+0000 7fead1fff700 1 -- 172.21.15.7:0/2517434272 shutdown_connections 2024-01-11T18:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.102+0000 7fead1fff700 1 -- 172.21.15.7:0/2517434272 wait complete. 2024-01-11T18:12:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.103+0000 7fead1fff700 1 Processor -- start 2024-01-11T18:12:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 -- start start 2024-01-11T18:12:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feacc1020c0 0x7feacc1104e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feacc107320 0x7feacc10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc10d810 con 0x7feacc1020c0 2024-01-11T18:12:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc10d980 con 0x7feacc107320 2024-01-11T18:12:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.104+0000 7fead1fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc10db40 con 0x7feacc1049f0 2024-01-11T18:12:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc110a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35928/0 (socket says 172.21.15.7:35928) 2024-01-11T18:12:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 -- 172.21.15.7:0/3153895051 learned_addr learned my addr 172.21.15.7:0/3153895051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacbfff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feacc107320 0x7feacc10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacb7fe700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feacc1020c0 0x7feacc1104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 -- 172.21.15.7:0/3153895051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feacc107320 msgr2=0x7feacc10d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feacc107320 0x7feacc10d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 -- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feacc1020c0 msgr2=0x7feacc1104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feacc1020c0 0x7feacc1104e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.105+0000 7feacaffd700 1 -- 172.21.15.7:0/3153895051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0009520 con 0x7feacc1049f0 2024-01-11T18:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.106+0000 7feacaffd700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc110a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feac0000c00 tx=0x7feac000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.109+0000 7feac8ff9700 1 -- 172.21.15.7:0/3153895051 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac001c070 con 0x7feacc1049f0 2024-01-11T18:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.109+0000 7feac8ff9700 1 -- 172.21.15.7:0/3153895051 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feac000bca0 con 0x7feacc1049f0 2024-01-11T18:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.109+0000 7feac8ff9700 1 -- 172.21.15.7:0/3153895051 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0004e50 con 0x7feacc1049f0 2024-01-11T18:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.110+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc1993f0 con 0x7feacc1049f0 2024-01-11T18:12:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.110+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc199850 con 0x7feacc1049f0 2024-01-11T18:12:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.115+0000 7feac8ff9700 1 -- 172.21.15.7:0/3153895051 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7feac0005030 con 0x7feacc1049f0 2024-01-11T18:12:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.116+0000 7feac8ff9700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feab40a20e0 0x7feab40a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.116+0000 7feac8ff9700 1 -- 172.21.15.7:0/3153895051 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feac00cebe0 con 0x7feacc1049f0 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.116+0000 7feacb7fe700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feab40a20e0 0x7feab40a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.117+0000 7feae5d94b80 1 Processor -- start 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.117+0000 7feacb7fe700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feab40a20e0 0x7feab40a45a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feabc009b10 tx=0x7feabc0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.117+0000 7feae5d94b80 1 -- start start 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.118+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d191340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.118+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 0x555b8d192090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.118+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d198e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.118+0000 7feae5d94b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d088030 con 0x555b8d190f20 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.118+0000 7feae5d94b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d0ebac0 con 0x555b8d1925d0 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.118+0000 7feae5d94b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d0e93e0 con 0x555b8d191c10 2024-01-11T18:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 0x555b8d192090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 0x555b8d192090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35944/0 (socket says 172.21.15.7:35944) 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 -- 172.21.15.7:0/2508817026 learned_addr learned my addr 172.21.15.7:0/2508817026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacbfff700 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 -- 172.21.15.7:0/2508817026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 msgr2=0x555b8d198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 -- 172.21.15.7:0/2508817026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 msgr2=0x555b8d191340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.119+0000 7feacaffd700 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d191340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.120+0000 7feacaffd700 1 -- 172.21.15.7:0/2508817026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b8cf59350 con 0x555b8d191c10 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.120+0000 7feacaffd700 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 0x555b8d192090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feac009b5d0 tx=0x7feac009b200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700d3eaaaae717f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.121+0000 7feab1ffb700 1 -- 172.21.15.7:0/2508817026 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac009f790 con 0x555b8d191c10 2024-01-11T18:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.121+0000 7feab1ffb700 1 -- 172.21.15.7:0/2508817026 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feac00a3940 con 0x555b8d191c10 2024-01-11T18:12:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.121+0000 7feab1ffb700 1 -- 172.21.15.7:0/2508817026 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0097d50 con 0x555b8d191c10 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 -- 172.21.15.7:0/2508817026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 msgr2=0x555b8d192090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 0x555b8d192090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feac009b5d0 tx=0x7feac009b200 comp rx=0 tx=0).stop 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 -- 172.21.15.7:0/2508817026 shutdown_connections 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d191340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2508817026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d191c10 0x555b8d192090 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feac009b5d0 tx=0x7feac009b200 comp rx=0 tx=0).stop 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.122+0000 7feae5d94b80 1 -- 172.21.15.7:0/2508817026 >> 172.21.15.7:0/2508817026 conn(0x555b8d106740 msgr2=0x555b8d18b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.123+0000 7feae5d94b80 1 -- 172.21.15.7:0/2508817026 shutdown_connections 2024-01-11T18:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.123+0000 7feae5d94b80 1 -- 172.21.15.7:0/2508817026 wait complete. 2024-01-11T18:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.124+0000 7feae5d94b80 1 -- start start 2024-01-11T18:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.124+0000 7feae5d94b80 1 Processor -- start 2024-01-11T18:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.125+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d19a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.125+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d1925d0 0x555b8d19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d19a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:35950/0 (socket says 172.21.15.7:35950) 2024-01-11T18:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 -- 172.21.15.7:0/3753596145 learned_addr learned my addr 172.21.15.7:0/3753596145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d19feb0 0x555b8d1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d0e4cd0 con 0x555b8d1925d0 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d053340 con 0x555b8d19feb0 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d052fe0 con 0x555b8d190f20 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feae5d94b80 10 client.0 _ll_register_callbacks cb 0x555b8d0856c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 -- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d19feb0 msgr2=0x555b8d1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d19feb0 0x555b8d1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 -- 172.21.15.7:0/3753596145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d1925d0 msgr2=0x555b8d19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacaffd700 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d1925d0 0x555b8d19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.126+0000 7feacb7fe700 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d1925d0 0x555b8d19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.127+0000 7feacb7fe700 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc199850 con 0x555b8d190f20 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.127+0000 7feacb7fe700 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d19a940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feabc011720 tx=0x7feabc011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ca3725daaeca3a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.127+0000 7feaaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.127+0000 7feaaaffd700 1 -- 172.21.15.7:0/3753596145 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00e7c0 con 0x555b8d190f20 2024-01-11T18:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.127+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b8d183d60 con 0x555b8d190f20 2024-01-11T18:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.127+0000 7feaaaffd700 1 -- 172.21.15.7:0/3753596145 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feabc01e640 con 0x555b8d190f20 2024-01-11T18:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feaaaffd700 1 -- 172.21.15.7:0/3753596145 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7feabc01e800 con 0x555b8d190f20 2024-01-11T18:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 10 client.26629 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555b8cf05b00 con 0x555b8d190f20 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feaaaffd700 1 -- 172.21.15.7:0/3753596145 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b8d183d60 con 0x555b8d190f20 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feaaaffd700 1 -- 172.21.15.7:0/3753596145 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7feabc010aa0 con 0x555b8d190f20 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 10 client.26629 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 20 client.26629 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d1a6c50 0x555b8d1a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 4 client.26629 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.128+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b8d0ddb60 con 0x555b8d1a6c50 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.129+0000 7feacaffd700 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d1a6c50 0x555b8d1a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.129+0000 7feaaaffd700 10 client.26629 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.129+0000 7feacaffd700 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d1a6c50 0x555b8d1a9070 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ac15e5def36cba server_cookie=617f740267363ad4 in_seq=0 out_seq=0 2024-01-11T18:12:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.130+0000 7feaaaffd700 1 -- 172.21.15.7:0/3753596145 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b8d0ddb60 con 0x555b8d1a6c50 2024-01-11T18:12:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.130+0000 7feaaaffd700 10 client.26629 handle_command_reply: tid=0 2024-01-11T18:12:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.130+0000 7feae5d94b80 1 client.26629 shutdown 2024-01-11T18:12:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 msgr2=0x555b8d19a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d19a940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feabc011720 tx=0x7feabc011ae0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 shutdown_connections 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d1a6c50 0x555b8d1a9070 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d19a940 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d19feb0 0x555b8d1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feaaaffd700 0 client.26629 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 --2- 172.21.15.7:0/3753596145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d1925d0 0x555b8d19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 >> 172.21.15.7:0/3753596145 conn(0x555b8d106740 msgr2=0x555b8d18af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 shutdown_connections 2024-01-11T18:12:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.131+0000 7feae5d94b80 1 -- 172.21.15.7:0/3753596145 wait complete. 2024-01-11T18:12:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.132+0000 7feae5d94b80 20 client.26629 trim_cache size 0 max 16384 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.169+0000 7feae5d94b80 1 Processor -- start 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.169+0000 7feae5d94b80 1 -- start start 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d1a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d19feb0 0x555b8d18af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feae5d94b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8cf5b640 con 0x555b8d190f20 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feae5d94b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d0ddb60 con 0x555b8d1925d0 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feae5d94b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d191a40 con 0x555b8d19feb0 2024-01-11T18:12:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feacaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feacaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54948/0 (socket says 172.21.15.7:54948) 2024-01-11T18:12:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.170+0000 7feacaffd700 1 -- 172.21.15.7:0/2622294687 learned_addr learned my addr 172.21.15.7:0/2622294687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.171+0000 7feacb7fe700 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d1a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.175+0000 7feacb7fe700 1 -- 172.21.15.7:0/2622294687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d19feb0 msgr2=0x555b8d18af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.175+0000 7feacb7fe700 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d19feb0 0x555b8d18af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.175+0000 7feacb7fe700 1 -- 172.21.15.7:0/2622294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 msgr2=0x555b8d1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.175+0000 7feacb7fe700 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.175+0000 7feacb7fe700 1 -- 172.21.15.7:0/2622294687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b8d18d2f0 con 0x555b8d190f20 2024-01-11T18:12:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.176+0000 7feacb7fe700 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d1a2980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feabc010ea0 tx=0x7feabc011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66be295a8222f54b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.180+0000 7feab17fa700 1 -- 172.21.15.7:0/2622294687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00e7c0 con 0x555b8d190f20 2024-01-11T18:12:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.180+0000 7feab17fa700 1 -- 172.21.15.7:0/2622294687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feabc015040 con 0x555b8d190f20 2024-01-11T18:12:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.180+0000 7feab17fa700 1 -- 172.21.15.7:0/2622294687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc01eb60 con 0x555b8d190f20 2024-01-11T18:12:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 -- 172.21.15.7:0/2622294687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 msgr2=0x555b8d1a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d1a2980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feabc010ea0 tx=0x7feabc011f40 comp rx=0 tx=0).stop 2024-01-11T18:12:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 -- 172.21.15.7:0/2622294687 shutdown_connections 2024-01-11T18:12:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d19feb0 0x555b8d18af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d190f20 0x555b8d1a2980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2622294687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 -- 172.21.15.7:0/2622294687 >> 172.21.15.7:0/2622294687 conn(0x555b8d106740 msgr2=0x555b8d1879b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 -- 172.21.15.7:0/2622294687 shutdown_connections 2024-01-11T18:12:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.181+0000 7feae5d94b80 1 -- 172.21.15.7:0/2622294687 wait complete. 2024-01-11T18:12:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.182+0000 7feae5d94b80 1 -- start start 2024-01-11T18:12:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.182+0000 7feae5d94b80 1 Processor -- start 2024-01-11T18:12:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.184+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d1b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.184+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feae5d94b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d19feb0 0x555b8d20cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feae5d94b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d1b4d90 con 0x555b8d19feb0 2024-01-11T18:12:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feae5d94b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d20d100 con 0x555b8d1925d0 2024-01-11T18:12:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feae5d94b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8d20d240 con 0x555b8d190f20 2024-01-11T18:12:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feae5d94b80 10 client.0 _ll_register_callbacks cb 0x555b8d0856c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feacaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feacaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54954/0 (socket says 172.21.15.7:54954) 2024-01-11T18:12:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.185+0000 7feacaffd700 1 -- 172.21.15.7:0/2802263424 learned_addr learned my addr 172.21.15.7:0/2802263424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.187+0000 7feacbfff700 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d19feb0 0x555b8d20cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.187+0000 7feacbfff700 1 -- 172.21.15.7:0/2802263424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 msgr2=0x555b8d1b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.187+0000 7feacbfff700 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d1b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.187+0000 7feacbfff700 1 -- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 msgr2=0x555b8d1b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.187+0000 7feacbfff700 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.187+0000 7feacbfff700 1 -- 172.21.15.7:0/2802263424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b8cf05b00 con 0x555b8d19feb0 2024-01-11T18:12:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.188+0000 7feaaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:12:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.188+0000 7feacbfff700 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d19feb0 0x555b8d20cbc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x555b8d186590 tx=0x7feac4004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b42629719a4c661 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.189+0000 7feaaaffd700 1 -- 172.21.15.7:0/2802263424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4003a20 con 0x555b8d19feb0 2024-01-11T18:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.191+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feac00a3940 con 0x555b8d19feb0 2024-01-11T18:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.191+0000 7feaaaffd700 1 -- 172.21.15.7:0/2802263424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feac4007910 con 0x555b8d19feb0 2024-01-11T18:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.191+0000 7feaaaffd700 1 -- 172.21.15.7:0/2802263424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7feac4003a20 con 0x555b8d19feb0 2024-01-11T18:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.191+0000 7feae5d94b80 10 client.16335 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.191+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555b8d0a61d0 con 0x555b8d19feb0 2024-01-11T18:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feaaaffd700 1 -- 172.21.15.7:0/2802263424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feac00a3940 con 0x555b8d19feb0 2024-01-11T18:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feaaaffd700 1 -- 172.21.15.7:0/2802263424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7feac4014040 con 0x555b8d19feb0 2024-01-11T18:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feae5d94b80 10 client.16335 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feae5d94b80 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feae5d94b80 20 client.16335 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d20ecd0 0x555b8d2110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feae5d94b80 4 client.16335 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.192+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b8d0dd880 con 0x555b8d20ecd0 2024-01-11T18:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.193+0000 7feacb7fe700 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d20ecd0 0x555b8d2110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.195+0000 7feacb7fe700 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d20ecd0 0x555b8d2110f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ac6a4a963b8f49 server_cookie=f97635f619cc4d21 in_seq=0 out_seq=0 2024-01-11T18:12:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.195+0000 7feaaaffd700 10 client.16335 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:26.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:25 smithi016 ceph-mon[43195]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-01-11T18:12:26.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:25 smithi038 ceph-mon[43582]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-01-11T18:12:26.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:25 smithi007 ceph-mon[33361]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-01-11T18:12:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.381+0000 7feaaaffd700 1 -- 172.21.15.7:0/2802263424 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b8d0dd880 con 0x555b8d20ecd0 2024-01-11T18:12:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.381+0000 7feaaaffd700 10 client.16335 handle_command_reply: tid=0 2024-01-11T18:12:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.381+0000 7feae5d94b80 1 client.16335 shutdown 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d19feb0 msgr2=0x555b8d20cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d19feb0 0x555b8d20cbc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x555b8d186590 tx=0x7feac4004320 comp rx=0 tx=0).stop 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 shutdown_connections 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555b8d20ecd0 0x555b8d2110f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b8d190f20 0x555b8d1b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555b8d19feb0 0x555b8d20cbc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 --2- 172.21.15.7:0/2802263424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b8d1925d0 0x555b8d1b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 >> 172.21.15.7:0/2802263424 conn(0x555b8d106740 msgr2=0x555b8d18af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.382+0000 7feaaaffd700 0 client.16335 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.384+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 shutdown_connections 2024-01-11T18:12:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.384+0000 7feae5d94b80 1 -- 172.21.15.7:0/2802263424 wait complete. 2024-01-11T18:12:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.384+0000 7feae5d94b80 20 client.16335 trim_cache size 0 max 16384 2024-01-11T18:12:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.385+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feab40a20e0 msgr2=0x7feab40a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.385+0000 7fead1fff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feab40a20e0 0x7feab40a45a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feabc009b10 tx=0x7feabc0094f0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.385+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 msgr2=0x7feacc110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.385+0000 7fead1fff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc110a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7feac0000c00 tx=0x7feac000b840 comp rx=0 tx=0).stop 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.385+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 shutdown_connections 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7feab40a20e0 0x7feab40a45a0 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feabc009b10 tx=0x7feabc0094f0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feacc107320 0x7feacc10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feacc1020c0 0x7feacc1104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 --2- 172.21.15.7:0/3153895051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feacc1049f0 0x7feacc110a20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 >> 172.21.15.7:0/3153895051 conn(0x7feacc0fba30 msgr2=0x7feacc105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 shutdown_connections 2024-01-11T18:12:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:26.386+0000 7fead1fff700 1 -- 172.21.15.7:0/3153895051 wait complete. 2024-01-11T18:12:27.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:27 smithi016 ceph-mon[43195]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.4 MiB/s wr, 515 op/s 2024-01-11T18:12:27.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:27 smithi038 ceph-mon[43582]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.4 MiB/s wr, 515 op/s 2024-01-11T18:12:27.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:27 smithi007 ceph-mon[33361]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.4 MiB/s wr, 515 op/s 2024-01-11T18:12:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 840.0 s, 398.2 tps, lat 254.135 ms stddev 300.821 2024-01-11T18:12:29.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:29 smithi016 ceph-mon[43195]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2024-01-11T18:12:29.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:29 smithi038 ceph-mon[43582]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2024-01-11T18:12:29.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:29 smithi007 ceph-mon[33361]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2024-01-11T18:12:31.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:31 smithi016 ceph-mon[43195]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.7 MiB/s wr, 496 op/s 2024-01-11T18:12:31.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:31 smithi038 ceph-mon[43582]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.7 MiB/s wr, 496 op/s 2024-01-11T18:12:31.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:31 smithi007 ceph-mon[33361]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.7 MiB/s wr, 496 op/s 2024-01-11T18:12:33.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:33 smithi016 ceph-mon[43195]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-01-11T18:12:33.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:33 smithi038 ceph-mon[43582]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-01-11T18:12:33.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:33 smithi007 ceph-mon[33361]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-01-11T18:12:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 845.0 s, 479.0 tps, lat 211.915 ms stddev 232.349 2024-01-11T18:12:36.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:35 smithi016 ceph-mon[43195]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 444 op/s 2024-01-11T18:12:36.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:35 smithi038 ceph-mon[43582]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 444 op/s 2024-01-11T18:12:36.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:35 smithi007 ceph-mon[33361]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 444 op/s 2024-01-11T18:12:37.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:37 smithi016 ceph-mon[43195]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 7.9 MiB/s wr, 548 op/s 2024-01-11T18:12:37.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:37 smithi038 ceph-mon[43582]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 7.9 MiB/s wr, 548 op/s 2024-01-11T18:12:37.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:37 smithi007 ceph-mon[33361]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 7.9 MiB/s wr, 548 op/s 2024-01-11T18:12:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 850.0 s, 460.4 tps, lat 213.632 ms stddev 244.531 2024-01-11T18:12:39.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:39 smithi016 ceph-mon[43195]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 7.1 MiB/s wr, 506 op/s 2024-01-11T18:12:39.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:39 smithi038 ceph-mon[43582]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 7.1 MiB/s wr, 506 op/s 2024-01-11T18:12:39.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:39 smithi007 ceph-mon[33361]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 7.1 MiB/s wr, 506 op/s 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.660+0000 7fec0cb9d700 1 -- 172.21.15.7:0/4256713682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 msgr2=0x7fec000a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.660+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/4256713682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec000a47e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7febf4009990 tx=0x7febf4009ca0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 -- 172.21.15.7:0/4256713682 shutdown_connections 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/4256713682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec000a47e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/4256713682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec00008d00 0x7fec00007150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/4256713682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec00007720 0x7fec00007ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 -- 172.21.15.7:0/4256713682 >> 172.21.15.7:0/4256713682 conn(0x7fec00091370 msgr2=0x7fec000937d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 -- 172.21.15.7:0/4256713682 shutdown_connections 2024-01-11T18:12:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.661+0000 7fec0cb9d700 1 -- 172.21.15.7:0/4256713682 wait complete. 2024-01-11T18:12:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.662+0000 7fec0cb9d700 1 Processor -- start 2024-01-11T18:12:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 -- start start 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec00007720 0x7fec001320e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec00008d00 0x7fec00132620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec00139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00139c10 con 0x7fec00007720 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00139d50 con 0x7fec00008d00 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec0cb9d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00139ee0 con 0x7fec000a23f0 2024-01-11T18:12:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.663+0000 7fec07fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec00139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec00139640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:36632/0 (socket says 172.21.15.7:36632) 2024-01-11T18:12:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 -- 172.21.15.7:0/679600575 learned_addr learned my addr 172.21.15.7:0/679600575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 -- 172.21.15.7:0/679600575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec00008d00 msgr2=0x7fec00132620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T18:12:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec00008d00 0x7fec00132620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 -- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec00007720 msgr2=0x7fec001320e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T18:12:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec00007720 0x7fec001320e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.664+0000 7fec07fff700 1 -- 172.21.15.7:0/679600575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf40095f0 con 0x7fec000a23f0 2024-01-11T18:12:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.666+0000 7fec07fff700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec00139640 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7febf400f690 tx=0x7febf400f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.666+0000 7fec04ff9700 1 -- 172.21.15.7:0/679600575 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf401d070 con 0x7fec000a23f0 2024-01-11T18:12:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.666+0000 7fec04ff9700 1 -- 172.21.15.7:0/679600575 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf4022930 con 0x7fec000a23f0 2024-01-11T18:12:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.666+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0013a160 con 0x7fec000a23f0 2024-01-11T18:12:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.666+0000 7fec04ff9700 1 -- 172.21.15.7:0/679600575 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf40176d0 con 0x7fec000a23f0 2024-01-11T18:12:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.666+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0013a580 con 0x7fec000a23f0 2024-01-11T18:12:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.673+0000 7fec21aaeb80 1 Processor -- start 2024-01-11T18:12:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.675+0000 7fec04ff9700 1 -- 172.21.15.7:0/679600575 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7febf4022470 con 0x7fec000a23f0 2024-01-11T18:12:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.675+0000 7fec21aaeb80 1 -- start start 2024-01-11T18:12:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.676+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.676+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.676+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.676+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2764cb030 con 0x55d2765d5870 2024-01-11T18:12:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.676+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27652eac0 con 0x55d2765d4120 2024-01-11T18:12:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.676+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27652c3e0 con 0x55d2765d4eb0 2024-01-11T18:12:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.677+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.677+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43518/0 (socket says 172.21.15.7:43518) 2024-01-11T18:12:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.677+0000 7fec06ffd700 1 -- 172.21.15.7:0/1468930683 learned_addr learned my addr 172.21.15.7:0/1468930683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.677+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.677+0000 7fec077fe700 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.678+0000 7fec06ffd700 1 -- 172.21.15.7:0/1468930683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 msgr2=0x55d2765d5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.678+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765d5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.678+0000 7fec06ffd700 1 -- 172.21.15.7:0/1468930683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 msgr2=0x55d2765d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.678+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765d4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.678+0000 7fec06ffd700 1 -- 172.21.15.7:0/1468930683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d27639c350 con 0x55d2765d5870 2024-01-11T18:12:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.678+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765dc100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55d2765d3ec0 tx=0x7febfc005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d29752684f0ecf8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.679+0000 7fec04ff9700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febf80a1f60 0x7febf80a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.679+0000 7febf1ffb700 1 -- 172.21.15.7:0/1468930683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc013e20 con 0x55d2765d5870 2024-01-11T18:12:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.679+0000 7fec077fe700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febf80a1f60 0x7febf80a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.679+0000 7fec04ff9700 1 -- 172.21.15.7:0/679600575 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7febf40cf320 con 0x7fec000a23f0 2024-01-11T18:12:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.680+0000 7febf1ffb700 1 -- 172.21.15.7:0/1468930683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febfc017070 con 0x55d2765d5870 2024-01-11T18:12:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.680+0000 7febf1ffb700 1 -- 172.21.15.7:0/1468930683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc013e20 con 0x55d2765d5870 2024-01-11T18:12:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.680+0000 7fec077fe700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febf80a1f60 0x7febf80a4420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fec080508c0 tx=0x7fec08066ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.681+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1468930683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 msgr2=0x55d2765dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.681+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765dc100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55d2765d3ec0 tx=0x7febfc005af0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.682+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1468930683 shutdown_connections 2024-01-11T18:12:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.682+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765dc100 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.683+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.683+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1468930683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765d5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.683+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1468930683 >> 172.21.15.7:0/1468930683 conn(0x55d276549740 msgr2=0x55d2765ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.683+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1468930683 shutdown_connections 2024-01-11T18:12:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.684+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1468930683 wait complete. 2024-01-11T18:12:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.685+0000 7fec21aaeb80 1 -- start start 2024-01-11T18:12:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.686+0000 7fec21aaeb80 1 Processor -- start 2024-01-11T18:12:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d276527cd0 con 0x55d2765d5870 2024-01-11T18:12:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d276496340 con 0x55d2765d4120 2024-01-11T18:12:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d276495fe0 con 0x55d2765d4eb0 2024-01-11T18:12:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.687+0000 7fec21aaeb80 10 client.0 _ll_register_callbacks cb 0x55d2764c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.688+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.688+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765de050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:36648/0 (socket says 172.21.15.7:36648) 2024-01-11T18:12:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.688+0000 7fec06ffd700 1 -- 172.21.15.7:0/1297527912 learned_addr learned my addr 172.21.15.7:0/1297527912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.688+0000 7fec077fe700 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.688+0000 7fec06ffd700 1 -- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 msgr2=0x55d2765ddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.688+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765ddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.689+0000 7fec06ffd700 1 -- 172.21.15.7:0/1297527912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 msgr2=0x55d2765e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.689+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.689+0000 7fec06ffd700 1 -- 172.21.15.7:0/1297527912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc013480 con 0x55d2765d4eb0 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.689+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765de050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7febfc013f10 tx=0x7febfc005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc0deda39ca4bd8f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.689+0000 7febe6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.690+0000 7febe6ffd700 1 -- 172.21.15.7:0/1297527912 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc003740 con 0x55d2765d4eb0 2024-01-11T18:12:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.690+0000 7febe6ffd700 1 -- 172.21.15.7:0/1297527912 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7febfc000e40 con 0x55d2765d4eb0 2024-01-11T18:12:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.690+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2765c6fb0 con 0x55d2765d4eb0 2024-01-11T18:12:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.691+0000 7febe6ffd700 1 -- 172.21.15.7:0/1297527912 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7febfc003740 con 0x55d2765d4eb0 2024-01-11T18:12:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.691+0000 7fec21aaeb80 10 client.26641 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.691+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d276348b00 con 0x55d2765d4eb0 2024-01-11T18:12:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.692+0000 7febe6ffd700 1 -- 172.21.15.7:0/1297527912 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2765c6fb0 con 0x55d2765d4eb0 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7febe6ffd700 1 -- 172.21.15.7:0/1297527912 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7febfc01c440 con 0x55d2765d4eb0 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec21aaeb80 10 client.26641 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec21aaeb80 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec21aaeb80 20 client.26641 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d2765fa900 0x55d2765fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec21aaeb80 4 client.26641 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d276520b60 con 0x55d2765fa900 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.693+0000 7fec077fe700 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d2765fa900 0x55d2765fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.694+0000 7fec077fe700 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d2765fa900 0x55d2765fcd20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bab2fab991ffda server_cookie=44bfdc8a0d51ab02 in_seq=0 out_seq=0 2024-01-11T18:12:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.694+0000 7febe6ffd700 10 client.26641 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.695+0000 7febe6ffd700 1 -- 172.21.15.7:0/1297527912 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d276520b60 con 0x55d2765fa900 2024-01-11T18:12:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.695+0000 7febe6ffd700 10 client.26641 handle_command_reply: tid=0 2024-01-11T18:12:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.695+0000 7fec21aaeb80 1 client.26641 shutdown 2024-01-11T18:12:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 msgr2=0x55d2765de050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765de050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7febfc013f10 tx=0x7febfc005720 comp rx=0 tx=0).stop 2024-01-11T18:12:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 shutdown_connections 2024-01-11T18:12:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d2765fa900 0x55d2765fcd20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d5870 0x55d2765e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765ddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1297527912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765de050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 >> 172.21.15.7:0/1297527912 conn(0x55d276549740 msgr2=0x55d2765ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7febe6ffd700 0 client.26641 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 shutdown_connections 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1297527912 wait complete. 2024-01-11T18:12:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.696+0000 7fec21aaeb80 20 client.26641 trim_cache size 0 max 16384 2024-01-11T18:12:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.723+0000 7fec21aaeb80 1 Processor -- start 2024-01-11T18:12:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.723+0000 7fec21aaeb80 1 -- start start 2024-01-11T18:12:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4120 0x55d2765f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 0x55d2765fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27639e640 con 0x55d2765d4120 2024-01-11T18:12:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2765fa440 con 0x55d2765d5870 2024-01-11T18:12:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2765d08f0 con 0x55d2765d4eb0 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec07fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 0x55d2765fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec07fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 0x55d2765fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43542/0 (socket says 172.21.15.7:43542) 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec07fff700 1 -- 172.21.15.7:0/913398262 learned_addr learned my addr 172.21.15.7:0/913398262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec06ffd700 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.724+0000 7fec077fe700 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4120 0x55d2765f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7fec07fff700 1 -- 172.21.15.7:0/913398262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 msgr2=0x55d2765e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7fec07fff700 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7fec07fff700 1 -- 172.21.15.7:0/913398262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4120 msgr2=0x55d2765f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7fec07fff700 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4120 0x55d2765f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7fec07fff700 1 -- 172.21.15.7:0/913398262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2764e91d0 con 0x55d2765d5870 2024-01-11T18:12:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7fec07fff700 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 0x55d2765fc910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febf40988f0 tx=0x7febf4098240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585fb353046eb03d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7febf17fa700 1 -- 172.21.15.7:0/913398262 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf40a20b0 con 0x55d2765d5870 2024-01-11T18:12:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7febf17fa700 1 -- 172.21.15.7:0/913398262 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf409c580 con 0x55d2765d5870 2024-01-11T18:12:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.725+0000 7febf17fa700 1 -- 172.21.15.7:0/913398262 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf409a030 con 0x55d2765d5870 2024-01-11T18:12:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 -- 172.21.15.7:0/913398262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 msgr2=0x55d2765fc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 0x55d2765fc910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febf40988f0 tx=0x7febf4098240 comp rx=0 tx=0).stop 2024-01-11T18:12:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 -- 172.21.15.7:0/913398262 shutdown_connections 2024-01-11T18:12:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d5870 0x55d2765fc910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4120 0x55d2765f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/913398262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d4eb0 0x55d2765e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 -- 172.21.15.7:0/913398262 >> 172.21.15.7:0/913398262 conn(0x55d276549740 msgr2=0x55d2765dc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 -- 172.21.15.7:0/913398262 shutdown_connections 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.726+0000 7fec21aaeb80 1 -- 172.21.15.7:0/913398262 wait complete. 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.727+0000 7fec21aaeb80 1 -- start start 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.727+0000 7fec21aaeb80 1 Processor -- start 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4eb0 0x55d2765e2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d5870 0x55d27664ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2765e5080 con 0x55d2765d4eb0 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2765e51f0 con 0x55d2765d4120 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2765e53b0 con 0x55d2765d5870 2024-01-11T18:12:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec21aaeb80 10 client.0 _ll_register_callbacks cb 0x55d2765dd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4eb0 0x55d2765e2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765e1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43552/0 (socket says 172.21.15.7:43552) 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 -- 172.21.15.7:0/1171741819 learned_addr learned my addr 172.21.15.7:0/1171741819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec07fff700 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d5870 0x55d27664ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 -- 172.21.15.7:0/1171741819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d5870 msgr2=0x55d27664ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d5870 0x55d27664ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 -- 172.21.15.7:0/1171741819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4eb0 msgr2=0x55d2765e2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4eb0 0x55d2765e2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.728+0000 7fec077fe700 1 -- 172.21.15.7:0/1171741819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0013a580 con 0x55d2765d4120 2024-01-11T18:12:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.729+0000 7fec077fe700 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765e1b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fec0804ff00 tx=0x7fec08081070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d7558b2f05802a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.729+0000 7febe6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:12:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.729+0000 7febe6ffd700 1 -- 172.21.15.7:0/1171741819 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0808f040 con 0x55d2765d4120 2024-01-11T18:12:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.729+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febfc017070 con 0x55d2765d4120 2024-01-11T18:12:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.729+0000 7febe6ffd700 1 -- 172.21.15.7:0/1171741819 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fec0806e050 con 0x55d2765d4120 2024-01-11T18:12:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.730+0000 7febe6ffd700 1 -- 172.21.15.7:0/1171741819 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec08088680 con 0x55d2765d4120 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.731+0000 7fec21aaeb80 10 client.26615 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.731+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d2763a0c70 con 0x55d2765d4120 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.731+0000 7febe6ffd700 1 -- 172.21.15.7:0/1171741819 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febfc017070 con 0x55d2765d4120 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7febe6ffd700 1 -- 172.21.15.7:0/1171741819 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fec0807f940 con 0x55d2765d4120 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec21aaeb80 10 client.26615 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec21aaeb80 10 client.26615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec21aaeb80 20 client.26615 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d276650d00 0x55d276653120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec21aaeb80 4 client.26615 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:12:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d276520880 con 0x55d276650d00 2024-01-11T18:12:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.732+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d276650d00 0x55d276653120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.733+0000 7fec06ffd700 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d276650d00 0x55d276653120 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89d82a7d7673ba8 server_cookie=4ad68f366d4f9c9e in_seq=0 out_seq=0 2024-01-11T18:12:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.733+0000 7febe6ffd700 10 client.26615 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.750+0000 7febe6ffd700 1 -- 172.21.15.7:0/1171741819 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d276520880 con 0x55d276650d00 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.750+0000 7febe6ffd700 10 client.26615 handle_command_reply: tid=0 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.750+0000 7fec21aaeb80 1 client.26615 shutdown 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 msgr2=0x55d2765e1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765e1b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fec0804ff00 tx=0x7fec08081070 comp rx=0 tx=0).stop 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 shutdown_connections 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55d276650d00 0x55d276653120 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d2765d5870 0x55d27664ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7febe6ffd700 0 client.26615 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2765d4120 0x55d2765e1b20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 --2- 172.21.15.7:0/1171741819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2765d4eb0 0x55d2765e2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 >> 172.21.15.7:0/1171741819 conn(0x55d276549740 msgr2=0x55d2765fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 shutdown_connections 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 1 -- 172.21.15.7:0/1171741819 wait complete. 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.751+0000 7fec21aaeb80 20 client.26615 trim_cache size 0 max 16384 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:12:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:12:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febf80a1f60 msgr2=0x7febf80a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febf80a1f60 0x7febf80a4420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fec080508c0 tx=0x7fec08066ec0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 msgr2=0x7fec00139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec00139640 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7febf400f690 tx=0x7febf400f770 comp rx=0 tx=0).stop 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 shutdown_connections 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7febf80a1f60 0x7febf80a4420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec000a23f0 0x7fec00139640 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec00007720 0x7fec001320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 --2- 172.21.15.7:0/679600575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec00008d00 0x7fec00132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.753+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 >> 172.21.15.7:0/679600575 conn(0x7fec00091370 msgr2=0x7fec00095790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.754+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 shutdown_connections 2024-01-11T18:12:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:39.754+0000 7fec0cb9d700 1 -- 172.21.15.7:0/679600575 wait complete. 2024-01-11T18:12:41.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:41 smithi016 ceph-mon[43195]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 8.5 MiB/s wr, 532 op/s 2024-01-11T18:12:41.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:41 smithi038 ceph-mon[43582]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 8.5 MiB/s wr, 532 op/s 2024-01-11T18:12:41.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:41 smithi007 ceph-mon[33361]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 8.5 MiB/s wr, 532 op/s 2024-01-11T18:12:43.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:43 smithi016 ceph-mon[43195]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-01-11T18:12:43.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:43 smithi038 ceph-mon[43582]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-01-11T18:12:43.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:43 smithi007 ceph-mon[33361]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 8.3 MiB/s wr, 515 op/s 2024-01-11T18:12:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 855.0 s, 582.2 tps, lat 174.140 ms stddev 211.197 2024-01-11T18:12:46.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:45 smithi016 ceph-mon[43195]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 7.1 MiB/s wr, 456 op/s 2024-01-11T18:12:46.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:45 smithi038 ceph-mon[43582]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 7.1 MiB/s wr, 456 op/s 2024-01-11T18:12:46.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:45 smithi007 ceph-mon[33361]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 7.1 MiB/s wr, 456 op/s 2024-01-11T18:12:47.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:47 smithi016 ceph-mon[43195]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 8.6 MiB/s wr, 542 op/s 2024-01-11T18:12:47.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:47 smithi038 ceph-mon[43582]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 8.6 MiB/s wr, 542 op/s 2024-01-11T18:12:47.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:47 smithi007 ceph-mon[33361]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 8.6 MiB/s wr, 542 op/s 2024-01-11T18:12:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 860.0 s, 436.4 tps, lat 222.813 ms stddev 266.648 2024-01-11T18:12:49.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:49 smithi016 ceph-mon[43195]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.4 MiB/s wr, 471 op/s 2024-01-11T18:12:49.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:49 smithi038 ceph-mon[43582]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.4 MiB/s wr, 471 op/s 2024-01-11T18:12:49.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:49 smithi007 ceph-mon[33361]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.4 MiB/s wr, 471 op/s 2024-01-11T18:12:51.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:51 smithi016 ceph-mon[43195]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.3 MiB/s wr, 472 op/s 2024-01-11T18:12:51.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:51 smithi038 ceph-mon[43582]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.3 MiB/s wr, 472 op/s 2024-01-11T18:12:51.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:51 smithi007 ceph-mon[33361]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.3 MiB/s wr, 472 op/s 2024-01-11T18:12:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.011+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2199979394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68107be0 msgr2=0x7f9b68108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.011+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2199979394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68107be0 0x7f9b68108000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b0d0 tx=0x7f9b5800b3e0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2199979394 shutdown_connections 2024-01-11T18:12:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2199979394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68109770 0x7f9b68110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2199979394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68107be0 0x7f9b68108000 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2199979394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b68108dd0 0x7f9b68109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2199979394 >> 172.21.15.7:0/2199979394 conn(0x7f9b68076b60 msgr2=0x7f9b68078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2199979394 shutdown_connections 2024-01-11T18:12:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.015+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2199979394 wait complete. 2024-01-11T18:12:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.016+0000 7f9b6d9ff700 1 Processor -- start 2024-01-11T18:12:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.017+0000 7f9b6d9ff700 1 -- start start 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.018+0000 7f9b6d9ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68107be0 0x7f9b6807ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.018+0000 7f9b6d9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b68108dd0 0x7f9b6807f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.018+0000 7f9b6d9ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68109770 0x7f9b681adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.018+0000 7f9b6d9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6807f950 con 0x7f9b68108dd0 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.018+0000 7f9b66ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68107be0 0x7f9b6807ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.018+0000 7f9b66ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68107be0 0x7f9b6807ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38502/0 (socket says 172.21.15.7:38502) 2024-01-11T18:12:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b66ffd700 1 -- 172.21.15.7:0/2678619574 learned_addr learned my addr 172.21.15.7:0/2678619574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68109770 0x7f9b681adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b681ae2e0 con 0x7f9b68109770 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b681ae420 con 0x7f9b68107be0 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b5e7fc700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b68108dd0 0x7f9b6807f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b677fe700 1 -- 172.21.15.7:0/2678619574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68107be0 msgr2=0x7f9b6807ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68107be0 0x7f9b6807ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b677fe700 1 -- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b68108dd0 msgr2=0x7f9b6807f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b68108dd0 0x7f9b6807f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.019+0000 7f9b677fe700 1 -- 172.21.15.7:0/2678619574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009520 con 0x7f9b68109770 2024-01-11T18:12:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.022+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68109770 0x7f9b681adda0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000e900 tx=0x7f9b6000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.023+0000 7f9b6c9fd700 1 -- 172.21.15.7:0/2678619574 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60004220 con 0x7f9b68109770 2024-01-11T18:12:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.023+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b58009d00 con 0x7f9b68109770 2024-01-11T18:12:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.023+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b681aea50 con 0x7f9b68109770 2024-01-11T18:12:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.023+0000 7f9b6c9fd700 1 -- 172.21.15.7:0/2678619574 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b60004380 con 0x7f9b68109770 2024-01-11T18:12:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.023+0000 7f9b6c9fd700 1 -- 172.21.15.7:0/2678619574 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60003d20 con 0x7f9b68109770 2024-01-11T18:12:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.029+0000 7f9b817a5b80 1 Processor -- start 2024-01-11T18:12:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.030+0000 7f9b817a5b80 1 -- start start 2024-01-11T18:12:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c6f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c70090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c76e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9b66030 con 0x555ac9c6fc10 2024-01-11T18:12:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9bc9ac0 con 0x555ac9c705d0 2024-01-11T18:12:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9bc73e0 con 0x555ac9c6ef20 2024-01-11T18:12:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b5e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c6f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b5e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c6f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38508/0 (socket says 172.21.15.7:38508) 2024-01-11T18:12:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.031+0000 7f9b5e7fc700 1 -- 172.21.15.7:0/3835204761 learned_addr learned my addr 172.21.15.7:0/3835204761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.032+0000 7f9b5e7fc700 1 -- 172.21.15.7:0/3835204761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 msgr2=0x555ac9c76e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.032+0000 7f9b5e7fc700 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c76e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.032+0000 7f9b5e7fc700 1 -- 172.21.15.7:0/3835204761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 msgr2=0x555ac9c70090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.032+0000 7f9b5e7fc700 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c70090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.032+0000 7f9b5e7fc700 1 -- 172.21.15.7:0/3835204761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ac9a37350 con 0x555ac9c6ef20 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.032+0000 7f9b5e7fc700 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c6f340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009b80 tx=0x7f9b50009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9804fae27dfcd5a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.033+0000 7f9b5dffb700 1 -- 172.21.15.7:0/3835204761 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50010ab0 con 0x555ac9c6ef20 2024-01-11T18:12:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.033+0000 7f9b5dffb700 1 -- 172.21.15.7:0/3835204761 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b50010c10 con 0x555ac9c6ef20 2024-01-11T18:12:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.033+0000 7f9b5dffb700 1 -- 172.21.15.7:0/3835204761 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500125c0 con 0x555ac9c6ef20 2024-01-11T18:12:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.037+0000 7f9b817a5b80 1 -- 172.21.15.7:0/3835204761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 msgr2=0x555ac9c6f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.037+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c6f340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009b80 tx=0x7f9b50009e90 comp rx=0 tx=0).stop 2024-01-11T18:12:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.045+0000 7f9b6c9fd700 1 -- 172.21.15.7:0/2678619574 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f9b6000c780 con 0x7f9b68109770 2024-01-11T18:12:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.046+0000 7f9b817a5b80 1 -- 172.21.15.7:0/3835204761 shutdown_connections 2024-01-11T18:12:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.046+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c6f340 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.046+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c76e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.046+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/3835204761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c70090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.046+0000 7f9b817a5b80 1 -- 172.21.15.7:0/3835204761 >> 172.21.15.7:0/3835204761 conn(0x555ac9be4740 msgr2=0x555ac9c69950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.047+0000 7f9b6c9fd700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9b540a2010 0x7f9b540a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.048+0000 7f9b6c9fd700 1 -- 172.21.15.7:0/2678619574 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9b60014070 con 0x7f9b68109770 2024-01-11T18:12:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.048+0000 7f9b817a5b80 1 -- 172.21.15.7:0/3835204761 shutdown_connections 2024-01-11T18:12:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.048+0000 7f9b817a5b80 1 -- 172.21.15.7:0/3835204761 wait complete. 2024-01-11T18:12:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.049+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9b540a2010 0x7f9b540a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.049+0000 7f9b817a5b80 1 -- start start 2024-01-11T18:12:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.049+0000 7f9b817a5b80 1 Processor -- start 2024-01-11T18:12:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.050+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.050+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c78ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.050+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c7feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9bc2cd0 con 0x555ac9c6fc10 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9b31340 con 0x555ac9c705d0 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9b30fe0 con 0x555ac9c6ef20 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c7feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c7feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57604/0 (socket says 172.21.15.7:57604) 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b677fe700 1 -- 172.21.15.7:0/2355594392 learned_addr learned my addr 172.21.15.7:0/2355594392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b817a5b80 10 client.0 _ll_register_callbacks cb 0x555ac9b636c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b5e7fc700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c78ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.051+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b677fe700 1 -- 172.21.15.7:0/2355594392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 msgr2=0x555ac9c78980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c78980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b677fe700 1 -- 172.21.15.7:0/2355594392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 msgr2=0x555ac9c78ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c78ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b677fe700 1 -- 172.21.15.7:0/2355594392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b50009830 con 0x555ac9c705d0 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b677fe700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c7feb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b6009a770 tx=0x7f9b60005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b12ecea62b6d57c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:12:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b46ffd700 1 -- 172.21.15.7:0/2355594392 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600850c0 con 0x555ac9c705d0 2024-01-11T18:12:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.052+0000 7f9b46ffd700 1 -- 172.21.15.7:0/2355594392 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9b600044f0 con 0x555ac9c705d0 2024-01-11T18:12:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.053+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ac9c61d60 con 0x555ac9c705d0 2024-01-11T18:12:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.054+0000 7f9b46ffd700 1 -- 172.21.15.7:0/2355594392 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b6009a7e0 con 0x555ac9c705d0 2024-01-11T18:12:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.054+0000 7f9b817a5b80 10 client.26624 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.054+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555ac99e3b00 con 0x555ac9c705d0 2024-01-11T18:12:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.054+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9b540a2010 0x7f9b540a44d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x555ac9c6b7f0 tx=0x7f9b58009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.054+0000 7f9b46ffd700 1 -- 172.21.15.7:0/2355594392 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ac9c61d60 con 0x555ac9c705d0 2024-01-11T18:12:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.055+0000 7f9b46ffd700 1 -- 172.21.15.7:0/2355594392 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b60099d40 con 0x555ac9c705d0 2024-01-11T18:12:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.055+0000 7f9b817a5b80 10 client.26624 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.055+0000 7f9b817a5b80 10 client.26624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.056+0000 7f9b817a5b80 20 client.26624 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.056+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9c84830 0x555ac9c86c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.056+0000 7f9b817a5b80 4 client.26624 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:12:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.056+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ac9bbbb60 con 0x555ac9c84830 2024-01-11T18:12:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.057+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9c84830 0x555ac9c86c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.061+0000 7f9b46ffd700 10 client.26624 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.061+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9c84830 0x555ac9c86c50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3947addcbcb116f0 server_cookie=a2092c1a083ac0ea in_seq=0 out_seq=0 2024-01-11T18:12:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.062+0000 7f9b46ffd700 1 -- 172.21.15.7:0/2355594392 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ac9bbbb60 con 0x555ac9c84830 2024-01-11T18:12:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.062+0000 7f9b46ffd700 10 client.26624 handle_command_reply: tid=0 2024-01-11T18:12:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.062+0000 7f9b817a5b80 1 client.26624 shutdown 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 msgr2=0x555ac9c7feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c7feb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b6009a770 tx=0x7f9b60005760 comp rx=0 tx=0).stop 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 shutdown_connections 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9c84830 0x555ac9c86c50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c78980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c7feb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2355594392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c78ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 >> 172.21.15.7:0/2355594392 conn(0x555ac9be4740 msgr2=0x555ac9c68f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.064+0000 7f9b46ffd700 0 client.26624 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.065+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 shutdown_connections 2024-01-11T18:12:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.065+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2355594392 wait complete. 2024-01-11T18:12:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.065+0000 7f9b817a5b80 20 client.26624 trim_cache size 0 max 16384 2024-01-11T18:12:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.099+0000 7f9b817a5b80 1 Processor -- start 2024-01-11T18:12:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.099+0000 7f9b817a5b80 1 -- start start 2024-01-11T18:12:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.099+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c86c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c73570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9a39640 con 0x555ac9c6fc10 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9c774a0 con 0x555ac9c705d0 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9c79ae0 con 0x555ac9c6ef20 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c86c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c86c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:38526/0 (socket says 172.21.15.7:38526) 2024-01-11T18:12:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 -- 172.21.15.7:0/2135332047 learned_addr learned my addr 172.21.15.7:0/2135332047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 -- 172.21.15.7:0/2135332047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 msgr2=0x555ac9c73570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c73570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 -- 172.21.15.7:0/2135332047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 msgr2=0x555ac9c7fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 -- 172.21.15.7:0/2135332047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ac9b841d0 con 0x555ac9c6ef20 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.100+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c86c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9b580060b0 tx=0x7f9b58021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1cda0b51eb0dd7c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.101+0000 7f9b5d7fa700 1 -- 172.21.15.7:0/2135332047 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800e040 con 0x555ac9c6ef20 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.101+0000 7f9b5d7fa700 1 -- 172.21.15.7:0/2135332047 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b5802ee90 con 0x555ac9c6ef20 2024-01-11T18:12:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.101+0000 7f9b5d7fa700 1 -- 172.21.15.7:0/2135332047 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5802a6a0 con 0x555ac9c6ef20 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2135332047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 msgr2=0x555ac9c86c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c86c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9b580060b0 tx=0x7f9b58021000 comp rx=0 tx=0).stop 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2135332047 shutdown_connections 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c6ef20 0x555ac9c86c20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c705d0 0x555ac9c73570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/2135332047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2135332047 >> 172.21.15.7:0/2135332047 conn(0x555ac9be4740 msgr2=0x555ac9c77bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2135332047 shutdown_connections 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 -- 172.21.15.7:0/2135332047 wait complete. 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.102+0000 7f9b817a5b80 1 -- start start 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 Processor -- start 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c6ef20 0x555ac9c7d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c705d0 0x555ac9c7ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9c7f320 con 0x555ac9c6fc10 2024-01-11T18:12:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9c7f490 con 0x555ac9c6ef20 2024-01-11T18:12:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ac9c7f650 con 0x555ac9c705d0 2024-01-11T18:12:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.103+0000 7f9b817a5b80 10 client.0 _ll_register_callbacks cb 0x555ac9b636c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:12:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b5e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b5e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37794/0 (socket says 172.21.15.7:37794) 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b5e7fc700 1 -- 172.21.15.7:0/166596596 learned_addr learned my addr 172.21.15.7:0/166596596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b66ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c6ef20 0x555ac9c7d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b677fe700 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c705d0 0x555ac9c7ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b677fe700 1 -- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c6ef20 msgr2=0x555ac9c7d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b677fe700 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c6ef20 0x555ac9c7d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b677fe700 1 -- 172.21.15.7:0/166596596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 msgr2=0x555ac9c7dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b677fe700 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.104+0000 7f9b677fe700 1 -- 172.21.15.7:0/166596596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b68110970 con 0x555ac9c705d0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b677fe700 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c705d0 0x555ac9c7ede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9b6009a4e0 tx=0x7f9b60098d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6b04088601395c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b46ffd700 1 -- 172.21.15.7:0/166596596 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6009bd10 con 0x555ac9c705d0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b50010c10 con 0x555ac9c705d0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b46ffd700 1 -- 172.21.15.7:0/166596596 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9b600044f0 con 0x555ac9c705d0 2024-01-11T18:12:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b46ffd700 1 -- 172.21.15.7:0/166596596 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b6009be70 con 0x555ac9c705d0 2024-01-11T18:12:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b817a5b80 10 client.26662 fetch_fsmap learned FSMap version 47 2024-01-11T18:12:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.105+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555ac9a3bc70 con 0x555ac9c705d0 2024-01-11T18:12:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.106+0000 7f9b46ffd700 1 -- 172.21.15.7:0/166596596 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b50010c10 con 0x555ac9c705d0 2024-01-11T18:12:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.106+0000 7f9b46ffd700 1 -- 172.21.15.7:0/166596596 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b60099740 con 0x555ac9c705d0 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b817a5b80 10 client.26662 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b817a5b80 10 client.26662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b817a5b80 20 client.26662 populate_metadata read hostname 'smithi007' 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9cdb620 0x555ac9cdda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b817a5b80 4 client.26662 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ac9bbb880 con 0x555ac9cdb620 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.107+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9cdb620 0x555ac9cdda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.108+0000 7f9b46ffd700 10 client.26662 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.108+0000 7f9b66ffd700 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9cdb620 0x555ac9cdda40 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d550c9ba02245e server_cookie=1743a87c58c6c301 in_seq=0 out_seq=0 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b46ffd700 1 -- 172.21.15.7:0/166596596 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ac9bbb880 con 0x555ac9cdb620 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b46ffd700 10 client.26662 handle_command_reply: tid=0 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 client.26662 shutdown 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c705d0 msgr2=0x555ac9c7ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c705d0 0x555ac9c7ede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9b6009a4e0 tx=0x7f9b60098d90 comp rx=0 tx=0).stop 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 shutdown_connections 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x555ac9cdb620 0x555ac9cdda40 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ac9c705d0 0x555ac9c7ede0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ac9c6ef20 0x555ac9c7d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b46ffd700 0 client.26662 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:12:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 --2- 172.21.15.7:0/166596596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ac9c6fc10 0x555ac9c7dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.125+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 >> 172.21.15.7:0/166596596 conn(0x555ac9be4740 msgr2=0x555ac9c73570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.126+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 shutdown_connections 2024-01-11T18:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.126+0000 7f9b817a5b80 1 -- 172.21.15.7:0/166596596 wait complete. 2024-01-11T18:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.126+0000 7f9b817a5b80 20 client.26662 trim_cache size 0 max 16384 2024-01-11T18:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:12:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.126+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9b540a2010 msgr2=0x7f9b540a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9b540a2010 0x7f9b540a44d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x555ac9c6b7f0 tx=0x7f9b58009590 comp rx=0 tx=0).stop 2024-01-11T18:12:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68109770 msgr2=0x7f9b681adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:12:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68109770 0x7f9b681adda0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000e900 tx=0x7f9b6000ec10 comp rx=0 tx=0).stop 2024-01-11T18:12:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 shutdown_connections 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f9b540a2010 0x7f9b540a44d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68107be0 0x7f9b6807ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b68109770 0x7f9b681adda0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 --2- 172.21.15.7:0/2678619574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b68108dd0 0x7f9b6807f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 >> 172.21.15.7:0/2678619574 conn(0x7f9b68076b60 msgr2=0x7f9b6810c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 shutdown_connections 2024-01-11T18:12:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:12:53.127+0000 7f9b6d9ff700 1 -- 172.21.15.7:0/2678619574 wait complete. 2024-01-11T18:12:53.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:53 smithi016 ceph-mon[43195]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-01-11T18:12:53.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:53 smithi038 ceph-mon[43582]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-01-11T18:12:53.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:53 smithi007 ceph-mon[33361]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-01-11T18:12:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 865.0 s, 472.6 tps, lat 214.254 ms stddev 259.661 2024-01-11T18:12:54.772 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:54 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:12:54.811 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:54 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:12:54.988 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:54 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:12:56.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:55 smithi016 ceph-mon[43195]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.7 MiB/s wr, 400 op/s 2024-01-11T18:12:56.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:55 smithi038 ceph-mon[43582]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.7 MiB/s wr, 400 op/s 2024-01-11T18:12:56.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:55 smithi007 ceph-mon[33361]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.7 MiB/s wr, 400 op/s 2024-01-11T18:12:57.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:12:57.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:12:57.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:12:57.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:12:57.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:12:57.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:12:57.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:12:57.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:12:57.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:12:58.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:58 smithi007 ceph-mon[33361]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 476 op/s 2024-01-11T18:12:58.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:58 smithi016 ceph-mon[43195]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 476 op/s 2024-01-11T18:12:58.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:58 smithi038 ceph-mon[43582]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 476 op/s 2024-01-11T18:12:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 870.0 s, 456.2 tps, lat 215.517 ms stddev 249.142 2024-01-11T18:12:59.490 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:12:59 smithi016 ceph-mon[43195]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.5 MiB/s wr, 426 op/s 2024-01-11T18:12:59.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:12:59 smithi038 ceph-mon[43582]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.5 MiB/s wr, 426 op/s 2024-01-11T18:12:59.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:12:59 smithi007 ceph-mon[33361]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.5 MiB/s wr, 426 op/s 2024-01-11T18:13:01.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:01 smithi016 ceph-mon[43195]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.1 MiB/s wr, 446 op/s 2024-01-11T18:13:01.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:01 smithi038 ceph-mon[43582]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.1 MiB/s wr, 446 op/s 2024-01-11T18:13:01.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:01 smithi007 ceph-mon[33361]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.1 MiB/s wr, 446 op/s 2024-01-11T18:13:03.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:03 smithi016 ceph-mon[43195]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.9 MiB/s wr, 456 op/s 2024-01-11T18:13:03.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:03 smithi038 ceph-mon[43582]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.9 MiB/s wr, 456 op/s 2024-01-11T18:13:03.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:03 smithi007 ceph-mon[33361]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.9 MiB/s wr, 456 op/s 2024-01-11T18:13:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 875.0 s, 501.0 tps, lat 195.810 ms stddev 266.778 2024-01-11T18:13:06.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:05 smithi016 ceph-mon[43195]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-01-11T18:13:06.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:05 smithi038 ceph-mon[43582]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-01-11T18:13:06.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:05 smithi007 ceph-mon[33361]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.6 MiB/s wr, 403 op/s 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 -- 172.21.15.7:0/3127966638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0105350 msgr2=0x7f83b010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 --2- 172.21.15.7:0/3127966638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0105350 0x7f83b010bc00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f83a00099c0 tx=0x7f83a0009cd0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 -- 172.21.15.7:0/3127966638 shutdown_connections 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 --2- 172.21.15.7:0/3127966638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b01037c0 0x7f83b0103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 --2- 172.21.15.7:0/3127966638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0105350 0x7f83b010bc00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 --2- 172.21.15.7:0/3127966638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b01049b0 0x7f83b0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 -- 172.21.15.7:0/3127966638 >> 172.21.15.7:0/3127966638 conn(0x7f83b00fed30 msgr2=0x7f83b0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 -- 172.21.15.7:0/3127966638 shutdown_connections 2024-01-11T18:13:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.407+0000 7f83b55ff700 1 -- 172.21.15.7:0/3127966638 wait complete. 2024-01-11T18:13:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.409+0000 7f83b55ff700 1 Processor -- start 2024-01-11T18:13:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.409+0000 7f83b55ff700 1 -- start start 2024-01-11T18:13:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.410+0000 7f83b55ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01037c0 0x7f83b0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.410+0000 7f83b55ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b01049b0 0x7f83b0199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83b55ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 0x7f83b01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83b55ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a11a0 con 0x7f83b01037c0 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83b55ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a12e0 con 0x7f83b01049b0 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83b55ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a1470 con 0x7f83b0105350 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01037c0 0x7f83b0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83af7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 0x7f83b01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83af7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 0x7f83b01a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45980/0 (socket says 172.21.15.7:45980) 2024-01-11T18:13:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83af7fe700 1 -- 172.21.15.7:0/2264497669 learned_addr learned my addr 172.21.15.7:0/2264497669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.411+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b01049b0 0x7f83b0199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.412+0000 7f83af7fe700 1 -- 172.21.15.7:0/2264497669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b01049b0 msgr2=0x7f83b0199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.412+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b01049b0 0x7f83b0199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.412+0000 7f83af7fe700 1 -- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01037c0 msgr2=0x7f83b0199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.412+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01037c0 0x7f83b0199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.412+0000 7f83af7fe700 1 -- 172.21.15.7:0/2264497669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8398009520 con 0x7f83b0105350 2024-01-11T18:13:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.412+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 0x7f83b01a0bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83a00098e0 tx=0x7f83a000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.413+0000 7f83a7fff700 1 -- 172.21.15.7:0/2264497669 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a001d070 con 0x7f83b0105350 2024-01-11T18:13:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.413+0000 7f83a7fff700 1 -- 172.21.15.7:0/2264497669 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a0005030 con 0x7f83b0105350 2024-01-11T18:13:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.413+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a00095f0 con 0x7f83b0105350 2024-01-11T18:13:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.413+0000 7f83a7fff700 1 -- 172.21.15.7:0/2264497669 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0004b30 con 0x7f83b0105350 2024-01-11T18:13:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.413+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01a1920 con 0x7f83b0105350 2024-01-11T18:13:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.418+0000 7f83a7fff700 1 -- 172.21.15.7:0/2264497669 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f83a0004c90 con 0x7f83b0105350 2024-01-11T18:13:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.421+0000 7f83a7fff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f839c0a1f10 0x7f839c0a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.421+0000 7f83a7fff700 1 -- 172.21.15.7:0/2264497669 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f83a00ce7c0 con 0x7f83b0105350 2024-01-11T18:13:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.422+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f839c0a1f10 0x7f839c0a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.423+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f839c0a1f10 0x7f839c0a43d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f83b0104810 tx=0x7f83a800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.425+0000 7f83c93b0b80 1 Processor -- start 2024-01-11T18:13:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.425+0000 7f83c93b0b80 1 -- start start 2024-01-11T18:13:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4dae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d3ca030 con 0x559d4d4d45d0 2024-01-11T18:13:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d42dac0 con 0x559d4d4d3c10 2024-01-11T18:13:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d42b3e0 con 0x559d4d4d2f20 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45984/0 (socket says 172.21.15.7:45984) 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83aeffd700 1 -- 172.21.15.7:0/2043951562 learned_addr learned my addr 172.21.15.7:0/2043951562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.426+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4dae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.427+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.427+0000 7f83aeffd700 1 -- 172.21.15.7:0/2043951562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 msgr2=0x559d4d4d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.427+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.427+0000 7f83aeffd700 1 -- 172.21.15.7:0/2043951562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 msgr2=0x559d4d4dae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.427+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4dae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.427+0000 7f83aeffd700 1 -- 172.21.15.7:0/2043951562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d4d29b350 con 0x559d4d4d2f20 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.428+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4dae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:13:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.428+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:13:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.428+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4d3340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009550 tx=0x7f83a8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918d7ac378b9e7db server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.429+0000 7f83a57fa700 1 -- 172.21.15.7:0/2043951562 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8025070 con 0x559d4d4d2f20 2024-01-11T18:13:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.429+0000 7f83a57fa700 1 -- 172.21.15.7:0/2043951562 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a801a070 con 0x559d4d4d2f20 2024-01-11T18:13:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.429+0000 7f83a57fa700 1 -- 172.21.15.7:0/2043951562 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800e050 con 0x559d4d4d2f20 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2043951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 msgr2=0x559d4d4d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4d3340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009550 tx=0x7f83a8013000 comp rx=0 tx=0).stop 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2043951562 shutdown_connections 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4d3340 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4dae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2043951562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2043951562 >> 172.21.15.7:0/2043951562 conn(0x559d4d448740 msgr2=0x559d4d4cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.430+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2043951562 shutdown_connections 2024-01-11T18:13:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.431+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2043951562 wait complete. 2024-01-11T18:13:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.431+0000 7f83c93b0b80 1 -- start start 2024-01-11T18:13:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.434+0000 7f83c93b0b80 1 Processor -- start 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.435+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.435+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 0x559d4d4e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.435+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.435+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d426cd0 con 0x559d4d4d2f20 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d395340 con 0x559d4d4d45d0 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d394fe0 con 0x559d4d4d3c10 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83c93b0b80 10 client.0 _ll_register_callbacks cb 0x559d4d3c76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 0x559d4d4e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83af7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 0x559d4d4e5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45986/0 (socket says 172.21.15.7:45986) 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.436+0000 7f83ae7fc700 1 -- 172.21.15.7:0/2195629168 learned_addr learned my addr 172.21.15.7:0/2195629168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83af7fe700 1 -- 172.21.15.7:0/2195629168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 msgr2=0x559d4d4e5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 0x559d4d4e5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83af7fe700 1 -- 172.21.15.7:0/2195629168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 msgr2=0x559d4d4e51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83af7fe700 1 -- 172.21.15.7:0/2195629168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01a1920 con 0x559d4d4d45d0 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 0x559d4d4e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f83a00cc300 tx=0x7f83a0005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7172f8b82083c9cd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.437+0000 7f83967fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.438+0000 7f83967fc700 1 -- 172.21.15.7:0/2195629168 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00a0160 con 0x559d4d4d45d0 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.438+0000 7f83967fc700 1 -- 172.21.15.7:0/2195629168 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f83a00a0350 con 0x559d4d4d45d0 2024-01-11T18:13:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.438+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d4d4c5d60 con 0x559d4d4d45d0 2024-01-11T18:13:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.440+0000 7f83967fc700 1 -- 172.21.15.7:0/2195629168 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a00a4500 con 0x559d4d4d45d0 2024-01-11T18:13:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.440+0000 7f83c93b0b80 10 client.26639 fetch_fsmap learned FSMap version 47 2024-01-11T18:13:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.440+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f83a8007de0 con 0x559d4d4d45d0 2024-01-11T18:13:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.440+0000 7f83967fc700 1 -- 172.21.15.7:0/2195629168 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d4d4c5d60 con 0x559d4d4d45d0 2024-01-11T18:13:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.443+0000 7f83967fc700 1 -- 172.21.15.7:0/2195629168 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f83a009b6f0 con 0x559d4d4d45d0 2024-01-11T18:13:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.444+0000 7f83c93b0b80 10 client.26639 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:13:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.444+0000 7f83c93b0b80 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:13:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.444+0000 7f83c93b0b80 20 client.26639 populate_metadata read hostname 'smithi007' 2024-01-11T18:13:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.444+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4f0c80 0x559d4d4f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.444+0000 7f83c93b0b80 4 client.26639 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:13:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.444+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d4d41fb60 con 0x559d4d4f0c80 2024-01-11T18:13:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.451+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4f0c80 0x559d4d4f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.452+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4f0c80 0x559d4d4f3070 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f63f4c16123f0f15 server_cookie=d07cee71b22467a7 in_seq=0 out_seq=0 2024-01-11T18:13:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.452+0000 7f83967fc700 10 client.26639 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.453+0000 7f83967fc700 1 -- 172.21.15.7:0/2195629168 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d4d41fb60 con 0x559d4d4f0c80 2024-01-11T18:13:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.453+0000 7f83967fc700 10 client.26639 handle_command_reply: tid=0 2024-01-11T18:13:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.453+0000 7f83c93b0b80 1 client.26639 shutdown 2024-01-11T18:13:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d45d0 msgr2=0x559d4d4ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f83a00cc300 tx=0x7f83a0005760 comp rx=0 tx=0).stop 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 shutdown_connections 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4f0c80 0x559d4d4f3070 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/2195629168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d3c10 0x559d4d4e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 >> 172.21.15.7:0/2195629168 conn(0x559d4d448740 msgr2=0x559d4d4ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.454+0000 7f83967fc700 0 client.26639 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.456+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 shutdown_connections 2024-01-11T18:13:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.456+0000 7f83c93b0b80 1 -- 172.21.15.7:0/2195629168 wait complete. 2024-01-11T18:13:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.457+0000 7f83c93b0b80 20 client.26639 trim_cache size 0 max 16384 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.494+0000 7f83c93b0b80 1 Processor -- start 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.494+0000 7f83c93b0b80 1 -- start start 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d29d640 con 0x559d4d4d2f20 2024-01-11T18:13:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d4ed880 con 0x559d4d4d3c10 2024-01-11T18:13:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d4f0ad0 con 0x559d4d4d45d0 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50232/0 (socket says 172.21.15.7:50232) 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83ae7fc700 1 -- 172.21.15.7:0/3426941938 learned_addr learned my addr 172.21.15.7:0/3426941938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83af7fe700 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.495+0000 7f83aeffd700 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.496+0000 7f83af7fe700 1 -- 172.21.15.7:0/3426941938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 msgr2=0x559d4d4d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.496+0000 7f83af7fe700 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.496+0000 7f83af7fe700 1 -- 172.21.15.7:0/3426941938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 msgr2=0x559d4d4c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.496+0000 7f83af7fe700 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.496+0000 7f83af7fe700 1 -- 172.21.15.7:0/3426941938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d4d247b00 con 0x559d4d4d45d0 2024-01-11T18:13:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.496+0000 7f83af7fe700 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec6c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f83a009a550 tx=0x7f83a009a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7bfc704206f821 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.502+0000 7f83a4ff9700 1 -- 172.21.15.7:0/3426941938 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00a0160 con 0x559d4d4d45d0 2024-01-11T18:13:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.502+0000 7f83a4ff9700 1 -- 172.21.15.7:0/3426941938 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a00a4070 con 0x559d4d4d45d0 2024-01-11T18:13:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.502+0000 7f83a4ff9700 1 -- 172.21.15.7:0/3426941938 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00ba290 con 0x559d4d4d45d0 2024-01-11T18:13:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 -- 172.21.15.7:0/3426941938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d45d0 msgr2=0x559d4d4ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec6c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f83a009a550 tx=0x7f83a009a7b0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 -- 172.21.15.7:0/3426941938 shutdown_connections 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d45d0 0x559d4d4ec6c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d2f20 0x559d4d4c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/3426941938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4d3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 -- 172.21.15.7:0/3426941938 >> 172.21.15.7:0/3426941938 conn(0x559d4d448740 msgr2=0x559d4d4dac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 -- 172.21.15.7:0/3426941938 shutdown_connections 2024-01-11T18:13:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.503+0000 7f83c93b0b80 1 -- 172.21.15.7:0/3426941938 wait complete. 2024-01-11T18:13:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.504+0000 7f83c93b0b80 1 -- start start 2024-01-11T18:13:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.504+0000 7f83c93b0b80 1 Processor -- start 2024-01-11T18:13:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.505+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.505+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.505+0000 7f83c93b0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.505+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.505+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:45998/0 (socket says 172.21.15.7:45998) 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.506+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.506+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4e9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50240/0 (socket says 172.21.15.7:50240) 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.506+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d4eb310 con 0x559d4d4d45d0 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.506+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d4eb480 con 0x559d4d4d3c10 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.506+0000 7f83c93b0b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d4eb640 con 0x559d4d4d2f20 2024-01-11T18:13:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.506+0000 7f83c93b0b80 10 client.0 _ll_register_callbacks cb 0x559d4d3c76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83aeffd700 1 -- 172.21.15.7:0/1492395775 learned_addr learned my addr 172.21.15.7:0/1492395775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83af7fe700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83ae7fc700 1 -- 172.21.15.7:0/1492395775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 msgr2=0x559d4d4e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83ae7fc700 1 -- 172.21.15.7:0/1492395775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 msgr2=0x559d4d4eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.507+0000 7f83ae7fc700 1 -- 172.21.15.7:0/1492395775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a8007de0 con 0x559d4d4d3c10 2024-01-11T18:13:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.508+0000 7f83af7fe700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:13:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.508+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4e9dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8398021800 tx=0x7f8398021b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a440d49ca936b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.508+0000 7f83967fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:13:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.512+0000 7f83967fc700 1 -- 172.21.15.7:0/1492395775 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398004520 con 0x559d4d4d3c10 2024-01-11T18:13:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.512+0000 7f83967fc700 1 -- 172.21.15.7:0/1492395775 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8398004830 con 0x559d4d4d3c10 2024-01-11T18:13:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.512+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83a801a070 con 0x559d4d4d3c10 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.514+0000 7f83967fc700 1 -- 172.21.15.7:0/1492395775 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83980049f0 con 0x559d4d4d3c10 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 10 client.26645 fetch_fsmap learned FSMap version 47 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559d4d3e81d0 con 0x559d4d4d3c10 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83967fc700 1 -- 172.21.15.7:0/1492395775 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a801a070 con 0x559d4d4d3c10 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83967fc700 1 -- 172.21.15.7:0/1492395775 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8398003680 con 0x559d4d4d3c10 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 10 client.26645 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 10 client.26645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 20 client.26645 populate_metadata read hostname 'smithi007' 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4e1b70 0x559d4d4e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 4 client.26645 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:13:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.515+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d4d41f880 con 0x559d4d4e1b70 2024-01-11T18:13:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.519+0000 7f83aeffd700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4e1b70 0x559d4d4e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.521+0000 7f83aeffd700 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4e1b70 0x559d4d4e1f80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d2e8f7fa319e2c server_cookie=ee5c1f6b8ab8de35 in_seq=0 out_seq=0 2024-01-11T18:13:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.521+0000 7f83967fc700 10 client.26645 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.564+0000 7f83967fc700 1 -- 172.21.15.7:0/1492395775 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d4d41f880 con 0x559d4d4e1b70 2024-01-11T18:13:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.564+0000 7f83967fc700 10 client.26645 handle_command_reply: tid=0 2024-01-11T18:13:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.564+0000 7f83c93b0b80 1 client.26645 shutdown 2024-01-11T18:13:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.564+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 msgr2=0x559d4d4e9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.564+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4e9dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8398021800 tx=0x7f8398021b10 comp rx=0 tx=0).stop 2024-01-11T18:13:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 shutdown_connections 2024-01-11T18:13:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x559d4d4e1b70 0x559d4d4e1f80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559d4d4d2f20 0x559d4d4e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d4d4d45d0 0x559d4d4eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 --2- 172.21.15.7:0/1492395775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d4d4d3c10 0x559d4d4e9dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 >> 172.21.15.7:0/1492395775 conn(0x559d4d448740 msgr2=0x559d4d4d7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83967fc700 0 client.26645 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 shutdown_connections 2024-01-11T18:13:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 1 -- 172.21.15.7:0/1492395775 wait complete. 2024-01-11T18:13:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.565+0000 7f83c93b0b80 20 client.26645 trim_cache size 0 max 16384 2024-01-11T18:13:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:13:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:13:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:13:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:13:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.567+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f839c0a1f10 msgr2=0x7f839c0a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.567+0000 7f83b55ff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f839c0a1f10 0x7f839c0a43d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f83b0104810 tx=0x7f83a800b040 comp rx=0 tx=0).stop 2024-01-11T18:13:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.567+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 msgr2=0x7f83b01a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.567+0000 7f83b55ff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 0x7f83b01a0bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83a00098e0 tx=0x7f83a000f690 comp rx=0 tx=0).stop 2024-01-11T18:13:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 shutdown_connections 2024-01-11T18:13:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f839c0a1f10 0x7f839c0a43d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83b0105350 0x7f83b01a0bd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01037c0 0x7f83b0199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 --2- 172.21.15.7:0/2264497669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b01049b0 0x7f83b0199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 >> 172.21.15.7:0/2264497669 conn(0x7f83b00fed30 msgr2=0x7f83b0107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 shutdown_connections 2024-01-11T18:13:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:06.569+0000 7f83b55ff700 1 -- 172.21.15.7:0/2264497669 wait complete. 2024-01-11T18:13:07.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:07 smithi016 ceph-mon[43195]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-01-11T18:13:07.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:07 smithi038 ceph-mon[43582]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-01-11T18:13:07.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:07 smithi007 ceph-mon[33361]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-01-11T18:13:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 880.0 s, 440.0 tps, lat 240.014 ms stddev 268.055 2024-01-11T18:13:09.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:09 smithi016 ceph-mon[43195]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-01-11T18:13:09.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:09 smithi038 ceph-mon[43582]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-01-11T18:13:09.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:09 smithi007 ceph-mon[33361]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.0 MiB/s wr, 435 op/s 2024-01-11T18:13:11.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:11 smithi016 ceph-mon[43195]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.6 MiB/s wr, 473 op/s 2024-01-11T18:13:11.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:11 smithi038 ceph-mon[43582]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.6 MiB/s wr, 473 op/s 2024-01-11T18:13:11.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:11 smithi007 ceph-mon[33361]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.6 MiB/s wr, 473 op/s 2024-01-11T18:13:13.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:13 smithi016 ceph-mon[43195]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 457 op/s 2024-01-11T18:13:13.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:13 smithi038 ceph-mon[43582]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 457 op/s 2024-01-11T18:13:13.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:13 smithi007 ceph-mon[33361]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 457 op/s 2024-01-11T18:13:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 885.0 s, 472.4 tps, lat 201.078 ms stddev 194.670 2024-01-11T18:13:16.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:16 smithi038 ceph-mon[43582]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.4 MiB/s wr, 410 op/s 2024-01-11T18:13:16.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:16 smithi007 ceph-mon[33361]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.4 MiB/s wr, 410 op/s 2024-01-11T18:13:16.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:16 smithi016 ceph-mon[43195]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.4 MiB/s wr, 410 op/s 2024-01-11T18:13:17.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:17 smithi016 ceph-mon[43195]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-01-11T18:13:17.518 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:17 smithi038 ceph-mon[43582]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-01-11T18:13:17.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:17 smithi007 ceph-mon[33361]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.2 MiB/s wr, 500 op/s 2024-01-11T18:13:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 890.0 s, 449.0 tps, lat 225.646 ms stddev 271.252 2024-01-11T18:13:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:19 smithi016 ceph-mon[43195]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.0 MiB/s wr, 434 op/s 2024-01-11T18:13:19.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:19 smithi038 ceph-mon[43582]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.0 MiB/s wr, 434 op/s 2024-01-11T18:13:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:19 smithi007 ceph-mon[33361]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.0 MiB/s wr, 434 op/s 2024-01-11T18:13:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.833+0000 7fe8401ff700 1 -- 172.21.15.7:0/3070052634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 msgr2=0x7fe8381019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.833+0000 7fe8401ff700 1 --2- 172.21.15.7:0/3070052634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 0x7fe8381019d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009920 tx=0x7fe82c009c30 comp rx=0 tx=0).stop 2024-01-11T18:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.835+0000 7fe8401ff700 1 -- 172.21.15.7:0/3070052634 shutdown_connections 2024-01-11T18:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.835+0000 7fe8401ff700 1 --2- 172.21.15.7:0/3070052634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838103140 0x7fe8381099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.835+0000 7fe8401ff700 1 --2- 172.21.15.7:0/3070052634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 0x7fe8381019d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.835+0000 7fe8401ff700 1 --2- 172.21.15.7:0/3070052634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8381027a0 0x7fe838102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.835+0000 7fe8401ff700 1 -- 172.21.15.7:0/3070052634 >> 172.21.15.7:0/3070052634 conn(0x7fe8380fcb00 msgr2=0x7fe8380fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.835+0000 7fe8401ff700 1 -- 172.21.15.7:0/3070052634 shutdown_connections 2024-01-11T18:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.836+0000 7fe8401ff700 1 -- 172.21.15.7:0/3070052634 wait complete. 2024-01-11T18:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.837+0000 7fe8401ff700 1 Processor -- start 2024-01-11T18:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.838+0000 7fe8401ff700 1 -- start start 2024-01-11T18:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.838+0000 7fe8401ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 0x7fe83806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.838+0000 7fe8401ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8381027a0 0x7fe838070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.839+0000 7fe8401ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 0x7fe8381a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.839+0000 7fe83e79c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 0x7fe8381a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.839+0000 7fe83e79c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 0x7fe8381a76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48532/0 (socket says 172.21.15.7:48532) 2024-01-11T18:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.839+0000 7fe83e79c700 1 -- 172.21.15.7:0/2867321430 learned_addr learned my addr 172.21.15.7:0/2867321430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.839+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838070a60 con 0x7fe8381015b0 2024-01-11T18:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.840+0000 7fe83df9b700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 0x7fe83806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.840+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838070bd0 con 0x7fe838103140 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.840+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8381a7c00 con 0x7fe8381027a0 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83d79a700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8381027a0 0x7fe838070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83e79c700 1 -- 172.21.15.7:0/2867321430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8381027a0 msgr2=0x7fe838070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83e79c700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8381027a0 0x7fe838070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83e79c700 1 -- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 msgr2=0x7fe83806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83e79c700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 0x7fe83806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83e79c700 1 -- 172.21.15.7:0/2867321430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c009520 con 0x7fe838103140 2024-01-11T18:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.841+0000 7fe83e79c700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 0x7fe8381a76c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c110 tx=0x7fe83000c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.842+0000 7fe82affd700 1 -- 172.21.15.7:0/2867321430 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830017070 con 0x7fe838103140 2024-01-11T18:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.842+0000 7fe82affd700 1 -- 172.21.15.7:0/2867321430 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe83000e050 con 0x7fe838103140 2024-01-11T18:13:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.842+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8381a7e80 con 0x7fe838103140 2024-01-11T18:13:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.842+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8381a8240 con 0x7fe838103140 2024-01-11T18:13:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.842+0000 7fe82affd700 1 -- 172.21.15.7:0/2867321430 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830007940 con 0x7fe838103140 2024-01-11T18:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.853+0000 7fe853f9bb80 1 Processor -- start 2024-01-11T18:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.853+0000 7fe82affd700 1 -- 172.21.15.7:0/2867321430 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fe8300092e0 con 0x7fe838103140 2024-01-11T18:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.853+0000 7fe853f9bb80 1 -- start start 2024-01-11T18:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.854+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce99f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.854+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99fc10 0x55bfce9a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.854+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.854+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce896030 con 0x55bfce99fc10 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.854+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce8f9ac0 con 0x55bfce99ef20 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.854+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce8f73e0 con 0x55bfce9a05d0 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.855+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.855+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9a6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44052/0 (socket says 172.21.15.7:44052) 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.855+0000 7fe83d79a700 1 -- 172.21.15.7:0/3455996266 learned_addr learned my addr 172.21.15.7:0/3455996266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.856+0000 7fe82affd700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe8240a1fd0 0x7fe8240a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.856+0000 7fe82affd700 1 -- 172.21.15.7:0/2867321430 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe8300cd6c0 con 0x7fe838103140 2024-01-11T18:13:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.857+0000 7fe83e79c700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe8240a1fd0 0x7fe8240a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99fc10 0x55bfce9a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83e79c700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe8240a1fd0 0x7fe8240a4490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe830096590 tx=0x7fe830009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83df9b700 1 -- 172.21.15.7:0/3455996266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 msgr2=0x55bfce9a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83df9b700 1 -- 172.21.15.7:0/3455996266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 msgr2=0x55bfce99f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce99f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.858+0000 7fe83df9b700 1 -- 172.21.15.7:0/3455996266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfce767350 con 0x55bfce99fc10 2024-01-11T18:13:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.859+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99fc10 0x55bfce9a0090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009230 tx=0x7fe82c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55e7fe388c5cd9d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.860+0000 7fe823fff700 1 -- 172.21.15.7:0/3455996266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c01d040 con 0x55bfce99fc10 2024-01-11T18:13:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.860+0000 7fe823fff700 1 -- 172.21.15.7:0/3455996266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe82c00b560 con 0x55bfce99fc10 2024-01-11T18:13:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.860+0000 7fe823fff700 1 -- 172.21.15.7:0/3455996266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0053c0 con 0x55bfce99fc10 2024-01-11T18:13:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.861+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3455996266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99fc10 msgr2=0x55bfce9a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.861+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99fc10 0x55bfce9a0090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009230 tx=0x7fe82c005ce0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3455996266 shutdown_connections 2024-01-11T18:13:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce99f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3455996266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99fc10 0x55bfce9a0090 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3455996266 >> 172.21.15.7:0/3455996266 conn(0x55bfce914740 msgr2=0x55bfce999950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3455996266 shutdown_connections 2024-01-11T18:13:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.862+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3455996266 wait complete. 2024-01-11T18:13:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.863+0000 7fe853f9bb80 1 -- start start 2024-01-11T18:13:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.863+0000 7fe853f9bb80 1 Processor -- start 2024-01-11T18:13:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.867+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.867+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.867+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce9a05d0 0x55bfce9afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.867+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce8f2cd0 con 0x55bfce99ef20 2024-01-11T18:13:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce861340 con 0x55bfce9a05d0 2024-01-11T18:13:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce860fe0 con 0x55bfce99fc10 2024-01-11T18:13:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe853f9bb80 10 client.0 _ll_register_callbacks cb 0x55bfce8936c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:13:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe83df9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35356/0 (socket says 172.21.15.7:35356) 2024-01-11T18:13:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.868+0000 7fe83d79a700 1 -- 172.21.15.7:0/3359847944 learned_addr learned my addr 172.21.15.7:0/3359847944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.869+0000 7fe83df9b700 1 -- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce9a05d0 msgr2=0x55bfce9afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.869+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce9a05d0 0x55bfce9afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.869+0000 7fe83df9b700 1 -- 172.21.15.7:0/3359847944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 msgr2=0x55bfce9a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.869+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.869+0000 7fe83df9b700 1 -- 172.21.15.7:0/3359847944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83810a340 con 0x55bfce99fc10 2024-01-11T18:13:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.875+0000 7fe83df9b700 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9a8db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009920 tx=0x7fe82c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46eb03bd42da1af9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.876+0000 7fe8217fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:13:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.876+0000 7fe8217fa700 1 -- 172.21.15.7:0/3359847944 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c01d040 con 0x55bfce99fc10 2024-01-11T18:13:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.876+0000 7fe8217fa700 1 -- 172.21.15.7:0/3359847944 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe82c00bae0 con 0x55bfce99fc10 2024-01-11T18:13:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.877+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfce991d60 con 0x55bfce99fc10 2024-01-11T18:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.878+0000 7fe8217fa700 1 -- 172.21.15.7:0/3359847944 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe82c005140 con 0x55bfce99fc10 2024-01-11T18:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.878+0000 7fe853f9bb80 10 client.26689 fetch_fsmap learned FSMap version 47 2024-01-11T18:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55bfce713b00 con 0x55bfce99fc10 2024-01-11T18:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe8217fa700 1 -- 172.21.15.7:0/3359847944 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfce991d60 con 0x55bfce99fc10 2024-01-11T18:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe8217fa700 1 -- 172.21.15.7:0/3359847944 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe82c015020 con 0x55bfce99fc10 2024-01-11T18:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 10 client.26689 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 10 client.26689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 20 client.26689 populate_metadata read hostname 'smithi007' 2024-01-11T18:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfce9b42c0 0x55bfce9b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 4 client.26689 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.879+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfce8ebb60 con 0x55bfce9b42c0 2024-01-11T18:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.880+0000 7fe83d79a700 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfce9b42c0 0x55bfce9b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.880+0000 7fe8217fa700 10 client.26689 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.882+0000 7fe83d79a700 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfce9b42c0 0x55bfce9b66e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1d4b3e07fa3217 server_cookie=ece0440dbbaa5983 in_seq=0 out_seq=0 2024-01-11T18:13:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.882+0000 7fe8217fa700 1 -- 172.21.15.7:0/3359847944 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfce8ebb60 con 0x55bfce9b42c0 2024-01-11T18:13:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe8217fa700 10 client.26689 handle_command_reply: tid=0 2024-01-11T18:13:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 client.26689 shutdown 2024-01-11T18:13:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 msgr2=0x55bfce9a8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9a8db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009920 tx=0x7fe82c00f690 comp rx=0 tx=0).stop 2024-01-11T18:13:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 shutdown_connections 2024-01-11T18:13:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfce9b42c0 0x55bfce9b66e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce9a05d0 0x55bfce9afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe8217fa700 0 client.26689 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/3359847944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9a8db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.883+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 >> 172.21.15.7:0/3359847944 conn(0x55bfce914740 msgr2=0x55bfce998f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.885+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 shutdown_connections 2024-01-11T18:13:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.885+0000 7fe853f9bb80 1 -- 172.21.15.7:0/3359847944 wait complete. 2024-01-11T18:13:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.885+0000 7fe853f9bb80 20 client.26689 trim_cache size 0 max 16384 2024-01-11T18:13:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.918+0000 7fe853f9bb80 1 Processor -- start 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.918+0000 7fe853f9bb80 1 -- start start 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99fc10 0x55bfce99fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce769640 con 0x55bfce99ef20 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce9bb8b0 con 0x55bfce99fc10 2024-01-11T18:13:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce99b690 con 0x55bfce9a05d0 2024-01-11T18:13:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44072/0 (socket says 172.21.15.7:44072) 2024-01-11T18:13:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.919+0000 7fe83d79a700 1 -- 172.21.15.7:0/1972163347 learned_addr learned my addr 172.21.15.7:0/1972163347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83df9b700 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99fc10 0x55bfce99fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83d79a700 1 -- 172.21.15.7:0/1972163347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99fc10 msgr2=0x55bfce99fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83d79a700 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99fc10 0x55bfce99fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83d79a700 1 -- 172.21.15.7:0/1972163347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 msgr2=0x55bfce9918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83d79a700 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83d79a700 1 -- 172.21.15.7:0/1972163347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfce8b41d0 con 0x55bfce9a05d0 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.920+0000 7fe83df9b700 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99fc10 0x55bfce99fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:13:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.921+0000 7fe83d79a700 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9afd10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55bfce9b6f80 tx=0x7fe83400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5ef7b53837e4db server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.921+0000 7fe8237fe700 1 -- 172.21.15.7:0/1972163347 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83401a040 con 0x55bfce9a05d0 2024-01-11T18:13:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.921+0000 7fe8237fe700 1 -- 172.21.15.7:0/1972163347 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe834029440 con 0x55bfce9a05d0 2024-01-11T18:13:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.921+0000 7fe8237fe700 1 -- 172.21.15.7:0/1972163347 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834010630 con 0x55bfce9a05d0 2024-01-11T18:13:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 -- 172.21.15.7:0/1972163347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 msgr2=0x55bfce9afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9afd10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55bfce9b6f80 tx=0x7fe83400bb70 comp rx=0 tx=0).stop 2024-01-11T18:13:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 -- 172.21.15.7:0/1972163347 shutdown_connections 2024-01-11T18:13:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce9a05d0 0x55bfce9afd10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce99ef20 0x55bfce9918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/1972163347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99fc10 0x55bfce99fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.925+0000 7fe853f9bb80 1 -- 172.21.15.7:0/1972163347 >> 172.21.15.7:0/1972163347 conn(0x55bfce914740 msgr2=0x55bfce9a6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.926+0000 7fe853f9bb80 1 -- 172.21.15.7:0/1972163347 shutdown_connections 2024-01-11T18:13:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.926+0000 7fe853f9bb80 1 -- 172.21.15.7:0/1972163347 wait complete. 2024-01-11T18:13:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.927+0000 7fe853f9bb80 1 -- start start 2024-01-11T18:13:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.927+0000 7fe853f9bb80 1 Processor -- start 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce9ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce9a05d0 0x55bfce9ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce9aec20 con 0x55bfce9a05d0 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce9aed90 con 0x55bfce99ef20 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce9aef50 con 0x55bfce99fc10 2024-01-11T18:13:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe853f9bb80 10 client.0 _ll_register_callbacks cb 0x55bfce8936c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:13:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.928+0000 7fe83d79a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce9ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:44088/0 (socket says 172.21.15.7:44088) 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 -- 172.21.15.7:0/88460853 learned_addr learned my addr 172.21.15.7:0/88460853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83d79a700 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce9a05d0 0x55bfce9ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 -- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 msgr2=0x55bfce9ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce9ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 -- 172.21.15.7:0/88460853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce9a05d0 msgr2=0x55bfce9ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.929+0000 7fe83df9b700 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce9a05d0 0x55bfce9ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.930+0000 7fe83df9b700 1 -- 172.21.15.7:0/88460853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83400bd80 con 0x55bfce99fc10 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.930+0000 7fe83df9b700 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9ad6d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009230 tx=0x7fe82c009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ccc84039ad4645 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.930+0000 7fe8217fa700 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.930+0000 7fe8217fa700 1 -- 172.21.15.7:0/88460853 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c01d040 con 0x55bfce99fc10 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.930+0000 7fe8217fa700 1 -- 172.21.15.7:0/88460853 <== mon.1 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe82c000e40 con 0x55bfce99fc10 2024-01-11T18:13:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.931+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe82c00b560 con 0x55bfce99fc10 2024-01-11T18:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.933+0000 7fe8217fa700 1 -- 172.21.15.7:0/88460853 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe82c0058e0 con 0x55bfce99fc10 2024-01-11T18:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.933+0000 7fe853f9bb80 10 client.26695 fetch_fsmap learned FSMap version 47 2024-01-11T18:13:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.933+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55bfce76bc70 con 0x55bfce99fc10 2024-01-11T18:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.933+0000 7fe8217fa700 1 -- 172.21.15.7:0/88460853 <== mon.1 v2:172.21.15.38:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe82c043030 con 0x55bfce99fc10 2024-01-11T18:13:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.934+0000 7fe8217fa700 1 -- 172.21.15.7:0/88460853 <== mon.1 v2:172.21.15.38:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe82c009950 con 0x55bfce99fc10 2024-01-11T18:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.934+0000 7fe853f9bb80 10 client.26695 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.935+0000 7fe853f9bb80 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.935+0000 7fe853f9bb80 20 client.26695 populate_metadata read hostname 'smithi007' 2024-01-11T18:13:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.935+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfcea13200 0x55bfcea155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.935+0000 7fe853f9bb80 4 client.26695 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.935+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bfce8eb880 con 0x55bfcea13200 2024-01-11T18:13:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.936+0000 7fe83d79a700 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfcea13200 0x55bfcea155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.936+0000 7fe83d79a700 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfcea13200 0x55bfcea155f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5481d002db31f52 server_cookie=346793a9a681248c in_seq=0 out_seq=0 2024-01-11T18:13:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:19.936+0000 7fe8217fa700 10 client.26695 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.015+0000 7fe8217fa700 1 -- 172.21.15.7:0/88460853 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bfce8eb880 con 0x55bfcea13200 2024-01-11T18:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.015+0000 7fe8217fa700 10 client.26695 handle_command_reply: tid=0 2024-01-11T18:13:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.015+0000 7fe853f9bb80 1 client.26695 shutdown 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 msgr2=0x55bfce9ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9ad6d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009230 tx=0x7fe82c009fa0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 shutdown_connections 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x55bfcea13200 0x55bfcea155f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bfce9a05d0 0x55bfce9ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bfce99ef20 0x55bfce9ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 --2- 172.21.15.7:0/88460853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfce99fc10 0x55bfce9ad6d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 >> 172.21.15.7:0/88460853 conn(0x55bfce914740 msgr2=0x55bfce9a75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.016+0000 7fe8217fa700 0 client.26695 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.017+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 shutdown_connections 2024-01-11T18:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.017+0000 7fe853f9bb80 1 -- 172.21.15.7:0/88460853 wait complete. 2024-01-11T18:13:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.017+0000 7fe853f9bb80 20 client.26695 trim_cache size 0 max 16384 2024-01-11T18:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:13:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe8240a1fd0 msgr2=0x7fe8240a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe8240a1fd0 0x7fe8240a4490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe830096590 tx=0x7fe830009040 comp rx=0 tx=0).stop 2024-01-11T18:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 msgr2=0x7fe8381a76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 0x7fe8381a76c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c110 tx=0x7fe83000c420 comp rx=0 tx=0).stop 2024-01-11T18:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 shutdown_connections 2024-01-11T18:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fe8240a1fd0 0x7fe8240a4490 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe838103140 0x7fe8381a76c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8381015b0 0x7fe83806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 --2- 172.21.15.7:0/2867321430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8381027a0 0x7fe838070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.018+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 >> 172.21.15.7:0/2867321430 conn(0x7fe8380fcb00 msgr2=0x7fe838105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.019+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 shutdown_connections 2024-01-11T18:13:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:20.019+0000 7fe8401ff700 1 -- 172.21.15.7:0/2867321430 wait complete. 2024-01-11T18:13:21.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:21 smithi016 ceph-mon[43195]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.7 MiB/s wr, 494 op/s 2024-01-11T18:13:21.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:21 smithi038 ceph-mon[43582]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.7 MiB/s wr, 494 op/s 2024-01-11T18:13:21.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:21 smithi007 ceph-mon[33361]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.7 MiB/s wr, 494 op/s 2024-01-11T18:13:23.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:23 smithi016 ceph-mon[43195]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.2 MiB/s wr, 449 op/s 2024-01-11T18:13:23.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:23 smithi038 ceph-mon[43582]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.2 MiB/s wr, 449 op/s 2024-01-11T18:13:23.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:23 smithi007 ceph-mon[33361]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.2 MiB/s wr, 449 op/s 2024-01-11T18:13:24.110 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 895.0 s, 429.8 tps, lat 231.872 ms stddev 252.777 2024-01-11T18:13:26.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:25 smithi016 ceph-mon[43195]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-01-11T18:13:26.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:25 smithi038 ceph-mon[43582]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-01-11T18:13:26.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:25 smithi007 ceph-mon[33361]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-01-11T18:13:27.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:27 smithi016 ceph-mon[43195]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2024-01-11T18:13:27.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:27 smithi038 ceph-mon[43582]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2024-01-11T18:13:27.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:27 smithi007 ceph-mon[33361]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2024-01-11T18:13:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 900.0 s, 447.0 tps, lat 220.011 ms stddev 281.177 2024-01-11T18:13:29.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:29 smithi016 ceph-mon[43195]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 455 op/s 2024-01-11T18:13:29.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:29 smithi038 ceph-mon[43582]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 455 op/s 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:transaction type: 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:scaling factor: 32 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:query mode: simple 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:number of clients: 100 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:number of threads: 4 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:duration: 900 s 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:number of transactions actually processed: 640025 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:latency average = 140.617 ms 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:latency stddev = 202.020 ms 2024-01-11T18:13:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:tps = 710.787816 (including connections establishing) 2024-01-11T18:13:29.556 INFO:teuthology.orchestra.run.smithi007.stdout:tps = 710.794187 (excluding connections establishing) 2024-01-11T18:13:29.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:29 smithi007 ceph-mon[33361]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 455 op/s 2024-01-11T18:13:29.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-01-11T18:13:31.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:31 smithi038 ceph-mon[43582]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.0 MiB/s wr, 491 op/s 2024-01-11T18:13:31.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:31 smithi007 ceph-mon[33361]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.0 MiB/s wr, 491 op/s 2024-01-11T18:13:31.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:31 smithi016 ceph-mon[43195]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.0 MiB/s wr, 491 op/s 2024-01-11T18:13:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.293+0000 7f468c5ff700 1 -- 172.21.15.7:0/673714250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4684108610 msgr2=0x7f468410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.293+0000 7f468c5ff700 1 --2- 172.21.15.7:0/673714250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4684108610 0x7f468410aa40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4680009990 tx=0x7f4680009ca0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 -- 172.21.15.7:0/673714250 shutdown_connections 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 --2- 172.21.15.7:0/673714250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468410b400 0x7f46841142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 --2- 172.21.15.7:0/673714250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4684108610 0x7f468410aa40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 --2- 172.21.15.7:0/673714250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f468410af80 0x7f4684111d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 -- 172.21.15.7:0/673714250 >> 172.21.15.7:0/673714250 conn(0x7f468406e7a0 msgr2=0x7f4684070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 -- 172.21.15.7:0/673714250 shutdown_connections 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.294+0000 7f468c5ff700 1 -- 172.21.15.7:0/673714250 wait complete. 2024-01-11T18:13:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.295+0000 7f468c5ff700 1 Processor -- start 2024-01-11T18:13:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 -- start start 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684108610 0x7f46841a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468410af80 0x7f46841a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 0x7f46841a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46841a9ae0 con 0x7f468410b400 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46841a9c20 con 0x7f468410af80 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.296+0000 7f468c5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46841a9db0 con 0x7f4684108610 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468a39b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684108610 0x7f46841a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 0x7f46841a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 0x7f46841a9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60100/0 (socket says 172.21.15.7:60100) 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 -- 172.21.15.7:0/1247920162 learned_addr learned my addr 172.21.15.7:0/1247920162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 -- 172.21.15.7:0/1247920162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684108610 msgr2=0x7f46841a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684108610 0x7f46841a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 -- 172.21.15.7:0/1247920162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468410af80 msgr2=0x7f46841a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468410af80 0x7f46841a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.297+0000 7f468ab9c700 1 -- 172.21.15.7:0/1247920162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800095f0 con 0x7f468410b400 2024-01-11T18:13:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.298+0000 7f468ab9c700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 0x7f46841a9510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b970 tx=0x7f467c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.298+0000 7f467b7fe700 1 -- 172.21.15.7:0/1247920162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00c9b0 con 0x7f468410b400 2024-01-11T18:13:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.298+0000 7f467b7fe700 1 -- 172.21.15.7:0/1247920162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467c014440 con 0x7f468410b400 2024-01-11T18:13:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.299+0000 7f467b7fe700 1 -- 172.21.15.7:0/1247920162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0075d0 con 0x7f468410b400 2024-01-11T18:13:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.299+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46841aa090 con 0x7f468410b400 2024-01-11T18:13:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.299+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46841aa550 con 0x7f468410b400 2024-01-11T18:13:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.316+0000 7f46a0366b80 1 Processor -- start 2024-01-11T18:13:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.316+0000 7f467b7fe700 1 -- 172.21.15.7:0/1247920162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f467c0145b0 con 0x7f468410b400 2024-01-11T18:13:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 -- start start 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60894f20 0x556a60895340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60895c10 0x556a60896090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a6089ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6078c030 con 0x556a60895c10 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a607efac0 con 0x556a60894f20 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.320+0000 7f46a0366b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a607ed3e0 con 0x556a608965d0 2024-01-11T18:13:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.321+0000 7f4689b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60894f20 0x556a60895340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.321+0000 7f468a39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60895c10 0x556a60896090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.321+0000 7f4689b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60894f20 0x556a60895340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49738/0 (socket says 172.21.15.7:49738) 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.321+0000 7f4689b9a700 1 -- 172.21.15.7:0/488467938 learned_addr learned my addr 172.21.15.7:0/488467938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.321+0000 7f467b7fe700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46700a2060 0x7f46700a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468a39b700 1 -- 172.21.15.7:0/488467938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 msgr2=0x556a6089ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468a39b700 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a6089ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468a39b700 1 -- 172.21.15.7:0/488467938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60894f20 msgr2=0x556a60895340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468a39b700 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60894f20 0x556a60895340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468a39b700 1 -- 172.21.15.7:0/488467938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a6065d350 con 0x556a60895c10 2024-01-11T18:13:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468ab9c700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46700a2060 0x7f46700a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f468a39b700 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60895c10 0x556a60896090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4680006010 tx=0x7f4680005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a0f24f14872ad1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.322+0000 7f467b7fe700 1 -- 172.21.15.7:0/1247920162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f467c006e40 con 0x7f468410b400 2024-01-11T18:13:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.323+0000 7f468ab9c700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46700a2060 0x7f46700a4520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f467c00cfd0 tx=0x7f467c0690b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.323+0000 7f4678bfe700 1 -- 172.21.15.7:0/488467938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680009ee0 con 0x556a60895c10 2024-01-11T18:13:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.323+0000 7f4678bfe700 1 -- 172.21.15.7:0/488467938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468001d040 con 0x556a60895c10 2024-01-11T18:13:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.323+0000 7f4678bfe700 1 -- 172.21.15.7:0/488467938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680004a30 con 0x556a60895c10 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.326+0000 7f46a0366b80 1 -- 172.21.15.7:0/488467938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60895c10 msgr2=0x556a60896090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60895c10 0x556a60896090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4680006010 tx=0x7f4680005ab0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 -- 172.21.15.7:0/488467938 shutdown_connections 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a6089ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60894f20 0x556a60895340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 --2- 172.21.15.7:0/488467938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60895c10 0x556a60896090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 -- 172.21.15.7:0/488467938 >> 172.21.15.7:0/488467938 conn(0x556a6080a740 msgr2=0x556a6088f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 -- 172.21.15.7:0/488467938 shutdown_connections 2024-01-11T18:13:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.327+0000 7f46a0366b80 1 -- 172.21.15.7:0/488467938 wait complete. 2024-01-11T18:13:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.328+0000 7f46a0366b80 1 -- start start 2024-01-11T18:13:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.328+0000 7f46a0366b80 1 Processor -- start 2024-01-11T18:13:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a6089e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a6089ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a607e8cd0 con 0x556a60894f20 2024-01-11T18:13:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a60757340 con 0x556a60895c10 2024-01-11T18:13:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a60756fe0 con 0x556a608965d0 2024-01-11T18:13:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.329+0000 7f46a0366b80 10 client.0 _ll_register_callbacks cb 0x556a607896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:13:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.331+0000 7f4689b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a6089e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.331+0000 7f4689b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a6089e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60120/0 (socket says 172.21.15.7:60120) 2024-01-11T18:13:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.331+0000 7f4689b9a700 1 -- 172.21.15.7:0/2498002093 learned_addr learned my addr 172.21.15.7:0/2498002093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.332+0000 7f4689b9a700 1 -- 172.21.15.7:0/2498002093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 msgr2=0x556a608a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.332+0000 7f468a39b700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a6089ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.332+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.332+0000 7f4689b9a700 1 -- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 msgr2=0x556a6089ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.332+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a6089ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.332+0000 7f4689b9a700 1 -- 172.21.15.7:0/2498002093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4684114ce0 con 0x556a60894f20 2024-01-11T18:13:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.333+0000 7f468a39b700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a6089ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T18:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.333+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a6089e8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f467400a580 tx=0x7f4674012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0c08cb41fb1f3c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.333+0000 7f466dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T18:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.334+0000 7f466dffb700 1 -- 172.21.15.7:0/2498002093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674004a30 con 0x556a60894f20 2024-01-11T18:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.334+0000 7f466dffb700 1 -- 172.21.15.7:0/2498002093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4674004b90 con 0x556a60894f20 2024-01-11T18:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.334+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a60887d60 con 0x556a60894f20 2024-01-11T18:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.334+0000 7f466dffb700 1 -- 172.21.15.7:0/2498002093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4674004d50 con 0x556a60894f20 2024-01-11T18:13:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.334+0000 7f46a0366b80 10 client.16389 fetch_fsmap learned FSMap version 47 2024-01-11T18:13:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.334+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556a60609b00 con 0x556a60894f20 2024-01-11T18:13:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.335+0000 7f466dffb700 1 -- 172.21.15.7:0/2498002093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a60887d60 con 0x556a60894f20 2024-01-11T18:13:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.335+0000 7f466dffb700 1 -- 172.21.15.7:0/2498002093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4674024820 con 0x556a60894f20 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.335+0000 7f46a0366b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.336+0000 7f46a0366b80 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.336+0000 7f46a0366b80 20 client.16389 populate_metadata read hostname 'smithi007' 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.336+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a608b2e90 0x556a608b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.336+0000 7f46a0366b80 4 client.16389 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.336+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a607e1b60 con 0x556a608b2e90 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.336+0000 7f468a39b700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a608b2e90 0x556a608b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.337+0000 7f468a39b700 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a608b2e90 0x556a608b5330 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae30debddc1e5cca server_cookie=e97d1b18830b2038 in_seq=0 out_seq=0 2024-01-11T18:13:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.337+0000 7f466dffb700 10 client.16389 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.338+0000 7f466dffb700 1 -- 172.21.15.7:0/2498002093 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a607e1b60 con 0x556a608b2e90 2024-01-11T18:13:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.338+0000 7f466dffb700 10 client.16389 handle_command_reply: tid=0 2024-01-11T18:13:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.338+0000 7f46a0366b80 1 client.16389 shutdown 2024-01-11T18:13:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.338+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 msgr2=0x556a6089e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a6089e8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f467400a580 tx=0x7f4674012730 comp rx=0 tx=0).stop 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 shutdown_connections 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a608b2e90 0x556a608b5330 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a6089e8d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2498002093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a6089ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 >> 172.21.15.7:0/2498002093 conn(0x556a6080a740 msgr2=0x556a6088ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f466dffb700 0 client.16389 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 shutdown_connections 2024-01-11T18:13:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.339+0000 7f46a0366b80 1 -- 172.21.15.7:0/2498002093 wait complete. 2024-01-11T18:13:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.340+0000 7f46a0366b80 20 client.16389 trim_cache size 0 max 16384 2024-01-11T18:13:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.382+0000 7f46a0366b80 1 Processor -- start 2024-01-11T18:13:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.382+0000 7f46a0366b80 1 -- start start 2024-01-11T18:13:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.383+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.383+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a60895c10 0x556a608a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.383+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a608965d0 0x556a6089cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.383+0000 7f46a0366b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6065f640 con 0x556a60894f20 2024-01-11T18:13:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.383+0000 7f46a0366b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a60894bb0 con 0x556a608965d0 2024-01-11T18:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.383+0000 7f46a0366b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6088c560 con 0x556a60895c10 2024-01-11T18:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.384+0000 7f468a39b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a60895c10 0x556a608a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.384+0000 7f468a39b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a60895c10 0x556a608a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:57972/0 (socket says 172.21.15.7:57972) 2024-01-11T18:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.384+0000 7f468a39b700 1 -- 172.21.15.7:0/3895367226 learned_addr learned my addr 172.21.15.7:0/3895367226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.384+0000 7f4689b9a700 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.384+0000 7f4689b9a700 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a608965d0 0x556a6089cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.385+0000 7f4689b9a700 1 -- 172.21.15.7:0/3895367226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a60895c10 msgr2=0x556a608a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.385+0000 7f4689b9a700 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a60895c10 0x556a608a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.385+0000 7f4689b9a700 1 -- 172.21.15.7:0/3895367226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a608965d0 msgr2=0x556a6089cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.385+0000 7f4689b9a700 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a608965d0 0x556a6089cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.385+0000 7f4689b9a700 1 -- 172.21.15.7:0/3895367226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a607aa1d0 con 0x556a60894f20 2024-01-11T18:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.385+0000 7f4689b9a700 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608878a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4674018ca0 tx=0x7f467400afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4675463e6100516 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.386+0000 7f466ffff700 1 -- 172.21.15.7:0/3895367226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674004d50 con 0x556a60894f20 2024-01-11T18:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.386+0000 7f466ffff700 1 -- 172.21.15.7:0/3895367226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4674024420 con 0x556a60894f20 2024-01-11T18:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.386+0000 7f466ffff700 1 -- 172.21.15.7:0/3895367226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400cdb0 con 0x556a60894f20 2024-01-11T18:13:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 -- 172.21.15.7:0/3895367226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 msgr2=0x556a608878a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608878a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4674018ca0 tx=0x7f467400afd0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 -- 172.21.15.7:0/3895367226 shutdown_connections 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a608965d0 0x556a6089cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608878a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 --2- 172.21.15.7:0/3895367226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a60895c10 0x556a608a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 -- 172.21.15.7:0/3895367226 >> 172.21.15.7:0/3895367226 conn(0x556a6080a740 msgr2=0x556a6088b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 -- 172.21.15.7:0/3895367226 shutdown_connections 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.387+0000 7f46a0366b80 1 -- 172.21.15.7:0/3895367226 wait complete. 2024-01-11T18:13:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.388+0000 7f46a0366b80 1 -- start start 2024-01-11T18:13:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.388+0000 7f46a0366b80 1 Processor -- start 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.389+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a608a3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f46a0366b80 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f46a0366b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a608a4da0 con 0x556a60894f20 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f46a0366b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a608a4f10 con 0x556a60895c10 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f46a0366b80 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a608a50d0 con 0x556a608965d0 2024-01-11T18:13:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f46a0366b80 10 client.0 _ll_register_callbacks cb 0x556a607896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T18:13:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f4689b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f468a39b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a608a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f4689b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60128/0 (socket says 172.21.15.7:60128) 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f4689b9a700 1 -- 172.21.15.7:0/2772891674 learned_addr learned my addr 172.21.15.7:0/2772891674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.390+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f468a39b700 1 -- 172.21.15.7:0/2772891674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 msgr2=0x556a608a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f468a39b700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f468a39b700 1 -- 172.21.15.7:0/2772891674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 msgr2=0x556a608a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f468a39b700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f468a39b700 1 -- 172.21.15.7:0/2772891674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a60609b00 con 0x556a60895c10 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f468a39b700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a608a3850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4680005b40 tx=0x7f468000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321c5d2d7fe1ee5a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.391+0000 7f466dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-01-11T18:13:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.392+0000 7f466dffb700 1 -- 172.21.15.7:0/2772891674 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680005040 con 0x556a60895c10 2024-01-11T18:13:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.392+0000 7f466dffb700 1 -- 172.21.15.7:0/2772891674 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f46800051a0 con 0x556a60895c10 2024-01-11T18:13:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.394+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f468001d040 con 0x556a60895c10 2024-01-11T18:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.398+0000 7f466dffb700 1 -- 172.21.15.7:0/2772891674 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4680005380 con 0x556a60895c10 2024-01-11T18:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.398+0000 7f46a0366b80 10 client.26663 fetch_fsmap learned FSMap version 47 2024-01-11T18:13:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.398+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556a60661c70 con 0x556a60895c10 2024-01-11T18:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f466dffb700 1 -- 172.21.15.7:0/2772891674 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4680004c90 con 0x556a60895c10 2024-01-11T18:13:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f466dffb700 1 -- 172.21.15.7:0/2772891674 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4680020070 con 0x556a60895c10 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f46a0366b80 10 client.26663 fetch_fsmap finished waiting for FSMap version 47 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f46a0366b80 10 client.26663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f46a0366b80 20 client.26663 populate_metadata read hostname 'smithi007' 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a60909790 0x556a6090bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f46a0366b80 4 client.26663 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.401+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 --> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f468001d040 con 0x556a60909790 2024-01-11T18:13:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.402+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a60909790 0x556a6090bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.403+0000 7f4689b9a700 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a60909790 0x556a6090bb80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a81fec54ef322d8 server_cookie=4c2304ebeda584ea in_seq=0 out_seq=0 2024-01-11T18:13:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:33.403+0000 7f466dffb700 10 client.26663 ms_handle_connect on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:33.491 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:33 smithi016 ceph-mon[43195]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 482 op/s 2024-01-11T18:13:33.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:33 smithi038 ceph-mon[43582]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 482 op/s 2024-01-11T18:13:33.566 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:33 smithi007 ceph-mon[33361]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 482 op/s 2024-01-11T18:13:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.817+0000 7f466dffb700 1 -- 172.21.15.7:0/2772891674 <== mds.0 v2:172.21.15.16:6838/1892769619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f468001d040 con 0x556a60909790 2024-01-11T18:13:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.817+0000 7f466dffb700 10 client.26663 handle_command_reply: tid=0 2024-01-11T18:13:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.817+0000 7f46a0366b80 1 client.26663 shutdown 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 msgr2=0x556a608a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a608a3850 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4680005b40 tx=0x7f468000b670 comp rx=0 tx=0).stop 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 shutdown_connections 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:6838/1892769619,v1:172.21.15.16:6839/1892769619] conn(0x556a60909790 0x556a6090bb80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556a608965d0 0x556a608a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a60894f20 0x556a608a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 --2- 172.21.15.7:0/2772891674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a60895c10 0x556a608a3850 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f466dffb700 0 client.26663 ms_handle_reset on v2:172.21.15.16:6838/1892769619 2024-01-11T18:13:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 >> 172.21.15.7:0/2772891674 conn(0x556a6080a740 msgr2=0x556a6089cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 shutdown_connections 2024-01-11T18:13:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.818+0000 7f46a0366b80 1 -- 172.21.15.7:0/2772891674 wait complete. 2024-01-11T18:13:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.819+0000 7f46a0366b80 20 client.26663 trim_cache size 0 max 16384 2024-01-11T18:13:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-01-11T18:13:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-01-11T18:13:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-01-11T18:13:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-01-11T18:13:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.821+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46700a2060 msgr2=0x7f46700a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.821+0000 7f468c5ff700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46700a2060 0x7f46700a4520 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f467c00cfd0 tx=0x7f467c0690b0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.821+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 msgr2=0x7f46841a9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.821+0000 7f468c5ff700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 0x7f46841a9510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b970 tx=0x7f467c00bd30 comp rx=0 tx=0).stop 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 shutdown_connections 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f46700a2060 0x7f46700a4520 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4684108610 0x7f46841a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468410b400 0x7f46841a9510 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 --2- 172.21.15.7:0/1247920162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468410af80 0x7f46841a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 >> 172.21.15.7:0/1247920162 conn(0x7f468406e7a0 msgr2=0x7f468406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 shutdown_connections 2024-01-11T18:13:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-01-11T18:13:34.822+0000 7f468c5ff700 1 -- 172.21.15.7:0/1247920162 wait complete. 2024-01-11T18:13:36.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:35 smithi016 ceph-mon[43195]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-01-11T18:13:36.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:35 smithi038 ceph-mon[43582]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-01-11T18:13:36.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:35 smithi007 ceph-mon[33361]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-01-11T18:13:37.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:37 smithi016 ceph-mon[43195]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.0 MiB/s wr, 658 op/s 2024-01-11T18:13:37.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:37 smithi038 ceph-mon[43582]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.0 MiB/s wr, 658 op/s 2024-01-11T18:13:37.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:37 smithi007 ceph-mon[33361]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.0 MiB/s wr, 658 op/s 2024-01-11T18:13:39.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:39 smithi016 ceph-mon[43195]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.3 MiB/s wr, 586 op/s 2024-01-11T18:13:39.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:39 smithi038 ceph-mon[43582]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.3 MiB/s wr, 586 op/s 2024-01-11T18:13:39.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:39 smithi007 ceph-mon[33361]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.3 MiB/s wr, 586 op/s 2024-01-11T18:13:41.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:41 smithi016 ceph-mon[43195]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.4 MiB/s wr, 692 op/s 2024-01-11T18:13:41.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:41 smithi038 ceph-mon[43582]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.4 MiB/s wr, 692 op/s 2024-01-11T18:13:41.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:41 smithi007 ceph-mon[33361]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.4 MiB/s wr, 692 op/s 2024-01-11T18:13:43.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:43 smithi016 ceph-mon[43195]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-01-11T18:13:43.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:43 smithi038 ceph-mon[43582]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-01-11T18:13:43.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:43 smithi007 ceph-mon[33361]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-01-11T18:13:43.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-01-11T18:13:44.048 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-01-11T18:13:44.048 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.048 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T18:13:44.048 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 .. 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 data 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:total 31 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 . 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 .. 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jan 11 17:58 base 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jan 11 17:58 current_logfiles 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Jan 11 17:58 global 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 log 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_commit_ts 2024-01-11T18:13:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_dynshmem 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jan 11 17:58 pg_hba.conf 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jan 11 17:58 pg_ident.conf 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 18:13 pg_logical 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 pg_multixact 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 pg_notify 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_replslot 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_serial 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_snapshots 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 18:13 pg_stat 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 18:13 pg_stat_tmp 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 18:13 pg_subtrans 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_tblspc 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pg_twophase 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T18:13:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Jan 11 18:13 pg_wal 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 pg_xact 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jan 11 17:58 postgresql.auto.conf 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jan 11 17:58 postgresql.conf 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jan 11 17:58 postmaster.opts 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base: 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jan 11 17:58 . 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 1 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 12346 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Jan 11 17:58 12347 2024-01-11T18:13:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 pgsql_tmp 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/1: 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:total 7209 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 . 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jan 11 17:58 .. 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 112 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 113 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jan 11 17:58 12182 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12182_fsm 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12182_vm 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12184 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12186 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187 2024-01-11T18:13:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12187_fsm 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187_vm 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12189 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12191 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12192_fsm 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192_vm 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12194 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12196 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12197_fsm 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197_vm 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12199 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12201 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202 2024-01-11T18:13:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12202_fsm 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202_vm 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12204 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12206 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12207_fsm 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207_vm 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12209 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12211 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12212 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12214 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12216 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jan 11 17:58 1247 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1247_fsm 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1247_vm 2024-01-11T18:13:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jan 11 17:58 1249 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1249_fsm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1249_vm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jan 11 17:58 1255 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1255_fsm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1255_vm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jan 11 17:58 1259 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1259_fsm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1259_vm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417_vm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418_vm 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 174 2024-01-11T18:13:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 175 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2187 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224_vm 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328_vm 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336_vm 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2337 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2600 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2600_fsm 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2600_vm 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2601_fsm 2024-01-11T18:13:44.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601_vm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2602 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2602_fsm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2602_vm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2603 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2603_fsm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2603_vm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604_vm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2605 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2605_fsm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2605_vm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2606_fsm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606_vm 2024-01-11T18:13:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607_fsm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2607_vm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jan 11 17:58 2608 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2608_fsm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2608_vm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jan 11 17:58 2609 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2609_fsm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2609_vm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610_fsm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2610_vm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611_vm 2024-01-11T18:13:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2612_fsm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612_vm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613_vm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2615_fsm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615_vm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616_fsm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2616_vm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jan 11 17:58 2617 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2617_fsm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2617_vm 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jan 11 17:58 2618 2024-01-11T18:13:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2618_fsm 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2618_vm 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jan 11 17:58 2619 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2619_fsm 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2619_vm 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620_vm 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2650 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2651 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2652 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2653 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2654 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2655 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2656 2024-01-11T18:13:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2657 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jan 11 17:58 2658 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2659 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2660 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2661 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2662 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2663 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2664 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2665 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2666 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2667 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2668 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2669 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2670 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jan 11 17:58 2673 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jan 11 17:58 2674 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jan 11 17:58 2675 2024-01-11T18:13:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2678 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2679 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2680 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2681 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2682 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2683 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2684 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2685 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2686 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2687 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2688 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2689 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2690 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jan 11 17:58 2691 2024-01-11T18:13:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2692 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2693 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2696 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2699 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2701 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2702 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2703 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2704 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2753 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2753_fsm 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2753_vm 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2754 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2755 2024-01-11T18:13:44.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2756 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2757 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830_vm 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2831 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832_vm 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2833 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834_vm 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2835 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836_vm 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2837 2024-01-11T18:13:44.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jan 11 17:58 2838 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2838_fsm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2838_vm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2839 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840_fsm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2840_vm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2841 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995_vm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2996 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3079_fsm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079_vm 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3080 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3081 2024-01-11T18:13:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3085 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118_vm 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3119 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3164 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256_vm 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3257 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3258 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350_vm 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3351 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3379 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3380 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381 2024-01-11T18:13:44.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381_vm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3394 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3394_fsm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3394_vm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3395 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439_vm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3440 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 3455 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3456 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3456_fsm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3456_vm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466_vm 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3467 2024-01-11T18:13:44.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3468 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501_vm 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3502 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3503 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3534 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3541_fsm 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541_vm 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3542 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3574 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3575 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576_vm 2024-01-11T18:13:44.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596_vm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3597 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598_vm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3599 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3600_fsm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600_vm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3601_fsm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601_vm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3602_fsm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602_vm 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3603 2024-01-11T18:13:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3603_fsm 2024-01-11T18:13:44.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3603_vm 2024-01-11T18:13:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3604 2024-01-11T18:13:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3605 2024-01-11T18:13:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3606 2024-01-11T18:13:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3607 2024-01-11T18:13:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3608 2024-01-11T18:13:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 3609 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3712 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3764_fsm 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764_vm 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3766 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3767 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3997 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 5002 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 548 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 549 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102_vm 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104_vm 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106 2024-01-11T18:13:44.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106_vm 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6110 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6111 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6112 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6113 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6117 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826_vm 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 827 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 828 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/12346: 2024-01-11T18:13:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:total 7209 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jan 11 17:58 . 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jan 11 17:58 .. 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 112 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 113 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jan 11 17:58 12182 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12182_fsm 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12182_vm 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12184 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12186 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12187_fsm 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187_vm 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12189 2024-01-11T18:13:44.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12191 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12192_fsm 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192_vm 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12194 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12196 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12197_fsm 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197_vm 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12199 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12201 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12202_fsm 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202_vm 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12204 2024-01-11T18:13:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12206 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12207_fsm 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207_vm 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12209 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12211 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12212 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12214 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12216 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jan 11 17:58 1247 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1247_fsm 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1247_vm 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jan 11 17:58 1249 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1249_fsm 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1249_vm 2024-01-11T18:13:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jan 11 17:58 1255 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1255_fsm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1255_vm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jan 11 17:58 1259 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1259_fsm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1259_vm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417_vm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418_vm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 174 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 175 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2187 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224_vm 2024-01-11T18:13:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328_vm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336_vm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2337 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2600 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2600_fsm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2600_vm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2601_fsm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601_vm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2602 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2602_fsm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2602_vm 2024-01-11T18:13:44.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2603 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2603_fsm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2603_vm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604_vm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2605 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2605_fsm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2605_vm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2606_fsm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606_vm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607_fsm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2607_vm 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jan 11 17:58 2608 2024-01-11T18:13:44.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2608_fsm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2608_vm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jan 11 17:58 2609 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2609_fsm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2609_vm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610_fsm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2610_vm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611_vm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2612_fsm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612_vm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613_vm 2024-01-11T18:13:44.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2615_fsm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615_vm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616_fsm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2616_vm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jan 11 17:58 2617 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2617_fsm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2617_vm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jan 11 17:58 2618 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2618_fsm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2618_vm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jan 11 17:58 2619 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2619_fsm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2619_vm 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620 2024-01-11T18:13:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620_vm 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2650 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2651 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2652 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2653 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2654 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2655 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2656 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2657 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jan 11 17:58 2658 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2659 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2660 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2661 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2662 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2663 2024-01-11T18:13:44.088 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2664 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2665 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2666 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2667 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2668 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2669 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2670 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jan 11 17:58 2673 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jan 11 17:58 2674 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jan 11 17:58 2675 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2678 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2679 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2680 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2681 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2682 2024-01-11T18:13:44.089 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2683 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2684 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2685 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2686 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2687 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2688 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2689 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2690 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jan 11 17:58 2691 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2692 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2693 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2696 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2699 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2701 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2702 2024-01-11T18:13:44.090 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2703 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2704 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2753 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2753_fsm 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2753_vm 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2754 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2755 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2756 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2757 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830_vm 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2831 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832_vm 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2833 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834 2024-01-11T18:13:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834_vm 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2835 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836_vm 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2837 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jan 11 17:58 2838 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2838_fsm 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2838_vm 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2839 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840_fsm 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2840_vm 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2841 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995 2024-01-11T18:13:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995_vm 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2996 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3079_fsm 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079_vm 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3080 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3081 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3085 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118_vm 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3119 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3164 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256_vm 2024-01-11T18:13:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3257 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3258 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350_vm 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3351 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3379 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3380 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381_vm 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3394 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3394_fsm 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3394_vm 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3395 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439_vm 2024-01-11T18:13:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3440 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 3455 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3456 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3456_fsm 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3456_vm 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466_vm 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3467 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3468 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501_vm 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3502 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3503 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3534 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541 2024-01-11T18:13:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3541_fsm 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541_vm 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3542 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3574 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3575 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576_vm 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596_vm 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3597 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598_vm 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3599 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600 2024-01-11T18:13:44.096 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3600_fsm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600_vm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3601_fsm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601_vm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3602_fsm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602_vm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3603 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3603_fsm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3603_vm 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3604 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3605 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3606 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3607 2024-01-11T18:13:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3608 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 3609 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3712 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3764_fsm 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764_vm 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3766 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3767 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3997 2024-01-11T18:13:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 5002 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 548 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 549 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102_vm 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104_vm 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106_vm 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6110 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6111 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6112 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6113 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6117 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826 2024-01-11T18:13:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826_vm 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 827 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 828 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/12347: 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:total 545240 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Jan 11 17:58 . 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jan 11 17:58 .. 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 112 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 113 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jan 11 17:58 12182 2024-01-11T18:13:44.120 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12182_fsm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12182_vm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12184 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12186 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12187_fsm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12187_vm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12189 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12191 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12192_fsm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12192_vm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12194 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12196 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12197_fsm 2024-01-11T18:13:44.121 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12197_vm 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12199 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12201 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12202_fsm 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12202_vm 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12204 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12206 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 12207_fsm 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12207_vm 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12209 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12211 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12212 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 12214 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 12216 2024-01-11T18:13:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jan 11 17:58 1247 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1247_fsm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1247_vm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jan 11 17:58 1249 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1249_fsm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1249_vm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jan 11 17:58 1255 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1255_fsm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1255_vm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jan 11 18:13 1259 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1259_fsm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1259_vm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1417_vm 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418 2024-01-11T18:13:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 1418_vm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2613248 Jan 11 18:13 16387 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 18:13 16387_fsm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 18:12 16387_vm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 368640 Jan 11 18:13 16393 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 18:13 16393_fsm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 18:13 16393_vm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 441409536 Jan 11 18:13 16396 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jan 11 18:11 16396_fsm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 18:11 16396_vm 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 18:13 16397 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Jan 11 18:13 16399 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Jan 11 18:13 16401 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 33988608 Jan 11 18:13 16403 2024-01-11T18:13:44.124 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 18:13 16403_fsm 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 174 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 175 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2187 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2224_vm 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2328_vm 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2336_vm 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2337 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2600 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2600_fsm 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2600_vm 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601 2024-01-11T18:13:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2601_fsm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2601_vm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2602 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2602_fsm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2602_vm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2603 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2603_fsm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2603_vm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2604_vm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2605 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2605_fsm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2605_vm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2606_fsm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2606_vm 2024-01-11T18:13:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2607_fsm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2607_vm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Jan 11 17:58 2608 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2608_fsm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2608_vm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jan 11 17:58 2609 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2609_fsm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2609_vm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2610 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2610_fsm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2610_vm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2611_vm 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612 2024-01-11T18:13:44.127 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2612_fsm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2612_vm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2613_vm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2615_fsm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2615_vm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2616_fsm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2616_vm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jan 11 17:58 2617 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2617_fsm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2617_vm 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jan 11 17:58 2618 2024-01-11T18:13:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2618_fsm 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2618_vm 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Jan 11 18:13 2619 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 18:01 2619_fsm 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2619_vm 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2620_vm 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2650 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2651 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2652 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2653 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2654 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2655 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2656 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2657 2024-01-11T18:13:44.129 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jan 11 17:58 2658 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2659 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2660 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2661 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2662 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2663 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2664 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2665 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2666 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2667 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2668 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2669 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2670 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jan 11 17:58 2673 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jan 11 17:58 2674 2024-01-11T18:13:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jan 11 17:58 2675 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2678 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2679 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2680 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2681 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2682 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2683 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2684 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2685 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2686 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2687 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2688 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jan 11 17:58 2689 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jan 11 17:58 2690 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jan 11 17:58 2691 2024-01-11T18:13:44.131 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2692 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2693 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 18:05 2696 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2699 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2701 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2702 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2703 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 2704 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2753 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2753_fsm 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2753_vm 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2754 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2755 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2756 2024-01-11T18:13:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 2757 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2830_vm 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2831 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2832_vm 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2833 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2834_vm 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2835 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2836_vm 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2837 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jan 11 17:58 2838 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2838_fsm 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2838_vm 2024-01-11T18:13:44.133 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2839 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2840_fsm 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2840_vm 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2841 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2995_vm 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2996 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3079_fsm 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3079_vm 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3080 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3081 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3085 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118 2024-01-11T18:13:44.134 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3118_vm 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3119 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3164 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3256_vm 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3257 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3258 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3350_vm 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3351 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3379 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3380 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3381_vm 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3394 2024-01-11T18:13:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3394_fsm 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3394_vm 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3395 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3439_vm 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3440 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jan 11 17:58 3455 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3456 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3456_fsm 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3456_vm 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3466_vm 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3467 2024-01-11T18:13:44.136 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3468 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3501_vm 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3502 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3503 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3534 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3541_fsm 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3541_vm 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3542 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3574 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3575 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3576_vm 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3596_vm 2024-01-11T18:13:44.137 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3597 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3598_vm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3599 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3600_fsm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3600_vm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3601_fsm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3601_vm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3602_fsm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3602_vm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3603 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3603_fsm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3603_vm 2024-01-11T18:13:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3604 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3605 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3606 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3607 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3608 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jan 11 17:58 3609 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3712 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 3764_fsm 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3764_vm 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3766 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 3767 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3997 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 5002 2024-01-11T18:13:44.139 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 548 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 549 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6102_vm 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6104_vm 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6106_vm 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6110 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6111 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6112 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6113 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6117 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826 2024-01-11T18:13:44.140 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 826_vm 2024-01-11T18:13:44.141 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 827 2024-01-11T18:13:44.141 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 828 2024-01-11T18:13:44.141 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T18:13:44.141 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Jan 11 17:58 pg_internal.init 2024-01-11T18:13:44.141 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 17:58 PG_VERSION 2024-01-11T18:13:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jan 11 17:58 .. 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/global: 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:total 574 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Jan 11 17:58 . 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1136 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1136_fsm 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1136_vm 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 1137 2024-01-11T18:13:44.142 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1213 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1213_fsm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1213_vm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1214 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1214_fsm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1214_vm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 1232 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 1233 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1260 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1260_fsm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1260_vm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1261 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1261_fsm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1261_vm 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1262 2024-01-11T18:13:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 1262_fsm 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 1262_vm 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2396 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jan 11 17:58 2396_fsm 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2396_vm 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2397 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2671 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2672 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2676 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2677 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2694 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2695 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2697 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jan 11 17:58 2698 2024-01-11T18:13:44.144 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2846 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2846_vm 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2847 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2964 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2964_vm 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2965 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2966 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 2966_vm 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 2967 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3592 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 3592_vm 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 3593 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 4060 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 4060_vm 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 4061 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6000 2024-01-11T18:13:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6000_vm 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6001 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6002 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6100 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 6100_vm 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6114 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 6115 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 18:13 pg_control 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jan 11 17:58 pg_filenode.map 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Jan 11 17:58 pg_internal.init 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/log: 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 900 Jan 11 18:13 postgresql-Thu.log 2024-01-11T18:13:44.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 18:13 . 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 mappings 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jan 11 18:13 replorigin_checkpoint 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 snapshots 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 18:13 .. 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.148 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 18:13 .. 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 . 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 members 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 offsets 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-01-11T18:13:44.149 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 .. 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jan 11 17:58 .. 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.150 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jan 11 17:58 0000 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.151 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:total 13 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jan 11 18:13 . 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Jan 11 18:13 db_0.stat 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Jan 11 18:13 db_12347.stat 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Jan 11 18:13 global.stat 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 18:13 . 2024-01-11T18:13:44.152 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:total 200 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 18:13 . 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 Jan 11 18:13 0009 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:total 1032192 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Jan 11 18:13 . 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000C9 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000CA 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000CB 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000CC 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000CD 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000CE 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000CF 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:13 0000000100000000000000D0 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:13 0000000100000000000000D1 2024-01-11T18:13:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:13 0000000100000000000000D2 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:13 0000000100000000000000D3 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:13 0000000100000000000000D4 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:13 0000000100000000000000D5 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:07 0000000100000000000000D6 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000D7 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000D8 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000D9 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000DA 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000DB 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000DC 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000DD 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:07 0000000100000000000000DE 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:08 0000000100000000000000DF 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:05 0000000100000000000000E0 2024-01-11T18:13:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:08 0000000100000000000000E1 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:07 0000000100000000000000E2 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000E3 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000E4 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000E5 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000E6 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:08 0000000100000000000000E7 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:07 0000000100000000000000E8 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000E9 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000EA 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:07 0000000100000000000000EB 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:06 0000000100000000000000EC 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:08 0000000100000000000000ED 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:07 0000000100000000000000EE 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:11 0000000100000000000000EF 2024-01-11T18:13:44.156 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000F0 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 0000000100000000000000F1 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000F2 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000F3 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 0000000100000000000000F4 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000F5 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000F6 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000F7 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000F8 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:12 0000000100000000000000F9 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000FA 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000FB 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 0000000100000000000000FC 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 0000000100000000000000FD 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:11 0000000100000000000000FE 2024-01-11T18:13:44.157 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 0000000100000000000000FF 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:11 000000010000000100000000 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 000000010000000100000001 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 000000010000000100000002 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:11 000000010000000100000003 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 000000010000000100000004 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 000000010000000100000005 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:10 000000010000000100000006 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jan 11 18:09 000000010000000100000007 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 archive_status 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jan 11 17:58 . 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Jan 11 18:13 .. 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:44.158 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-01-11T18:13:44.159 INFO:teuthology.orchestra.run.smithi007.stdout:total 160 2024-01-11T18:13:44.159 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jan 11 17:58 . 2024-01-11T18:13:44.159 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jan 11 18:13 .. 2024-01-11T18:13:44.159 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Jan 11 18:13 0000 2024-01-11T18:13:44.160 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-01-11T18:13:44.184 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T18:13:44.195 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-01-11T18:13:44.204 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-01-11T18:13:44.204 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_c27531add9155519f390b228e498f0a2c5eae117/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-01-11T18:13:44.205 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-01-11T18:13:44.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 -- 172.21.15.7:0/711297161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0102780 msgr2=0x7fbaa0102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/711297161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0102780 0x7fbaa0102be0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fba940099c0 tx=0x7fba94009cd0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 -- 172.21.15.7:0/711297161 shutdown_connections 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/711297161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0103120 0x7fbaa01099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/711297161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa0101590 0x7fbaa01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/711297161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0102780 0x7fbaa0102be0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 -- 172.21.15.7:0/711297161 >> 172.21.15.7:0/711297161 conn(0x7fbaa00fcb00 msgr2=0x7fbaa00fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 -- 172.21.15.7:0/711297161 shutdown_connections 2024-01-11T18:13:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.318+0000 7fbaa4dff700 1 -- 172.21.15.7:0/711297161 wait complete. 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.319+0000 7fbaa4dff700 1 Processor -- start 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 -- start start 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 0x7fbaa0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa0102780 0x7fbaa0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0103120 0x7fbaa01a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01a1210 con 0x7fbaa0103120 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01a1350 con 0x7fbaa0102780 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fbaa4dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01a14e0 con 0x7fbaa0101590 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 0x7fbaa0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 0x7fbaa0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43238/0 (socket says 172.21.15.7:43238) 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 -- 172.21.15.7:0/3684476990 learned_addr learned my addr 172.21.15.7:0/3684476990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9ed9d700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0103120 0x7fbaa01a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9dd9b700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa0102780 0x7fbaa0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 -- 172.21.15.7:0/3684476990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa0102780 msgr2=0x7fbaa0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa0102780 0x7fbaa0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 -- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0103120 msgr2=0x7fbaa01a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.320+0000 7fba9e59c700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0103120 0x7fbaa01a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fba9e59c700 1 -- 172.21.15.7:0/3684476990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940095f0 con 0x7fbaa0101590 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fba9e59c700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 0x7fbaa0199690 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fba9000e970 tx=0x7fba9000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90009790 con 0x7fbaa0101590 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa01a17c0 con 0x7fbaa0101590 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba9000c7e0 con 0x7fbaa0101590 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90003c70 con 0x7fbaa0101590 2024-01-11T18:13:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.321+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa01a1cd0 con 0x7fbaa0101590 2024-01-11T18:13:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.322+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa00683f0 con 0x7fbaa0101590 2024-01-11T18:13:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.324+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fba9000cc50 con 0x7fbaa0101590 2024-01-11T18:13:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.326+0000 7fba8f7fe700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fba880a1ee0 0x7fba880a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.326+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fba90014070 con 0x7fbaa0101590 2024-01-11T18:13:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.326+0000 7fba9dd9b700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fba880a1ee0 0x7fba880a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.327+0000 7fba9dd9b700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fba880a1ee0 0x7fba880a43a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba94006010 tx=0x7fba9400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.328+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fba90096d30 con 0x7fbaa0101590 2024-01-11T18:13:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.548+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaa010ce00 con 0x7fbaa0101590 2024-01-11T18:13:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.550+0000 7fba8f7fe700 1 -- 172.21.15.7:0/3684476990 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24382 (secure 0 0 0) 0x7fba900b8e60 con 0x7fbaa0101590 2024-01-11T18:13:45.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:45.541 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T18:02:43.786606+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T18:13:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fba880a1ee0 msgr2=0x7fba880a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fba880a1ee0 0x7fba880a43a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba94006010 tx=0x7fba9400b540 comp rx=0 tx=0).stop 2024-01-11T18:13:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 msgr2=0x7fbaa0199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 0x7fbaa0199690 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fba9000e970 tx=0x7fba9000ec80 comp rx=0 tx=0).stop 2024-01-11T18:13:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 shutdown_connections 2024-01-11T18:13:45.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fba880a1ee0 0x7fba880a43a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbaa0101590 0x7fbaa0199690 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaa0103120 0x7fbaa01a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 --2- 172.21.15.7:0/3684476990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaa0102780 0x7fbaa0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 >> 172.21.15.7:0/3684476990 conn(0x7fbaa00fcb00 msgr2=0x7fbaa0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 shutdown_connections 2024-01-11T18:13:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.553+0000 7fbaa4dff700 1 -- 172.21.15.7:0/3684476990 wait complete. 2024-01-11T18:13:45.563 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T18:13:45.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/1816421774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d381015b0 msgr2=0x7f3d381019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/1816421774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d381015b0 0x7f3d381019d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3d280099c0 tx=0x7f3d28009cd0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/1816421774 shutdown_connections 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/1816421774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d38103140 0x7f3d381099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/1816421774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d381015b0 0x7f3d381019d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/1816421774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381027a0 0x7f3d38102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/1816421774 >> 172.21.15.7:0/1816421774 conn(0x7f3d380fcb00 msgr2=0x7f3d380fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:45.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/1816421774 shutdown_connections 2024-01-11T18:13:45.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.783+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/1816421774 wait complete. 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.784+0000 7f3d3d5ff700 1 Processor -- start 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.784+0000 7f3d3d5ff700 1 -- start start 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d3d5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 0x7f3d38197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d3d5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d381027a0 0x7f3d381979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d3d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d38103140 0x7f3d3819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d3d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819f010 con 0x7f3d38103140 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d3d5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819f150 con 0x7f3d381027a0 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d3d5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819f2e0 con 0x7f3d381015b0 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d36ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 0x7f3d38197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d38103140 0x7f3d3819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d36ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 0x7f3d38197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43254/0 (socket says 172.21.15.7:43254) 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d36ffd700 1 -- 172.21.15.7:0/3963085300 learned_addr learned my addr 172.21.15.7:0/3963085300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:45.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.785+0000 7f3d367fc700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d381027a0 0x7f3d381979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d36ffd700 1 -- 172.21.15.7:0/3963085300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d381027a0 msgr2=0x7f3d381979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d36ffd700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d381027a0 0x7f3d381979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d36ffd700 1 -- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d38103140 msgr2=0x7f3d3819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d36ffd700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d38103140 0x7f3d3819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d36ffd700 1 -- 172.21.15.7:0/3963085300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d280095f0 con 0x7f3d381015b0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d36ffd700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 0x7f3d38197490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d28003730 tx=0x7f3d2800f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2801d070 con 0x7f3d381015b0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3819f560 con 0x7f3d381015b0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d280059a0 con 0x7f3d381015b0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28004990 con 0x7f3d381015b0 2024-01-11T18:13:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.786+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3819fa10 con 0x7f3d381015b0 2024-01-11T18:13:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.787+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d381089e0 con 0x7f3d381015b0 2024-01-11T18:13:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.790+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f3d28005210 con 0x7f3d381015b0 2024-01-11T18:13:45.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.791+0000 7f3d1ffff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d200a1ee0 0x7f3d200a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:45.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.791+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3d280ceca0 con 0x7f3d381015b0 2024-01-11T18:13:45.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.791+0000 7f3d367fc700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d200a1ee0 0x7f3d200a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:45.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.792+0000 7f3d367fc700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d200a1ee0 0x7f3d200a43a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009c60 tx=0x7f3d2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:45.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:45.793+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f3d28097a40 con 0x7f3d381015b0 2024-01-11T18:13:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.009+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d3810d090 con 0x7f3d381015b0 2024-01-11T18:13:46.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.010+0000 7f3d1ffff700 1 -- 172.21.15.7:0/3963085300 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d2809b060 con 0x7f3d381015b0 2024-01-11T18:13:46.000 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T18:13:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d200a1ee0 msgr2=0x7f3d200a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d200a1ee0 0x7f3d200a43a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009c60 tx=0x7f3d2c009260 comp rx=0 tx=0).stop 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 msgr2=0x7f3d38197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 0x7f3d38197490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d28003730 tx=0x7f3d2800f670 comp rx=0 tx=0).stop 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 shutdown_connections 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f3d200a1ee0 0x7f3d200a43a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d381015b0 0x7f3d38197490 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d38103140 0x7f3d3819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 --2- 172.21.15.7:0/3963085300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d381027a0 0x7f3d381979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 >> 172.21.15.7:0/3963085300 conn(0x7f3d380fcb00 msgr2=0x7f3d38105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 shutdown_connections 2024-01-11T18:13:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.013+0000 7f3d3d5ff700 1 -- 172.21.15.7:0/3963085300 wait complete. 2024-01-11T18:13:46.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T18:13:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 -- 172.21.15.7:0/903456046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241015b0 msgr2=0x7f0b241019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 --2- 172.21.15.7:0/903456046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241015b0 0x7f0b241019d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0b140099c0 tx=0x7f0b14009cd0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 -- 172.21.15.7:0/903456046 shutdown_connections 2024-01-11T18:13:46.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 --2- 172.21.15.7:0/903456046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b24103140 0x7f0b241099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 --2- 172.21.15.7:0/903456046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241015b0 0x7f0b241019d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 --2- 172.21.15.7:0/903456046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b241027a0 0x7f0b24102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.240+0000 7f0b28fff700 1 -- 172.21.15.7:0/903456046 >> 172.21.15.7:0/903456046 conn(0x7f0b240fcb00 msgr2=0x7f0b240fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.241+0000 7f0b28fff700 1 -- 172.21.15.7:0/903456046 shutdown_connections 2024-01-11T18:13:46.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.241+0000 7f0b28fff700 1 -- 172.21.15.7:0/903456046 wait complete. 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 Processor -- start 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 -- start start 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b241015b0 0x7f0b241996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 0x7f0b24199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b24103140 0x7f0b241a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241a1240 con 0x7f0b241027a0 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241a1380 con 0x7f0b24103140 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b28fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241a1510 con 0x7f0b241015b0 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b21d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 0x7f0b24199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b2259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b241015b0 0x7f0b241996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b21d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 0x7f0b24199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41108/0 (socket says 172.21.15.7:41108) 2024-01-11T18:13:46.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b21d9b700 1 -- 172.21.15.7:0/2514333003 learned_addr learned my addr 172.21.15.7:0/2514333003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.242+0000 7f0b22d9d700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b24103140 0x7f0b241a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b21d9b700 1 -- 172.21.15.7:0/2514333003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b241015b0 msgr2=0x7f0b241996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b21d9b700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b241015b0 0x7f0b241996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b21d9b700 1 -- 172.21.15.7:0/2514333003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b24103140 msgr2=0x7f0b241a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b21d9b700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b24103140 0x7f0b241a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b21d9b700 1 -- 172.21.15.7:0/2514333003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140095f0 con 0x7f0b241027a0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b21d9b700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 0x7f0b24199c00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800e9a0 tx=0x7f0b1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18009790 con 0x7f0b241027a0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b18010430 con 0x7f0b241027a0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18018680 con 0x7f0b241027a0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b241a17f0 con 0x7f0b241027a0 2024-01-11T18:13:46.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.243+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b241a1d00 con 0x7f0b241027a0 2024-01-11T18:13:46.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:45 smithi016 ceph-mon[43195]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 643 op/s 2024-01-11T18:13:46.236 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:45 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3684476990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:13:46.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.244+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b240683f0 con 0x7f0b241027a0 2024-01-11T18:13:46.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.246+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0b180105a0 con 0x7f0b241027a0 2024-01-11T18:13:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.248+0000 7f0b137fe700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0b0c0a1ee0 0x7f0b0c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.248+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0b18014070 con 0x7f0b241027a0 2024-01-11T18:13:46.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.248+0000 7f0b2259c700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0b0c0a1ee0 0x7f0b0c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.249+0000 7f0b2259c700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0b0c0a1ee0 0x7f0b0c0a43a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b14003730 tx=0x7f0b14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:46.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.252+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0b18052380 con 0x7f0b241027a0 2024-01-11T18:13:46.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:45 smithi038 ceph-mon[43582]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 643 op/s 2024-01-11T18:13:46.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:45 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3684476990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:13:46.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:45 smithi007 ceph-mon[33361]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 643 op/s 2024-01-11T18:13:46.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:45 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3684476990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:13:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.467+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b241a2130 con 0x7f0b241027a0 2024-01-11T18:13:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.467+0000 7f0b137fe700 1 -- 172.21.15.7:0/2514333003 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b180d3020 con 0x7f0b241027a0 2024-01-11T18:13:46.458 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0b0c0a1ee0 msgr2=0x7f0b0c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0b0c0a1ee0 0x7f0b0c0a43a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b14003730 tx=0x7f0b14000bc0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 msgr2=0x7f0b24199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 0x7f0b24199c00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800e9a0 tx=0x7f0b1800ecb0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 shutdown_connections 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0b0c0a1ee0 0x7f0b0c0a43a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b241015b0 0x7f0b241996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b24103140 0x7f0b241a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 --2- 172.21.15.7:0/2514333003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b241027a0 0x7f0b24199c00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.469+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 >> 172.21.15.7:0/2514333003 conn(0x7f0b240fcb00 msgr2=0x7f0b24073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.470+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 shutdown_connections 2024-01-11T18:13:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.470+0000 7f0b28fff700 1 -- 172.21.15.7:0/2514333003 wait complete. 2024-01-11T18:13:46.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 -- 172.21.15.7:0/189902173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 msgr2=0x7fc274073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/189902173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc274073aa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc264009990 tx=0x7fc264009ca0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 -- 172.21.15.7:0/189902173 shutdown_connections 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/189902173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc27410db80 0x7fc27410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/189902173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc274073aa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/189902173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc274074070 0x7fc2740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 -- 172.21.15.7:0/189902173 >> 172.21.15.7:0/189902173 conn(0x7fc2740fcb00 msgr2=0x7fc2740fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 -- 172.21.15.7:0/189902173 shutdown_connections 2024-01-11T18:13:46.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.697+0000 7fc27a5ff700 1 -- 172.21.15.7:0/189902173 wait complete. 2024-01-11T18:13:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.698+0000 7fc27a5ff700 1 Processor -- start 2024-01-11T18:13:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.698+0000 7fc27a5ff700 1 -- start start 2024-01-11T18:13:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc27a5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc274074070 0x7fc27419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc27a5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc27419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc27a5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27410db80 0x7fc2741a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc27a5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741a55e0 con 0x7fc274075650 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc27a5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741a5720 con 0x7fc27410db80 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc27a5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741a58b0 con 0x7fc274074070 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc27419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc27419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41114/0 (socket says 172.21.15.7:41114) 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 -- 172.21.15.7:0/3849005288 learned_addr learned my addr 172.21.15.7:0/3849005288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc278b9c700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27410db80 0x7fc2741a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc273fff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc274074070 0x7fc27419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 -- 172.21.15.7:0/3849005288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc274074070 msgr2=0x7fc27419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc274074070 0x7fc27419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 -- 172.21.15.7:0/3849005288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27410db80 msgr2=0x7fc2741a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27410db80 0x7fc2741a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.699+0000 7fc2737fe700 1 -- 172.21.15.7:0/3849005288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2640095f0 con 0x7fc274075650 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.700+0000 7fc2737fe700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc27419dfa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc26800e970 tx=0x7fc26800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.700+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268009790 con 0x7fc274075650 2024-01-11T18:13:46.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.700+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc26800c7e0 con 0x7fc274075650 2024-01-11T18:13:46.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.700+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2741a5b90 con 0x7fc274075650 2024-01-11T18:13:46.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.700+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268003c70 con 0x7fc274075650 2024-01-11T18:13:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.700+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2741a60a0 con 0x7fc274075650 2024-01-11T18:13:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.701+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274103de0 con 0x7fc274075650 2024-01-11T18:13:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.703+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7fc26800cc50 con 0x7fc274075650 2024-01-11T18:13:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.704+0000 7fc2717fa700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc25c0a1ee0 0x7fc25c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.704+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc268014070 con 0x7fc274075650 2024-01-11T18:13:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.705+0000 7fc273fff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc25c0a1ee0 0x7fc25c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.705+0000 7fc273fff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc25c0a1ee0 0x7fc25c0a43a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b2e0 tx=0x7fc264005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:46.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.708+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7fc268096bc0 con 0x7fc274075650 2024-01-11T18:13:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.922+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc27402d460 con 0x7fc274075650 2024-01-11T18:13:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.922+0000 7fc2717fa700 1 -- 172.21.15.7:0/3849005288 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc268003dd0 con 0x7fc274075650 2024-01-11T18:13:46.913 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.924+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc25c0a1ee0 msgr2=0x7fc25c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.924+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc25c0a1ee0 0x7fc25c0a43a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b2e0 tx=0x7fc264005fd0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.924+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 msgr2=0x7fc27419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.924+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc27419dfa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc26800e970 tx=0x7fc26800ec80 comp rx=0 tx=0).stop 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 shutdown_connections 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7fc25c0a1ee0 0x7fc25c0a43a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc274074070 0x7fc27419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27410db80 0x7fc2741a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 --2- 172.21.15.7:0/3849005288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc274075650 0x7fc27419dfa0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 >> 172.21.15.7:0/3849005288 conn(0x7fc2740fcb00 msgr2=0x7fc274100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 shutdown_connections 2024-01-11T18:13:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:46.925+0000 7fc27a5ff700 1 -- 172.21.15.7:0/3849005288 wait complete. 2024-01-11T18:13:46.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 -- 172.21.15.7:0/3698706513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c0074010 msgr2=0x7f72c0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 --2- 172.21.15.7:0/3698706513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c0074010 0x7f72c0074490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009990 tx=0x7f72b4009ca0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 -- 172.21.15.7:0/3698706513 shutdown_connections 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 --2- 172.21.15.7:0/3698706513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c00755f0 0x7f72c0073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 --2- 172.21.15.7:0/3698706513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c0109490 0x7f72c010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 --2- 172.21.15.7:0/3698706513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c0074010 0x7f72c0074490 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 -- 172.21.15.7:0/3698706513 >> 172.21.15.7:0/3698706513 conn(0x7f72c00fcb30 msgr2=0x7f72c00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 -- 172.21.15.7:0/3698706513 shutdown_connections 2024-01-11T18:13:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.154+0000 7f72c55ff700 1 -- 172.21.15.7:0/3698706513 wait complete. 2024-01-11T18:13:47.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.155+0000 7f72c55ff700 1 Processor -- start 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 -- start start 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0074010 0x7f72c0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c00755f0 0x7f72c0199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c0109490 0x7f72c01a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01a1210 con 0x7f72c00755f0 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01a1350 con 0x7f72c0109490 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72c55ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01a14e0 con 0x7f72c0074010 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72beffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0074010 0x7f72c0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72beffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0074010 0x7f72c0199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43310/0 (socket says 172.21.15.7:43310) 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72beffd700 1 -- 172.21.15.7:0/4041045276 learned_addr learned my addr 172.21.15.7:0/4041045276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:47.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72be7fc700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c00755f0 0x7f72c0199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.156+0000 7f72bf7fe700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c0109490 0x7f72c01a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72be7fc700 1 -- 172.21.15.7:0/4041045276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0074010 msgr2=0x7f72c0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72be7fc700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0074010 0x7f72c0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72be7fc700 1 -- 172.21.15.7:0/4041045276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c0109490 msgr2=0x7f72c01a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72be7fc700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c0109490 0x7f72c01a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72be7fc700 1 -- 172.21.15.7:0/4041045276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40095f0 con 0x7f72c00755f0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72be7fc700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c00755f0 0x7f72c0199c70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f72b400b680 tx=0x7f72b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b401d070 con 0x7f72c00755f0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b4021410 con 0x7f72c00755f0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01a1760 con 0x7f72c00755f0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4017650 con 0x7f72c00755f0 2024-01-11T18:13:47.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.157+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c01a1b80 con 0x7f72c00755f0 2024-01-11T18:13:47.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.158+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c01082a0 con 0x7f72c00755f0 2024-01-11T18:13:47.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.160+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f72b400fb10 con 0x7f72c00755f0 2024-01-11T18:13:47.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.161+0000 7f72a7fff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f72a80a1ee0 0x7f72a80a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.162+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f72b40cef00 con 0x7f72c00755f0 2024-01-11T18:13:47.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.162+0000 7f72beffd700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f72a80a1ee0 0x7f72a80a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.163+0000 7f72beffd700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f72a80a1ee0 0x7f72a80a43a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009d20 tx=0x7f72b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:47.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.166+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f72b4097ca0 con 0x7f72c00755f0 2024-01-11T18:13:47.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:46 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3963085300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:46 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2514333003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:46 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3849005288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:46 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3963085300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:46 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2514333003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:46 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3849005288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:46 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3963085300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:46 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2514333003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:46 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3849005288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T18:13:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.378+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f72c002d3f0 con 0x7f72c00755f0 2024-01-11T18:13:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.379+0000 7f72a7fff700 1 -- 172.21.15.7:0/4041045276 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72b4021e60 con 0x7f72c00755f0 2024-01-11T18:13:47.369 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f72a80a1ee0 msgr2=0x7f72a80a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f72a80a1ee0 0x7f72a80a43a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009d20 tx=0x7f72b0009260 comp rx=0 tx=0).stop 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c00755f0 msgr2=0x7f72c0199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c00755f0 0x7f72c0199c70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f72b400b680 tx=0x7f72b4004730 comp rx=0 tx=0).stop 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 shutdown_connections 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f72a80a1ee0 0x7f72a80a43a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0074010 0x7f72c0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c0109490 0x7f72c01a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 --2- 172.21.15.7:0/4041045276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72c00755f0 0x7f72c0199c70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 >> 172.21.15.7:0/4041045276 conn(0x7f72c00fcb30 msgr2=0x7f72c01053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.381+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 shutdown_connections 2024-01-11T18:13:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.382+0000 7f72c55ff700 1 -- 172.21.15.7:0/4041045276 wait complete. 2024-01-11T18:13:47.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 -- 172.21.15.7:0/793633155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c101590 msgr2=0x7f036c1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 --2- 172.21.15.7:0/793633155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c101590 0x7f036c1019b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03680099c0 tx=0x7f0368009cd0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 -- 172.21.15.7:0/793633155 shutdown_connections 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 --2- 172.21.15.7:0/793633155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c103120 0x7f036c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 --2- 172.21.15.7:0/793633155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c101590 0x7f036c1019b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 --2- 172.21.15.7:0/793633155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c102780 0x7f036c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 -- 172.21.15.7:0/793633155 >> 172.21.15.7:0/793633155 conn(0x7f036c0fcb00 msgr2=0x7f036c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 -- 172.21.15.7:0/793633155 shutdown_connections 2024-01-11T18:13:47.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.606+0000 7f0373fff700 1 -- 172.21.15.7:0/793633155 wait complete. 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.607+0000 7f0373fff700 1 Processor -- start 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.607+0000 7f0373fff700 1 -- start start 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0373fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c101590 0x7f036c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0373fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c102780 0x7f036c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0373fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c103120 0x7f036c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0373fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1a1200 con 0x7f036c103120 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0373fff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1a1340 con 0x7f036c101590 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0373fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1a14d0 con 0x7f036c102780 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f037259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c103120 0x7f036c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f037159a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c102780 0x7f036c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0371d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c101590 0x7f036c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f037159a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c102780 0x7f036c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:43322/0 (socket says 172.21.15.7:43322) 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f037159a700 1 -- 172.21.15.7:0/2598371976 learned_addr learned my addr 172.21.15.7:0/2598371976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0371d9b700 1 -- 172.21.15.7:0/2598371976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c102780 msgr2=0x7f036c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0371d9b700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c102780 0x7f036c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0371d9b700 1 -- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c103120 msgr2=0x7f036c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0371d9b700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c103120 0x7f036c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.608+0000 7f0371d9b700 1 -- 172.21.15.7:0/2598371976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680095f0 con 0x7f036c101590 2024-01-11T18:13:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.609+0000 7f0371d9b700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c101590 0x7f036c199680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03680098e0 tx=0x7f036800f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:47.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.609+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036801d070 con 0x7f036c101590 2024-01-11T18:13:47.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.609+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c1a1750 con 0x7f036c101590 2024-01-11T18:13:47.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.609+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03680059a0 con 0x7f036c101590 2024-01-11T18:13:47.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.609+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368004950 con 0x7f036c101590 2024-01-11T18:13:47.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.609+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c1a1bc0 con 0x7f036c101590 2024-01-11T18:13:47.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.610+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c0683f0 con 0x7f036c101590 2024-01-11T18:13:47.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.612+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0368005200 con 0x7f036c101590 2024-01-11T18:13:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.614+0000 7f0362ffd700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f03580a1ee0 0x7f03580a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:13:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.614+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f03680ce7c0 con 0x7f036c101590 2024-01-11T18:13:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.614+0000 7f037159a700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f03580a1ee0 0x7f03580a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:13:47.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.614+0000 7f037159a700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f03580a1ee0 0x7f03580a43a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f035c009d20 tx=0x7f035c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:13:47.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.616+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0368097560 con 0x7f036c101590 2024-01-11T18:13:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.889+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f036c10cdb0 con 0x7f036c101590 2024-01-11T18:13:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.900+0000 7f0362ffd700 1 -- 172.21.15.7:0/2598371976 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+736080 (secure 0 0 0) 0x7f036809ab80 con 0x7f036c101590 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 18, 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 14152, 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T18:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 314922909 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 314922909 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/314922909", 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-01-11T17:50:24.497439+0000", 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701272860131327, 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24112, 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701272860131327, 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-01-11T18:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-01-11T18:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-01-11T18:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-01-11T18:13:47.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.925 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:47.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.930 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap" 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:47.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-01-11T18:13:47.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-01-11T18:13:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.950 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-01-11T18:13:47.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-01-11T18:13:47.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-11T18:13:47.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-01-11T18:13:47.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-01-11T18:13:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:47.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.967 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:47.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-11T18:13:47.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-11T18:13:47.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.983 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.984 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-01-11T18:13:47.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-01-11T18:13:47.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:47.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:47.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:47.993 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-01-11T18:13:47.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-01-11T18:13:47.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-11T18:13:47.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-11T18:13:47.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:47.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.009 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-01-11T18:13:48.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-11T18:13:48.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.019 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-11T18:13:48.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-11T18:13:48.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.035 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-11T18:13:48.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-11T18:13:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-01-11T18:13:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-01-11T18:13:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-01-11T18:13:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-01-11T18:13:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-01-11T18:13:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-01-11T18:13:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.121 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.126 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.129 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.131 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.135 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.137 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.141 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.165 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-11T18:13:48.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.172 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-01-11T18:13:48.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-01-11T18:13:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-01-11T18:13:48.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-01-11T18:13:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-01-11T18:13:48.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.184 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "drive_group_interval": { 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "drive_group_interval", 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300.0", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-01-11T18:13:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-01-11T18:13:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-11T18:13:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-01-11T18:13:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-01-11T18:13:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-01-11T18:13:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.296 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24119, 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701272860131327, 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-01-11T18:13:48.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.307 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-01-11T18:13:48.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-01-11T18:13:48.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.311 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.313 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-01-11T18:13:48.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap" 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-01-11T18:13:48.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-11T18:13:48.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-11T18:13:48.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-11T18:13:48.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-01-11T18:13:48.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-11T18:13:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.349 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-01-11T18:13:48.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-01-11T18:13:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-01-11T18:13:48.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-01-11T18:13:48.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-01-11T18:13:48.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-01-11T18:13:48.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-01-11T18:13:48.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-01-11T18:13:48.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.387 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-11T18:13:48.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.414 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-11T18:13:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-11T18:13:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-01-11T18:13:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-01-11T18:13:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-01-11T18:13:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-01-11T18:13:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-01-11T18:13:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-01-11T18:13:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-01-11T18:13:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-01-11T18:13:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-01-11T18:13:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-01-11T18:13:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-01-11T18:13:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.550 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.554 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.556 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.557 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.559 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-01-11T18:13:48.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-01-11T18:13:48.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.568 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.570 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.595 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "drive_group_interval": { 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "drive_group_interval", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300.0", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.601 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.605 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.613 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-01-11T18:13:48.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-01-11T18:13:48.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.626 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.633 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.638 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-01-11T18:13:48.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.643 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.655 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-01-11T18:13:48.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.661 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.662 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.668 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.674 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-11T18:13:48.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "restful" 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-01-11T18:13:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-01-11T18:13:48.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-01-11T18:13:48.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-01-11T18:13:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-11T18:13:48.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.715 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.718 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-01-11T18:13:48.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap" 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-01-11T18:13:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-01-11T18:13:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-01-11T18:13:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-11T18:13:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-01-11T18:13:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-01-11T18:13:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-01-11T18:13:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-01-11T18:13:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-01-11T18:13:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-01-11T18:13:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-01-11T18:13:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-01-11T18:13:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-01-11T18:13:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-01-11T18:13:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-01-11T18:13:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-01-11T18:13:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-01-11T18:13:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-11T18:13:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-11T18:13:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-11T18:13:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-11T18:13:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.825 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-11T18:13:48.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.830 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.833 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.834 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-01-11T18:13:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.848 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.853 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-01-11T18:13:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.862 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-01-11T18:13:48.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.871 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.875 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-01-11T18:13:48.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-01-11T18:13:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-01-11T18:13:48.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.881 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.889 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-01-11T18:13:48.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-01-11T18:13:48.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-11T18:13:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-01-11T18:13:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-01-11T18:13:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-01-11T18:13:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-01-11T18:13:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-01-11T18:13:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "drive_group_interval": { 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "drive_group_interval", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300.0", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-01-11T18:13:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-01-11T18:13:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-01-11T18:13:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-01-11T18:13:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-01-11T18:13:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-01-11T18:13:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-01-11T18:13:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-01-11T18:13:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-01-11T18:13:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-01-11T18:13:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-01-11T18:13:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-01-11T18:13:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-01-11T18:13:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 3, 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2542526837 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-01-11T18:13:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2039092284 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-01-11T18:13:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2434319049 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 206690448 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T18:13:49.093 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T18:13:49.094 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:47 smithi038 ceph-mon[43582]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.4 MiB/s wr, 591 op/s 2024-01-11T18:13:49.094 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:47 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4041045276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T18:13:49.094 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:47 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2598371976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-11T18:13:49.094 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:47 smithi016 ceph-mon[43195]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.4 MiB/s wr, 591 op/s 2024-01-11T18:13:49.094 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:47 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4041045276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T18:13:49.094 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:47 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2598371976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-11T18:13:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f03580a1ee0 msgr2=0x7f03580a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f03580a1ee0 0x7f03580a43a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f035c009d20 tx=0x7f035c009260 comp rx=0 tx=0).stop 2024-01-11T18:13:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c101590 msgr2=0x7f036c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:13:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c101590 0x7f036c199680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03680098e0 tx=0x7f036800f670 comp rx=0 tx=0).stop 2024-01-11T18:13:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 shutdown_connections 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f03580a1ee0 0x7f03580a43a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f036c103120 0x7f036c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f036c101590 0x7f036c199680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 --2- 172.21.15.7:0/2598371976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f036c102780 0x7f036c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 >> 172.21.15.7:0/2598371976 conn(0x7f036c0fcb00 msgr2=0x7f036c107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 shutdown_connections 2024-01-11T18:13:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:13:47.907+0000 7f0373fff700 1 -- 172.21.15.7:0/2598371976 wait complete. 2024-01-11T18:13:49.101 DEBUG:tasks.check_counter:Getting stats from a 2024-01-11T18:13:49.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.a perf dump 2024-01-11T18:13:49.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:47 smithi007 ceph-mon[33361]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.4 MiB/s wr, 591 op/s 2024-01-11T18:13:49.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:47 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4041045276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T18:13:49.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:47 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2598371976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-11T18:13:49.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:49 smithi016 ceph-mon[43195]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-01-11T18:13:49.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:49 smithi038 ceph-mon[43582]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-01-11T18:13:49.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:49 smithi007 ceph-mon[33361]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-01-11T18:13:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T18:13:51.184 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 5222, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4585, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2291093, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 6438922, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 18, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 7, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.661224567, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.230082211, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.424891604, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.104055252, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4581, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.105008834, 2024-01-11T18:13:51.185 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000241215 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 6301, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.002917407, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000463 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 7398, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 7575, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2416994, 2024-01-11T18:13:51.186 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 4073787, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 9, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 5, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 1.778884630, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 1.323643436, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.686682657, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.001990212, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 7576, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.833847721, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000242060 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 9436, 2024-01-11T18:13:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.005309104, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000562 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 4736, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 453904 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 668, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 645, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 728080, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 388808, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 7, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 6, 2024-01-11T18:13:51.188 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.120779598, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.036688061, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.040862190, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.033777018, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 644, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.192155864, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000298378 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 621760, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 213984 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 1, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-MDSRank": { 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 2973, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 2.212168454, 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000744086 2024-01-11T18:13:51.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-PurgeQueue": { 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 98, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.010557789, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000107732 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "request": 2019, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "reply": 1974, 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "reply_latency": { 2024-01-11T18:13:51.191 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1974, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 6.423336130, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.003253969 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_reply": 0, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "forward": 45, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_complete": 2, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_keys": 0, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_commit": 11, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_split": 0, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_merge": 0, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes": 951, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_top": 18, 2024-01-11T18:13:51.192 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_bottom": 768, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pin_tail": 165, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pinned": 924, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_expired": 712, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_with_caps": 497, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": 497, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "subtrees": 20, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse": 5622, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_hit": 4346, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_forward": 37, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_discover": 126, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_dir_fetch": 0, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_remote_ino": 0, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_lock": 385, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "load_cent": 46706, 2024-01-11T18:13:51.193 INFO:teuthology.orchestra.run.smithi007.stdout: "q": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "exported": 32, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "exported_inodes": 407, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "imported": 30, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "imported_inodes": 843, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_dir_fetch": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_backtrace_fetch": 247, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_peer_discover": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rfiles": 1038, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rbytes": 1600818061, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rsnaps": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_fetch": 247, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_set_tag": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_repaired": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_inotable_repaired": 0, 2024-01-11T18:13:51.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_inodes": 6, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_base_inodes": 0, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dirfrag_rstats": 6, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_file_inodes": 241, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_inode_file_caps": 1, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_revoke": 420, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_grant": 1101, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_trunc": 0, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps": 1487, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps_dirty": 718, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_cap_release": 78, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "process_request_cap_release": 1225 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache": { 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update": 129, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update_receipt": 114, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_try_discover": 67, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_send_discover": 192, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_handle_discover": 148, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays": 0, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_delayed": 0, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_enqueuing": 0, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_created": 282, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_enqueued": 25, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_reintegrated": 257, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_migrated": 0, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_processing": 0, 2024-01-11T18:13:51.196 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_enqueued": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_prioritized": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_started": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_completed": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_enqueue_scrub": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_exportdir": 32, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_flush": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragmentdir": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragstats": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_inodestats": 0 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log": { 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "evlrg": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "evadd": 2402, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "evex": 0, 2024-01-11T18:13:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: "evtrm": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "ev": 2402, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "evexg": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "evexd": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "segadd": 5, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "segex": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "segtrm": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "seg": 5, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "segmjr": 2, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "segexg": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "segexd": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "expos": 4194304, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "wrpos": 8876774, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "rdpos": 0, 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "jlat": { 2024-01-11T18:13:51.198 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1385, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 8.959961934, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.006469286 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "replayed": 0 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mem": { 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 544, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "ino+": 1103, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "ino-": 559, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "dir": 68, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "dir+": 109, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "dir-": 41, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "dn": 951, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "dn+": 1693, 2024-01-11T18:13:51.199 INFO:teuthology.orchestra.run.smithi007.stdout: "dn-": 742, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "cap": 497, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "cap+": 1034, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "cap-": 537, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "rss": 58720, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "heap": 199036 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server": { 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_client_request": 3286, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_server_request": 288, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_request": 2019, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_session": 541, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_peer_request": 423, 2024-01-11T18:13:51.200 INFO:teuthology.orchestra.run.smithi007.stdout: "req_create_latency": { 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 86, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.113721914, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001322347 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getattr_latency": { 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 11, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013866201, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001260563 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getfilelock_latency": { 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "req_link_latency": { 2024-01-11T18:13:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 257, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.752873469, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.002929468 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookup_latency": { 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 901, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.891355205, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.002099173 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookuphash_latency": { 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupino_latency": { 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupname_latency": { 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupparent_latency": { 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupsnap_latency": { 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lssnap_latency": { 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mkdir_latency": { 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mknod_latency": { 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mksnap_latency": { 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "req_open_latency": { 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 152, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.207094123, 2024-01-11T18:13:51.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001362461 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "req_readdir_latency": { 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 26, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.036998215, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001423008 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rename_latency": { 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 283, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 2.732944646, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.009657048 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "req_renamesnap_latency": { 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.206 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "req_snapdiff_latency": { 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmdir_latency": { 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmsnap_latency": { 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.207 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmxattr_latency": { 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setattr_latency": { 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setdirlayout_latency": { 2024-01-11T18:13:51.208 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setfilelock_latency": { 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setlayout_latency": { 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setxattr_latency": { 2024-01-11T18:13:51.209 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "req_symlink_latency": { 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "req_unlink_latency": { 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 258, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.674482357, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.002614272 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_revoke_eviction": 0, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_acquisition_throttle": 0, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getvxattr_latency": { 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessions": { 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "session_count": 1, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "session_add": 2, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "session_remove": 1, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_open": 1, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_stale": 0, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "total_load": 278, 2024-01-11T18:13:51.211 INFO:teuthology.orchestra.run.smithi007.stdout: "average_load": 278, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "avg_session_uptime": 950, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "mdthresh_evicted": 0 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:13:51.212 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:13:51.213 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 370035, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 631, 2024-01-11T18:13:51.214 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 8792, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 168, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 2225990, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 34765, 2024-01-11T18:13:51.215 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter": { 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_active": 0, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_laggy": 0, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send": 1925, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send_bytes": 4759411, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_resend": 27, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_reply": 1925, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "op_latency": { 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1925, 2024-01-11T18:13:51.216 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 16.350757287, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.008493899 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "op_inflight": 0, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "oplen_avg": { 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1925, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 2093 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "op": 1925, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "op_r": 260, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "op_w": 1665, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "op_rmw": 0, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "op_pg": 0, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_stat": 0, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_create": 10, 2024-01-11T18:13:51.217 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_read": 11, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_write": 1396, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writefull": 71, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writesame": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_append": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_zero": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_truncate": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_delete": 63, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_mapext": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_sparse_read": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_clonerange": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_getxattr": 247, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_setxattr": 20, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_cmpxattr": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_rmxattr": 0, 2024-01-11T18:13:51.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_resetxattrs": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_call": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_watch": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_notify": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_src_cmpxattr": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls_filter": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_other": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_active": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_send": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_resend": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_ping": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_active": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_send": 0, 2024-01-11T18:13:51.219 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_resend": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_active": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_send": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_resend": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_active": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_send": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_resend": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "command_active": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "command_send": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "command_resend": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "map_epoch": 89, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "map_full": 0, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "map_inc": 3, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_sessions": 12, 2024-01-11T18:13:51.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_open": 12, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_close": 0, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_laggy": 0, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_wr": 196, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_rd": 4, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_del": 75 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "oft": { 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_objs": 1, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_kv_pairs": 0, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_updates": 0, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_removes": 0 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "osd-slow-ops": { 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_ops_count": 0 2024-01-11T18:13:51.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "purge_queue": { 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops": 0, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops_high_water": 16, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing": 0, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_high_water": 8, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed": 25, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_item_in_journal": 0 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 13284, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 4286544, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:51.222 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 13284, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 13284, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4286544, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_bytes": { 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:51.223 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 1925, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 4762104, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 1925, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4762104, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_ops": { 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 1024, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 1925, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 1925, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 1925, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1925, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.225 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle": { 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 25, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 2725, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 25, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 11, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 2725, 2024-01-11T18:13:51.226 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle-0x55fb5c04c640": { 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 1846, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 2402, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 4682470, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 2402, 2024-01-11T18:13:51.227 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 1385, 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4680624, 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:51.228 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T18:13:51.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:51 smithi016 ceph-mon[43195]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-01-11T18:13:51.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:51 smithi038 ceph-mon[43582]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-01-11T18:13:51.539 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:51 smithi007 ceph-mon[33361]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-01-11T18:13:51.574 INFO:tasks.check_counter:Daemon mds.a mds.exported=32 2024-01-11T18:13:51.575 INFO:tasks.check_counter:Daemon mds.a mds.imported=30 2024-01-11T18:13:51.575 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=129 2024-01-11T18:13:51.575 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=114 2024-01-11T18:13:51.575 DEBUG:tasks.check_counter:Getting stats from d 2024-01-11T18:13:51.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.d perf dump 2024-01-11T18:13:52.004 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T18:13:53.201 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.003804256, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000903030, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:53.202 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 293, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 290, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 496032, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 174576, 2024-01-11T18:13:53.203 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.077049280, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.042121363, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.022835238, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.009138648, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 289, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.073245532, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000253444 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:53.204 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 496032, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 174576 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 6, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 224, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3312, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 193568, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.032647878, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.026995568, 2024-01-11T18:13:53.205 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002674705, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 223, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.054991280, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000246597 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 193568 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:13:53.207 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-01-11T18:13:53.208 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149735, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-01-11T18:13:53.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-01-11T18:13:53.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 295, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 454639, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 295, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 295, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 454639, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:53.211 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:53.212 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:53.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:53.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:53.212 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T18:13:53.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:53 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:13:53.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:53 smithi016 ceph-mon[43195]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-01-11T18:13:53.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:53 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:13:53.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:53 smithi038 ceph-mon[43582]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-01-11T18:13:53.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:53 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:13:53.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:53 smithi007 ceph-mon[33361]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-01-11T18:13:53.586 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-01-11T18:13:53.586 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-01-11T18:13:53.586 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-01-11T18:13:53.586 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-01-11T18:13:53.586 DEBUG:tasks.check_counter:Getting stats from g 2024-01-11T18:13:53.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.g perf dump 2024-01-11T18:13:54.016 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004259502, 2024-01-11T18:13:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000898318, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-01-11T18:13:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 220, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 192768, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.040284461, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.034096109, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003146925, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 218, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.050904308, 2024-01-11T18:13:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000233506 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 192768 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 295, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 290, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 497920, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 173600, 2024-01-11T18:13:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.089499280, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.052574259, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.022363366, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.010114818, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 290, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.071558081, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000246752 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 497920, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 173600 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:13:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:13:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149735, 2024-01-11T18:13:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-01-11T18:13:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 293, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 453806, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 293, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 293, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 453806, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:55.302 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T18:13:55.778 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-01-11T18:13:55.779 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-01-11T18:13:55.779 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-01-11T18:13:55.779 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-01-11T18:13:55.779 DEBUG:tasks.check_counter:Getting stats from j 2024-01-11T18:13:55.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.j perf dump 2024-01-11T18:13:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.a/config 2024-01-11T18:13:56.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:55 smithi016 ceph-mon[43195]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 543 KiB/s wr, 64 op/s 2024-01-11T18:13:56.266 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:55 smithi038 ceph-mon[43582]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 543 KiB/s wr, 64 op/s 2024-01-11T18:13:56.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:55 smithi007 ceph-mon[33361]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 543 KiB/s wr, 64 op/s 2024-01-11T18:13:57.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:56 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:13:57.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:56 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:13:57.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:56 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4, 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2992, 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 448, 2024-01-11T18:13:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004393873, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000102486, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001734989, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.001344334, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000448111 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 223, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 193296, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.029588678, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.023666036, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003175419, 2024-01-11T18:13:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 221, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.040574750, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000183596 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 193296 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:13:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 296, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 293, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 496912, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 175984, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.086499057, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.049016187, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.023904543, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.010624849, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 292, 2024-01-11T18:13:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.059292689, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000203057 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 496912, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 175984 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-01-11T18:13:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:13:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-01-11T18:13:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149735, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-01-11T18:13:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-01-11T18:13:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 298, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 455093, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 298, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-01-11T18:13:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 298, 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 455093, 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T18:13:57.526 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T18:13:57.889 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-01-11T18:13:57.889 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-01-11T18:13:57.890 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-01-11T18:13:57.890 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-01-11T18:13:57.890 DEBUG:tasks.check_counter:Getting stats from b 2024-01-11T18:13:57.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.b perf dump 2024-01-11T18:13:58.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:58 smithi038 ceph-mon[43582]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:13:58.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:58 smithi038 ceph-mon[43582]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 177 KiB/s wr, 26 op/s 2024-01-11T18:13:58.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:58 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:13:58.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:58 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:13:58.265 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:58 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:13:58.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:58 smithi007 ceph-mon[33361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:13:58.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:58 smithi007 ceph-mon[33361]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 177 KiB/s wr, 26 op/s 2024-01-11T18:13:58.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:13:58.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:13:58.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:58 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:13:58.336 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T18:13:58.439 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:58 smithi016 ceph-mon[43195]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-01-11T18:13:58.440 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:58 smithi016 ceph-mon[43195]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 177 KiB/s wr, 26 op/s 2024-01-11T18:13:58.440 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:58 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-11T18:13:58.440 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:58 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-11T18:13:58.440 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:58 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' 2024-01-11T18:13:59.276 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:13:59 smithi016 ceph-mon[43195]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-01-11T18:13:59.448 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 7477, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 9934, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2325516, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 15095347, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 163, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 9, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.662752566, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.213885397, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.328944217, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.135468948, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 9930, 2024-01-11T18:13:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3.364804968, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000338852 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5929, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002264224, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000381 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 53056, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 29792 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3928, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4069, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1921663, 2024-01-11T18:13:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 3890540, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.331172906, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.105999223, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.158875580, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.076093501, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4070, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3.302999190, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000811547 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3844, 2024-01-11T18:13:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000967076, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000251 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 643904, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 184416 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4786, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 5609, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2194353, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 4104429, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 16, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.384245703, 2024-01-11T18:13:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.131573874, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.203710896, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.070463451, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5608, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2.701191556, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000481667 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5915, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001540889, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000260 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 557216 2024-01-11T18:13:59.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3359, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2.090466996, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000622348 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-01-11T18:13:59.454 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 287, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015329034, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000053411 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 2281, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 2084, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2084, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 66.241839192, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.031785911 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-01-11T18:13:59.455 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 197, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 4, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 692, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 3, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 452, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 8, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 432, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 12, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 440, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 1295, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 422, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 422, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 19, 2024-01-11T18:13:59.456 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 3037, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 2215, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 191, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 13, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 9, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 6317, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 41, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 1295, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 39, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 675, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 403, 2024-01-11T18:13:59.457 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 1037, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 1630939938, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 403, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 37, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 2, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 35, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 366, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 7, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 117, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 1359, 2024-01-11T18:13:59.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 1, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 1348, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 1097, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 77, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 282 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 83, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 168, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 86, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 171, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 219, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-01-11T18:13:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 91, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 85, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 6, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 2, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 41, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 3, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-01-11T18:13:59.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 3717, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 3684, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 3684, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 33, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 18, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 73, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 72, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 72, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 1, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 1, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-01-11T18:13:59.461 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 1, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 13706152, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 13926505, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1101, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 11.434466639, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.010385528 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 455, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 1721, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 1266, 2024-01-11T18:13:59.462 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 50, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 165, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 115, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 452, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 2123, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 1671, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 422, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 1769, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 1347, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 67988, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 207228 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 2797, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 39, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 2281, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 195, 2024-01-11T18:13:59.463 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 184, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1061, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 47.997952682, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.045238409 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 78, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 14.487728272, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.185740106 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.011339733, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001889955 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 229, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.210533018, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000919358 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 35, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.300650216, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.008590006 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 510, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.204024868, 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002360833 2024-01-11T18:13:59.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 48, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.138027796, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002875579 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 79, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.788975986, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.022645265 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-01-11T18:13:59.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.471 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.005947078, 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001982359 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.024477599, 2024-01-11T18:13:59.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003059699 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 27, 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.072181944, 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002673405 2024-01-11T18:13:59.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-01-11T18:13:59.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 37, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 37, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 962, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 360, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 360, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:13:59.476 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:13:59.477 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-01-11T18:13:59.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:13:59.479 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 315422, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 559, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 88, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 1, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-01-11T18:13:59.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8792, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 168, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 1582914, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 14709, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-01-11T18:13:59.481 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 5568, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 12419143, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 31, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 5568, 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-01-11T18:13:59.482 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5568, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 394.309894683, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.070817150 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5568, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 10207 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 5568, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 442, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 5126, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-01-11T18:13:59.483 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 625, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 2172, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 33, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 1132, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 264, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 3, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 701, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-01-11T18:13:59.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 407, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 3289, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 1, 2024-01-11T18:13:59.485 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-01-11T18:13:59.486 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 89, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 10, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 12, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 12, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 1490, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 8, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 82 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2024-01-11T18:13:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 17, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 8, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 85, 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-01-11T18:13:59.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 16187, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 5035005, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 16187, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 16187, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 5035005, 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:13:59.489 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 5568, 2024-01-11T18:13:59.490 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 12428344, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 5568, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 12428344, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-01-11T18:13:59.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 5568, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 5568, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 5568, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 5568, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 85, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 9265, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 85, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 31, 2024-01-11T18:13:59.493 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 9265, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x560c997e65a0": { 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3717, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 9732201, 2024-01-11T18:13:59.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3717, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1101, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 9732201, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:13:59.495 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-01-11T18:13:59.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:13:59 smithi038 ceph-mon[43582]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-01-11T18:13:59.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:13:59 smithi007 ceph-mon[33361]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-01-11T18:13:59.824 INFO:tasks.check_counter:Daemon mds.b mds.exported=41 2024-01-11T18:13:59.824 INFO:tasks.check_counter:Daemon mds.b mds.imported=39 2024-01-11T18:13:59.824 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=83 2024-01-11T18:13:59.825 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=168 2024-01-11T18:13:59.825 DEBUG:tasks.check_counter:Getting stats from e 2024-01-11T18:13:59.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.e perf dump 2024-01-11T18:14:00.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T18:14:01.094 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 648, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 650, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 218079, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 201538, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.076928193, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.024363331, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.019592591, 2024-01-11T18:14:01.095 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.024582522, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 648, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.244862103, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000377873 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-01-11T18:14:01.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1744, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1723, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2294842, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 619906, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.171748827, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.041180161, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.059331217, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.062434785, 2024-01-11T18:14:01.097 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1722, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.433056504, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000251484 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 584928, 2024-01-11T18:14:01.098 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 176976 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 58, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 277, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 273056, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 441837, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.019474335, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.011204579, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003853961, 2024-01-11T18:14:01.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.001600666, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 276, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.092844793, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000336394 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-01-11T18:14:01.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 425664 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3560, 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.108105471, 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000030366 2024-01-11T18:14:01.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-01-11T18:14:01.102 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 429, 2024-01-11T18:14:01.103 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 283, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 146, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 24, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 15, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-01-11T18:14:01.104 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 1037, 2024-01-11T18:14:01.105 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 1630939938, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-01-11T18:14:01.106 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-01-11T18:14:01.107 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-01-11T18:14:01.108 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-01-11T18:14:01.109 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 481, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 49, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 13706152, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 13931718, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 13926505, 2024-01-11T18:14:01.110 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 481 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 401, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 465, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 64, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 25, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 38, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 13, 2024-01-11T18:14:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 429, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 470, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 41, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 52680, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 207228 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-01-11T18:14:01.112 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-01-11T18:14:01.113 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.114 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.116 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.117 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.118 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-01-11T18:14:01.120 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.122 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.123 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 0, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 2608, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:01.125 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:01.126 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:01.127 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 204335, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 442, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-01-11T18:14:01.128 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8792, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 168, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 1232444, 2024-01-11T18:14:01.129 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 10071, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2124, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 26, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 4, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2124, 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-01-11T18:14:01.130 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2124, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 34.644430655, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.016310937 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2124, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2125 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2124, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2124, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-01-11T18:14:01.131 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1290, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 831, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-01-11T18:14:01.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 2, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-01-11T18:14:01.133 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-01-11T18:14:01.134 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 5, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-01-11T18:14:01.135 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-01-11T18:14:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2446, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2546452, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2446, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2446, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2546452, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-01-11T18:14:01.138 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2124, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1605121, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2124, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1605121, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2124, 2024-01-11T18:14:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2124, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2124, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2124, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x5570326046e0": { 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-01-11T18:14:01.143 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:01.144 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:01.144 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:01.144 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:01.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-01-11T18:14:01.471 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:01 smithi016 ceph-mon[43195]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-01-11T18:14:01.474 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-01-11T18:14:01.474 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-01-11T18:14:01.474 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-01-11T18:14:01.474 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-01-11T18:14:01.474 DEBUG:tasks.check_counter:Getting stats from h 2024-01-11T18:14:01.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.h perf dump 2024-01-11T18:14:01.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:01 smithi038 ceph-mon[43582]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-01-11T18:14:01.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:01 smithi007 ceph-mon[33361]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-01-11T18:14:01.782 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T18:14:02.825 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 184, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 183, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 191328, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 109456, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.028375097, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.008179576, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.011481858, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005289344, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 182, 2024-01-11T18:14:02.826 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.047667052, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000261906 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 191328, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 109456 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 139, 2024-01-11T18:14:02.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 101312, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.007973781, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.005678733, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000897996, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 137, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.039300026, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000286861 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 101312 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2992, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001039802, 2024-01-11T18:14:02.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000062187, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000920120, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001561315, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000520438 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-01-11T18:14:02.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:02.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148967, 2024-01-11T18:14:02.833 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-01-11T18:14:02.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 185, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 165325, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 185, 2024-01-11T18:14:02.835 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 185, 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 165325, 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:02.836 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-01-11T18:14:03.193 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-01-11T18:14:03.193 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-01-11T18:14:03.193 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.h 2024-01-11T18:14:03.193 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.h 2024-01-11T18:14:03.193 DEBUG:tasks.check_counter:Getting stats from k 2024-01-11T18:14:03.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.k perf dump 2024-01-11T18:14:03.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:03 smithi016 ceph-mon[43195]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-01-11T18:14:03.496 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.b/config 2024-01-11T18:14:03.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:03 smithi038 ceph-mon[43582]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-01-11T18:14:03.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:03 smithi007 ceph-mon[33361]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-01-11T18:14:04.494 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-01-11T18:14:04.494 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:04.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-01-11T18:14:04.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-01-11T18:14:04.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-01-11T18:14:04.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001599729, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000380514, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 227, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3312, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 194096, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.014147166, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.009674442, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002030757, 2024-01-11T18:14:04.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 226, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.056859372, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000251590 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194096 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 298, 2024-01-11T18:14:04.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 295, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 497456, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 177232, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.046323609, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.016682440, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.020315761, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.008118322, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 294, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.055984134, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000190422 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:04.498 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 497456, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 177232 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:04.499 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:04.500 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148704, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-01-11T18:14:04.502 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 300, 2024-01-11T18:14:04.503 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 455353, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 300, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 300, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 455353, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-01-11T18:14:04.504 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-01-11T18:14:04.857 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-01-11T18:14:04.857 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-01-11T18:14:04.857 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-01-11T18:14:04.857 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-01-11T18:14:04.857 DEBUG:tasks.check_counter:Getting stats from c 2024-01-11T18:14:04.858 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.c perf dump 2024-01-11T18:14:05.279 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T18:14:06.235 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:05 smithi016 ceph-mon[43195]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-01-11T18:14:06.241 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:05 smithi038 ceph-mon[43582]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-01-11T18:14:06.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:05 smithi007 ceph-mon[33361]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-01-11T18:14:06.468 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-01-11T18:14:06.469 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:06.469 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 3325, 2024-01-11T18:14:06.469 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3325, 2024-01-11T18:14:06.469 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 5670709, 2024-01-11T18:14:06.469 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 1033286, 2024-01-11T18:14:06.469 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-01-11T18:14:06.470 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:06.470 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.292719705, 2024-01-11T18:14:06.470 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.079020304, 2024-01-11T18:14:06.470 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.080392649, 2024-01-11T18:14:06.470 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.111073755, 2024-01-11T18:14:06.470 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:06.471 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3324, 2024-01-11T18:14:06.471 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.746213434, 2024-01-11T18:14:06.471 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000224492 2024-01-11T18:14:06.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.471 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:06.471 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.472 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 225, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 876, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 544700, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.013788620, 2024-01-11T18:14:06.473 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.009438314, 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.002001672, 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000086816, 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 223, 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.069469399, 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000311521 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:06.474 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 544016 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.475 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 332, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 302, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 622780, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 177655, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.048020167, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.016527744, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.022135978, 2024-01-11T18:14:06.476 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.008439932, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 301, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.073320742, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000243590 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 622512, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 177344 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-01-11T18:14:06.477 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5455, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.150077008, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000027511 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.478 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-01-11T18:14:06.479 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 666, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 105, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 561, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 516, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 23, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-01-11T18:14:06.480 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-01-11T18:14:06.481 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-01-11T18:14:06.482 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-01-11T18:14:06.483 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-01-11T18:14:06.484 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 2413, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 5, 2024-01-11T18:14:06.485 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 8933968, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 8933394, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 2413 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 231, 2024-01-11T18:14:06.486 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 889, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 658, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 40, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 78, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 38, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 666, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 1380, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 714, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 54584, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 207228 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-01-11T18:14:06.487 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.488 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-01-11T18:14:06.490 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.491 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.492 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.493 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.494 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.495 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.496 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.497 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-01-11T18:14:06.498 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-01-11T18:14:06.499 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 2609, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:06.500 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:06.501 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 192026, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 270, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:14:06.502 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8792, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 168, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:06.503 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 1056021, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 20531, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3324, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-01-11T18:14:06.504 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3324, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3324, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 16.518689335, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.004969521 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3324, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3325 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3324, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 3324, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-01-11T18:14:06.505 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 1999, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 1323, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-01-11T18:14:06.506 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-01-11T18:14:06.507 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 89, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 3, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 4, 2024-01-11T18:14:06.508 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 5, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 1, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.509 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:06.510 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3657, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 5945372, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3657, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3657, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 5945372, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.511 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3324, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 4739669, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3324, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 4739669, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.512 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3324, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3324, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3324, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3324, 2024-01-11T18:14:06.513 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:06.514 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55f9acf32960": { 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:06.515 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:06.516 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-01-11T18:14:06.879 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-01-11T18:14:06.880 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-01-11T18:14:06.880 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-01-11T18:14:06.880 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-01-11T18:14:06.880 DEBUG:tasks.check_counter:Getting stats from f 2024-01-11T18:14:06.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.f perf dump 2024-01-11T18:14:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T18:14:07.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:07 smithi016 ceph-mon[43195]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-01-11T18:14:07.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:07 smithi038 ceph-mon[43582]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-01-11T18:14:07.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:07 smithi007 ceph-mon[33361]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1297, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1297, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 4152894, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 402635, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.123374197, 2024-01-11T18:14:08.320 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.031220626, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.038916819, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.040936160, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1296, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.340944982, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000263074 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.321 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1292, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1262, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 858966, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 476533, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.144548627, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.038024514, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.046225654, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.051284198, 2024-01-11T18:14:08.322 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1261, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.364216849, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000288831 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 622784, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 177968 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:08.323 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1041, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1263, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 349424, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 868889, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.134288890, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.042947799, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.034689594, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.043007178, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1261, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.425580266, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000337494 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.324 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 545760 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.325 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5551, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.166563861, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000030006 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.326 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-01-11T18:14:08.327 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 629, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 73, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 556, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 525, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 14, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-01-11T18:14:08.328 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-01-11T18:14:08.331 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-01-11T18:14:08.332 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-01-11T18:14:08.333 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-01-11T18:14:08.334 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 2183, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-01-11T18:14:08.335 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 4, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 8008817, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 8006746, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 2183 2024-01-11T18:14:08.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 440, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 666, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 226, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 41, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 43, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 2, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 629, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 713, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 84, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 51652, 2024-01-11T18:14:08.337 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 207228 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.338 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.339 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.340 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.341 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-01-11T18:14:08.342 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.343 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.343 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.343 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.343 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-01-11T18:14:08.343 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.343 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.344 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.344 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-01-11T18:14:08.344 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.344 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.344 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.346 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.347 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-01-11T18:14:08.348 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.349 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-01-11T18:14:08.350 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 2612, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:08.351 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:08.352 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:08.353 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 206188, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 485, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:08.354 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8792, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 168, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 1494331, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 12934, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-01-11T18:14:08.355 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3292, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3292, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3292, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 16.160695427, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.004909081 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3292, 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3293 2024-01-11T18:14:08.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3292, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 3292, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 1997, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 1293, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-01-11T18:14:08.357 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-01-11T18:14:08.358 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-01-11T18:14:08.359 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 89, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 3, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 5, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 6, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 1, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-01-11T18:14:08.360 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-01-11T18:14:08.361 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3626, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 5015029, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3626, 2024-01-11T18:14:08.362 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3626, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 5015029, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:08.363 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3292, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3814518, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3292, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3814518, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-01-11T18:14:08.364 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3292, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3292, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3292, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3292, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.366 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55fdf6e20960": { 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-01-11T18:14:08.367 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:08.368 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:08.368 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:08.368 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:08.368 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.368 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:08.368 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-01-11T18:14:08.715 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-01-11T18:14:08.715 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-01-11T18:14:08.715 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-01-11T18:14:08.715 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-01-11T18:14:08.715 DEBUG:tasks.check_counter:Getting stats from i 2024-01-11T18:14:08.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.i perf dump 2024-01-11T18:14:09.019 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T18:14:09.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:09 smithi016 ceph-mon[43195]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-01-11T18:14:09.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:09 smithi038 ceph-mon[43582]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-01-11T18:14:09.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:09 smithi007 ceph-mon[33361]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-01-11T18:14:10.113 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 3181, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3016, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1464867, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 4548655, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 11, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.273178269, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.095175792, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.153841157, 2024-01-11T18:14:10.114 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.041561196, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3014, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.750580414, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000249031 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3929, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.001103389, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000280 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:10.115 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2827, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3615, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1429055, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 2222418, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 12, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 5, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.246465681, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.097659157, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.158456924, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.020459641, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3615, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.706205163, 2024-01-11T18:14:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000195354 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2217, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.001396478, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000629 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 632544, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 313056 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2089, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3700, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 714923, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 1837889, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 13, 2024-01-11T18:14:10.117 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.190506216, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.077006686, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.118556979, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.009446636, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3698, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.952025036, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000257443 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.118 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3865, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.001045598, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000270 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 546384 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-01-11T18:14:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1667, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.686206364, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000411641 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 551, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.030257969, 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000054914 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 1075, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 938, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 938, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 42.449958037, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.045255818 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 137, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 2, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 11, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-01-11T18:14:10.121 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 477, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 7, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 456, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 14, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 466, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 86, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 436, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 436, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 25, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 2328, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 1975, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 137, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 18, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-01-11T18:14:10.122 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 6641, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 3, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 84, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 14, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 541, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 412, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 1038, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 1600818061, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 412, 2024-01-11T18:14:10.123 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 412, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 4, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 30, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 483, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 649, 2024-01-11T18:14:10.124 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 399, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 85, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 318 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 18, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 220, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 119, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 174, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 170, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 144, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 144, 2024-01-11T18:14:10.125 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 3, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-01-11T18:14:10.126 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 2183, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 2183, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 4, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 4, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 8008817, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-01-11T18:14:10.127 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 630, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 6.496318351, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.010311616 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 459, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 676, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 217, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 64, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 131, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 67, 2024-01-11T18:14:10.128 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 474, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 730, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 256, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 433, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 991, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 558, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 61428, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 199036 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 1978, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 7, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 1075, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 153, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 288, 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-01-11T18:14:10.129 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 147, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.349771384, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002379397 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 42, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 39.846414663, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.948724158 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-01-11T18:14:10.130 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 510, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.319980538, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000627412 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-01-11T18:14:10.131 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000262614, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000131307 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.132 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.134 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 66, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.036218768, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000548769 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 24, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.034047413, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001418642 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 136, 2024-01-11T18:14:10.135 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.817119966, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.013361176 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-01-11T18:14:10.136 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.137 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-01-11T18:14:10.138 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 11, 2024-01-11T18:14:10.139 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.046142691, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.004194790 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-01-11T18:14:10.140 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 122, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 122, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 958, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:10.141 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:10.142 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 325302, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 572, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 176, 2024-01-11T18:14:10.143 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 2, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8792, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 168, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 1658513, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 17741, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:10.144 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 1709, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 3912770, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 28, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 1709, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1709, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 24.955529621, 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.014602416 2024-01-11T18:14:10.145 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1709, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1843 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 1709, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 487, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 1222, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 10, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 73, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 703, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 116, 2024-01-11T18:14:10.146 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 300, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 414, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 20, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-01-11T18:14:10.147 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-01-11T18:14:10.148 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 89, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 3, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 12, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 12, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 150, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 4, 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 53 2024-01-11T18:14:10.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 16, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 8, 2024-01-11T18:14:10.150 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 144, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 8093, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 2892137, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 8093, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 8093, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 2892137, 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:10.151 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1709, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3928434, 2024-01-11T18:14:10.152 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1709, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3928434, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-01-11T18:14:10.153 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1709, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1709, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1709, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1709, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 144, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 15696, 2024-01-11T18:14:10.154 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 144, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 73, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 15696, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55ee108ec5a0": { 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:10.155 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 2183, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 3814513, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 2183, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 630, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3814513, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:10.156 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-01-11T18:14:10.498 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-01-11T18:14:10.498 INFO:tasks.check_counter:Daemon mds.i mds.imported=14 2024-01-11T18:14:10.498 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=18 2024-01-11T18:14:10.498 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=220 2024-01-11T18:14:10.498 DEBUG:tasks.check_counter:Getting stats from l 2024-01-11T18:14:10.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c27531add9155519f390b228e498f0a2c5eae117 shell --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 -- ceph daemon mds.l perf dump 2024-01-11T18:14:10.805 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/mon.c/config 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4, 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4, 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 2992, 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 448, 2024-01-11T18:14:11.890 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.002190952, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000056321, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001001584, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.001182469, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000394156 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:11.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 227, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 320, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 194000, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.014076577, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.009382232, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.002313756, 2024-01-11T18:14:11.892 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 225, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.082060327, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000364712 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 194000 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-01-11T18:14:11.893 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 301, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 298, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 498272, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 179104, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.044995613, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.016105893, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.018815947, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.008314529, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 297, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.100740180, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000339192 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-01-11T18:14:11.894 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 498272, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 179104 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-01-11T18:14:11.895 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-01-11T18:14:11.896 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-01-11T18:14:11.897 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148704, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 29, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-01-11T18:14:11.898 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-01-11T18:14:11.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 303, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 455743, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 303, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 303, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 455743, 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-01-11T18:14:11.900 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-01-11T18:14:11.901 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-01-11T18:14:11.901 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-01-11T18:14:11.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:11.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-01-11T18:14:11.901 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-01-11T18:14:12.307 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-01-11T18:14:12.307 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-01-11T18:14:12.307 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-01-11T18:14:12.307 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-01-11T18:14:12.307 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-11T18:14:12.318 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-11T18:14:12.319 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T18:14:12.319 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-11T18:14:12.380 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T18:14:12.380 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-11T18:14:12.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T18:14:12.484 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:12 smithi016 ceph-mon[43195]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-01-11T18:14:12.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:12 smithi038 ceph-mon[43582]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-01-11T18:14:12.531 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:12 smithi007 ceph-mon[33361]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/2461491526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74101590 msgr2=0x7f1f741019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/2461491526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74101590 0x7f1f741019b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009990 tx=0x7f1f70009ca0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/2461491526 shutdown_connections 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/2461491526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74101590 0x7f1f741019b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/2461491526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74103120 0x7f1f741099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/2461491526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74102780 0x7f1f74102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/2461491526 >> 172.21.15.7:0/2461491526 conn(0x7f1f740fcb00 msgr2=0x7f1f740fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/2461491526 shutdown_connections 2024-01-11T18:14:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.688+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/2461491526 wait complete. 2024-01-11T18:14:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.690+0000 7f1f7b5ff700 1 Processor -- start 2024-01-11T18:14:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.690+0000 7f1f7b5ff700 1 -- start start 2024-01-11T18:14:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.690+0000 7f1f7b5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 0x7f1f740753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.690+0000 7f1f7b5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74102780 0x7f1f74073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7b5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74103120 0x7f1f74073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7b5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740746e0 con 0x7f1f74101590 2024-01-11T18:14:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7b5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f74074850 con 0x7f1f74102780 2024-01-11T18:14:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7b5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741a15a0 con 0x7f1f74103120 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 0x7f1f740753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 0x7f1f740753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44970/0 (socket says 172.21.15.7:44970) 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 -- 172.21.15.7:0/3304400599 learned_addr learned my addr 172.21.15.7:0/3304400599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f78b9a700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74102780 0x7f1f74073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f79b9c700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74103120 0x7f1f74073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 -- 172.21.15.7:0/3304400599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74103120 msgr2=0x7f1f74073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74103120 0x7f1f74073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 -- 172.21.15.7:0/3304400599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74102780 msgr2=0x7f1f74073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74102780 0x7f1f74073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.691+0000 7f1f7939b700 1 -- 172.21.15.7:0/3304400599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64009520 con 0x7f1f74101590 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.692+0000 7f1f7939b700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 0x7f1f740753c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f70009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.692+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7001d070 con 0x7f1f74101590 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.692+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f70021410 con 0x7f1f74101590 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.692+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c009c40 con 0x7f1f74101590 2024-01-11T18:14:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.692+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70017650 con 0x7f1f74101590 2024-01-11T18:14:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.692+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f700095f0 con 0x7f1f74101590 2024-01-11T18:14:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.693+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f74108ab0 con 0x7f1f74101590 2024-01-11T18:14:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.695+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f1f70021580 con 0x7f1f74101590 2024-01-11T18:14:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.697+0000 7f1f6a7fc700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1f600a1f60 0x7f1f600a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.698+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1f700ceef0 con 0x7f1f74101590 2024-01-11T18:14:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.698+0000 7f1f78b9a700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1f600a1f60 0x7f1f600a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.698+0000 7f1f78b9a700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1f600a1f60 0x7f1f600a4420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009df0 tx=0x7f1f64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.702+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f1f70097c90 con 0x7f1f74101590 2024-01-11T18:14:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.915+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f740683f0 con 0x7f1f74101590 2024-01-11T18:14:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.917+0000 7f1f6a7fc700 1 -- 172.21.15.7:0/3304400599 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24382 (secure 0 0 0) 0x7f1f7009b2b0 con 0x7f1f74101590 2024-01-11T18:14:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:14:12.907 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T18:02:43.786606+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.919+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1f600a1f60 msgr2=0x7f1f600a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.919+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1f600a1f60 0x7f1f600a4420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009df0 tx=0x7f1f64009260 comp rx=0 tx=0).stop 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.919+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 msgr2=0x7f1f740753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.919+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 0x7f1f740753c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f70009580 comp rx=0 tx=0).stop 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 shutdown_connections 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f1f600a1f60 0x7f1f600a4420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f74103120 0x7f1f74073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74101590 0x7f1f740753c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 --2- 172.21.15.7:0/3304400599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f74102780 0x7f1f74073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 >> 172.21.15.7:0/3304400599 conn(0x7f1f740fcb00 msgr2=0x7f1f74105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 shutdown_connections 2024-01-11T18:14:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:12.920+0000 7f1f7b5ff700 1 -- 172.21.15.7:0/3304400599 wait complete. 2024-01-11T18:14:12.929 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T17:52:31.107037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T17:56:06.693316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T17:56:07.769186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-11T17:56:11.302707+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T18:14:12.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T18:14:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.148+0000 7f86639ff700 1 -- 172.21.15.7:0/800330832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c075650 msgr2=0x7f865c073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.148+0000 7f86639ff700 1 --2- 172.21.15.7:0/800330832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c075650 0x7f865c073aa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8658009990 tx=0x7f8658009ca0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 -- 172.21.15.7:0/800330832 shutdown_connections 2024-01-11T18:14:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 --2- 172.21.15.7:0/800330832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c075650 0x7f865c073aa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 --2- 172.21.15.7:0/800330832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c10db80 0x7f865c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 --2- 172.21.15.7:0/800330832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 -- 172.21.15.7:0/800330832 >> 172.21.15.7:0/800330832 conn(0x7f865c0fcb00 msgr2=0x7f865c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 -- 172.21.15.7:0/800330832 shutdown_connections 2024-01-11T18:14:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.149+0000 7f86639ff700 1 -- 172.21.15.7:0/800330832 wait complete. 2024-01-11T18:14:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 Processor -- start 2024-01-11T18:14:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 -- start start 2024-01-11T18:14:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c075650 0x7f865c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c10db80 0x7f865c1a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1a5630 con 0x7f865c074070 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1a5770 con 0x7f865c075650 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f86639ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1a5900 con 0x7f865c10db80 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f866179b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f866179b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44980/0 (socket says 172.21.15.7:44980) 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.150+0000 7f866179b700 1 -- 172.21.15.7:0/1379429852 learned_addr learned my addr 172.21.15.7:0/1379429852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f8661f9c700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c10db80 0x7f865c1a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f8660f9a700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c075650 0x7f865c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f866179b700 1 -- 172.21.15.7:0/1379429852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c10db80 msgr2=0x7f865c1a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f866179b700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c10db80 0x7f865c1a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f866179b700 1 -- 172.21.15.7:0/1379429852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c075650 msgr2=0x7f865c19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f866179b700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c075650 0x7f865c19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f866179b700 1 -- 172.21.15.7:0/1379429852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009520 con 0x7f865c074070 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f8660f9a700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c075650 0x7f865c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f866179b700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c19dab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8658006010 tx=0x7f8658005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865801d070 con 0x7f865c074070 2024-01-11T18:14:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86580095f0 con 0x7f865c074070 2024-01-11T18:14:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.151+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8658004ca0 con 0x7f865c074070 2024-01-11T18:14:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.152+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c1a5e20 con 0x7f865c074070 2024-01-11T18:14:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.152+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658005520 con 0x7f865c074070 2024-01-11T18:14:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.153+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c103de0 con 0x7f865c074070 2024-01-11T18:14:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.155+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f865800fb80 con 0x7f865c074070 2024-01-11T18:14:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.157+0000 7f86527fc700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f86480a1ee0 0x7f86480a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.157+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f86580cee30 con 0x7f865c074070 2024-01-11T18:14:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.157+0000 7f8660f9a700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f86480a1ee0 0x7f86480a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.158+0000 7f8660f9a700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f86480a1ee0 0x7f86480a43a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f865c075370 tx=0x7f864c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.161+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f865805c380 con 0x7f865c074070 2024-01-11T18:14:13.310 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:13 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3304400599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:14:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.380+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f865c02d3f0 con 0x7f865c074070 2024-01-11T18:14:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.381+0000 7f86527fc700 1 -- 172.21.15.7:0/1379429852 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8658097bd0 con 0x7f865c074070 2024-01-11T18:14:13.371 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f86480a1ee0 msgr2=0x7f86480a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f86480a1ee0 0x7f86480a43a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f865c075370 tx=0x7f864c009260 comp rx=0 tx=0).stop 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 msgr2=0x7f865c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c19dab0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8658006010 tx=0x7f8658005ef0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 shutdown_connections 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f86480a1ee0 0x7f86480a43a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f865c10db80 0x7f865c1a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f865c074070 0x7f865c19dab0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 --2- 172.21.15.7:0/1379429852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c075650 0x7f865c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 >> 172.21.15.7:0/1379429852 conn(0x7f865c0fcb00 msgr2=0x7f865c100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.383+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 shutdown_connections 2024-01-11T18:14:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.384+0000 7f86639ff700 1 -- 172.21.15.7:0/1379429852 wait complete. 2024-01-11T18:14:13.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T18:14:13.484 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:13 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3304400599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:14:13.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:13 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3304400599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 -- 172.21.15.7:0/3577385187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81020b0 msgr2=0x7f27a81044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 --2- 172.21.15.7:0/3577385187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81020b0 0x7f27a81044a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009990 tx=0x7f27a4009ca0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 -- 172.21.15.7:0/3577385187 shutdown_connections 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 --2- 172.21.15.7:0/3577385187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a8107310 0x7f27a8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 --2- 172.21.15.7:0/3577385187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81020b0 0x7f27a81044a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 --2- 172.21.15.7:0/3577385187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27a81049e0 0x7f27a8106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 -- 172.21.15.7:0/3577385187 >> 172.21.15.7:0/3577385187 conn(0x7f27a80fba00 msgr2=0x7f27a80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 -- 172.21.15.7:0/3577385187 shutdown_connections 2024-01-11T18:14:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.611+0000 7f27b07ff700 1 -- 172.21.15.7:0/3577385187 wait complete. 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.612+0000 7f27b07ff700 1 Processor -- start 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 -- start start 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27a81020b0 0x7f27a8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 0x7f27a8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a8107310 0x7f27a81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81a11a0 con 0x7f27a8107310 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81a12e0 con 0x7f27a81049e0 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27b07ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81a1470 con 0x7f27a81020b0 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27aed9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a8107310 0x7f27a81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27add9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 0x7f27a8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27ae59b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27a81020b0 0x7f27a8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27add9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 0x7f27a8199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57952/0 (socket says 172.21.15.7:57952) 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.613+0000 7f27add9a700 1 -- 172.21.15.7:0/1084037645 learned_addr learned my addr 172.21.15.7:0/1084037645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27add9a700 1 -- 172.21.15.7:0/1084037645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27a81020b0 msgr2=0x7f27a8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27add9a700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27a81020b0 0x7f27a8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27add9a700 1 -- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a8107310 msgr2=0x7f27a81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27add9a700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a8107310 0x7f27a81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27add9a700 1 -- 172.21.15.7:0/1084037645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40095f0 con 0x7f27a81049e0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27add9a700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 0x7f27a8199b60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f279800e9a0 tx=0x7f279800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798009790 con 0x7f27a81049e0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2798010430 con 0x7f27a81049e0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798018680 con 0x7f27a81049e0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a81a1750 con 0x7f27a81049e0 2024-01-11T18:14:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.614+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a81a1cb0 con 0x7f27a81049e0 2024-01-11T18:14:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.615+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a8108e90 con 0x7f27a81049e0 2024-01-11T18:14:13.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.618+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f27980105a0 con 0x7f27a81049e0 2024-01-11T18:14:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.619+0000 7f279f7fe700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f27940a1f60 0x7f27940a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.619+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2798014070 con 0x7f27a81049e0 2024-01-11T18:14:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.619+0000 7f27ae59b700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f27940a1f60 0x7f27940a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:13.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.620+0000 7f27ae59b700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f27940a1f60 0x7f27940a4420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f27a400b2e0 tx=0x7f27a400b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:13.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.621+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f2798096e20 con 0x7f27a81049e0 2024-01-11T18:14:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.835+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27a810cf20 con 0x7f27a81049e0 2024-01-11T18:14:13.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.835+0000 7f279f7fe700 1 -- 172.21.15.7:0/1084037645 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27980260a0 con 0x7f27a81049e0 2024-01-11T18:14:13.826 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T18:14:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f27940a1f60 msgr2=0x7f27940a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f27940a1f60 0x7f27940a4420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f27a400b2e0 tx=0x7f27a400b750 comp rx=0 tx=0).stop 2024-01-11T18:14:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 msgr2=0x7f27a8199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 0x7f27a8199b60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f279800e9a0 tx=0x7f279800ecb0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 shutdown_connections 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f27940a1f60 0x7f27940a4420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27a81020b0 0x7f27a8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27a8107310 0x7f27a81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 --2- 172.21.15.7:0/1084037645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a81049e0 0x7f27a8199b60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 >> 172.21.15.7:0/1084037645 conn(0x7f27a80fba00 msgr2=0x7f27a8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 shutdown_connections 2024-01-11T18:14:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:13.838+0000 7f27b07ff700 1 -- 172.21.15.7:0/1084037645 wait complete. 2024-01-11T18:14:13.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T18:14:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 -- 172.21.15.7:0/3682591139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb81027a0 msgr2=0x7effb8102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 --2- 172.21.15.7:0/3682591139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb81027a0 0x7effb8102c00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7effa80099c0 tx=0x7effa8009cd0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 -- 172.21.15.7:0/3682591139 shutdown_connections 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 --2- 172.21.15.7:0/3682591139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb8103140 0x7effb81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 --2- 172.21.15.7:0/3682591139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81015b0 0x7effb81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 --2- 172.21.15.7:0/3682591139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb81027a0 0x7effb8102c00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 -- 172.21.15.7:0/3682591139 >> 172.21.15.7:0/3682591139 conn(0x7effb80fcb00 msgr2=0x7effb80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 -- 172.21.15.7:0/3682591139 shutdown_connections 2024-01-11T18:14:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.066+0000 7effbfbff700 1 -- 172.21.15.7:0/3682591139 wait complete. 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.067+0000 7effbfbff700 1 Processor -- start 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.067+0000 7effbfbff700 1 -- start start 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbfbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81015b0 0x7effb81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbfbff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb81027a0 0x7effb8199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbfbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 0x7effb81a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbfbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81a1230 con 0x7effb8103140 2024-01-11T18:14:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbfbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81a1370 con 0x7effb81015b0 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbfbff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81a1500 con 0x7effb81027a0 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 0x7effb81a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbd19a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb81027a0 0x7effb8199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 0x7effb81a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45018/0 (socket says 172.21.15.7:45018) 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 -- 172.21.15.7:0/111297238 learned_addr learned my addr 172.21.15.7:0/111297238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbd99b700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81015b0 0x7effb81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 -- 172.21.15.7:0/111297238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb81027a0 msgr2=0x7effb8199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb81027a0 0x7effb8199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 -- 172.21.15.7:0/111297238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81015b0 msgr2=0x7effb81996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81015b0 0x7effb81996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbe19c700 1 -- 172.21.15.7:0/111297238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80095f0 con 0x7effb8103140 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.068+0000 7effbd19a700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb81027a0 0x7effb8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.069+0000 7effbe19c700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 0x7effb81a0c60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7effb000ef30 tx=0x7effb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.069+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0018070 con 0x7effb8103140 2024-01-11T18:14:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.069+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb000f040 con 0x7effb8103140 2024-01-11T18:14:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.069+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb81a1780 con 0x7effb8103140 2024-01-11T18:14:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.069+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000cd00 con 0x7effb8103140 2024-01-11T18:14:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.069+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb81a1c40 con 0x7effb8103140 2024-01-11T18:14:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.070+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb80683f0 con 0x7effb8103140 2024-01-11T18:14:14.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.074+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7effb0003a00 con 0x7effb8103140 2024-01-11T18:14:14.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.075+0000 7effaeffd700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7effa40a1ee0 0x7effa40a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.075+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7effb00ce1e0 con 0x7effb8103140 2024-01-11T18:14:14.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.075+0000 7effbd99b700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7effa40a1ee0 0x7effa40a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.076+0000 7effbd99b700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7effa40a1ee0 0x7effa40a43a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7effb8102600 tx=0x7effb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.077+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7effb0096f80 con 0x7effb8103140 2024-01-11T18:14:14.179 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:14 smithi038 ceph-mon[43582]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-01-11T18:14:14.180 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:14 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1379429852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T18:14:14.180 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:14 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1084037645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T18:14:14.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.290+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7effb810cfb0 con 0x7effb8103140 2024-01-11T18:14:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.291+0000 7effaeffd700 1 -- 172.21.15.7:0/111297238 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7effb009a5a0 con 0x7effb8103140 2024-01-11T18:14:14.281 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T18:14:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7effa40a1ee0 msgr2=0x7effa40a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7effa40a1ee0 0x7effa40a43a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7effb8102600 tx=0x7effb4009260 comp rx=0 tx=0).stop 2024-01-11T18:14:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 msgr2=0x7effb81a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 0x7effb81a0c60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7effb000ef30 tx=0x7effb000c3a0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 shutdown_connections 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7effa40a1ee0 0x7effa40a43a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb8103140 0x7effb81a0c60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81015b0 0x7effb81996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 --2- 172.21.15.7:0/111297238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effb81027a0 0x7effb8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 >> 172.21.15.7:0/111297238 conn(0x7effb80fcb00 msgr2=0x7effb8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 shutdown_connections 2024-01-11T18:14:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.293+0000 7effbfbff700 1 -- 172.21.15.7:0/111297238 wait complete. 2024-01-11T18:14:14.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-11T18:14:14.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:14 smithi016 ceph-mon[43195]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-01-11T18:14:14.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:14 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1379429852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T18:14:14.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:14 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1084037645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 -- 172.21.15.7:0/2058360213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c103140 msgr2=0x7f895c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 --2- 172.21.15.7:0/2058360213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c103140 0x7f895c1099f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f895400b0d0 tx=0x7f895400b3e0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 -- 172.21.15.7:0/2058360213 shutdown_connections 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 --2- 172.21.15.7:0/2058360213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c1015b0 0x7f895c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 --2- 172.21.15.7:0/2058360213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c103140 0x7f895c1099f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 --2- 172.21.15.7:0/2058360213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1027a0 0x7f895c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 -- 172.21.15.7:0/2058360213 >> 172.21.15.7:0/2058360213 conn(0x7f895c0fcb00 msgr2=0x7f895c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.521+0000 7f8960dff700 1 -- 172.21.15.7:0/2058360213 shutdown_connections 2024-01-11T18:14:14.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.522+0000 7f8960dff700 1 -- 172.21.15.7:0/2058360213 wait complete. 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.522+0000 7f8960dff700 1 Processor -- start 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 -- start start 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1015b0 0x7f895c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 0x7f895c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c103140 0x7f895c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c19f170 con 0x7f895c1027a0 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c19f2b0 con 0x7f895c1015b0 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8960dff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c19f440 con 0x7f895c103140 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8959d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 0x7f895c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f895a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1015b0 0x7f895c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8959d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 0x7f895c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45040/0 (socket says 172.21.15.7:45040) 2024-01-11T18:14:14.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f895a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1015b0 0x7f895c1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57990/0 (socket says 172.21.15.7:57990) 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f8959d9b700 1 -- 172.21.15.7:0/818725998 learned_addr learned my addr 172.21.15.7:0/818725998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.523+0000 7f895ad9d700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c103140 0x7f895c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8959d9b700 1 -- 172.21.15.7:0/818725998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c103140 msgr2=0x7f895c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8959d9b700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c103140 0x7f895c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8959d9b700 1 -- 172.21.15.7:0/818725998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1015b0 msgr2=0x7f895c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8959d9b700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1015b0 0x7f895c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8959d9b700 1 -- 172.21.15.7:0/818725998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954009d00 con 0x7f895c1027a0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8959d9b700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 0x7f895c197b30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f895000e9a0 tx=0x7f895000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950009790 con 0x7f895c1027a0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8950010430 con 0x7f895c1027a0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c19f720 con 0x7f895c1027a0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950018680 con 0x7f895c1027a0 2024-01-11T18:14:14.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.524+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c19fc80 con 0x7f895c1027a0 2024-01-11T18:14:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.525+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c108900 con 0x7f895c1027a0 2024-01-11T18:14:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.528+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f89500105a0 con 0x7f895c1027a0 2024-01-11T18:14:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.529+0000 7f894b7fe700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89440a1f60 0x7f89440a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.529+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8950014070 con 0x7f895c1027a0 2024-01-11T18:14:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.529+0000 7f895a59c700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89440a1f60 0x7f89440a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.530+0000 7f895a59c700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89440a1f60 0x7f89440a4420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f895c102600 tx=0x7f894c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.531+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f8950096e20 con 0x7f895c1027a0 2024-01-11T18:14:14.559 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:14 smithi007 ceph-mon[33361]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-01-11T18:14:14.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:14 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1379429852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-11T18:14:14.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:14 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1084037645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-11T18:14:14.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.745+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f895c10cfb0 con 0x7f895c1027a0 2024-01-11T18:14:14.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.745+0000 7f894b7fe700 1 -- 172.21.15.7:0/818725998 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8950026020 con 0x7f895c1027a0 2024-01-11T18:14:14.736 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89440a1f60 msgr2=0x7f89440a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89440a1f60 0x7f89440a4420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f895c102600 tx=0x7f894c009260 comp rx=0 tx=0).stop 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 msgr2=0x7f895c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 0x7f895c197b30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f895000e9a0 tx=0x7f895000ecb0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 shutdown_connections 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f89440a1f60 0x7f89440a4420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c103140 0x7f895c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1015b0 0x7f895c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 --2- 172.21.15.7:0/818725998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f895c1027a0 0x7f895c197b30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 >> 172.21.15.7:0/818725998 conn(0x7f895c0fcb00 msgr2=0x7f895c105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 shutdown_connections 2024-01-11T18:14:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.748+0000 7f8960dff700 1 -- 172.21.15.7:0/818725998 wait complete. 2024-01-11T18:14:14.758 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-11T18:14:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 -- 172.21.15.7:0/2333482339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1027a0 msgr2=0x7f0e6c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2333482339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c102c00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e540099c0 tx=0x7f0e54009cd0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 -- 172.21.15.7:0/2333482339 shutdown_connections 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2333482339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c103140 0x7f0e6c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2333482339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6c1015b0 0x7f0e6c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2333482339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c102c00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 -- 172.21.15.7:0/2333482339 >> 172.21.15.7:0/2333482339 conn(0x7f0e6c0fcb00 msgr2=0x7f0e6c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 -- 172.21.15.7:0/2333482339 shutdown_connections 2024-01-11T18:14:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.975+0000 7f0e721ff700 1 -- 172.21.15.7:0/2333482339 wait complete. 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.976+0000 7f0e721ff700 1 Processor -- start 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.976+0000 7f0e721ff700 1 -- start start 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e721ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1015b0 0x7f0e6c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e721ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e721ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6c103140 0x7f0e6c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e721ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1a1240 con 0x7f0e6c1015b0 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e721ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1a1380 con 0x7f0e6c103140 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e721ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1a1510 con 0x7f0e6c1027a0 2024-01-11T18:14:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e6b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1015b0 0x7f0e6c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:56206/0 (socket says 172.21.15.7:56206) 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 -- 172.21.15.7:0/2186224779 learned_addr learned my addr 172.21.15.7:0/2186224779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e6bfff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6c103140 0x7f0e6c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 -- 172.21.15.7:0/2186224779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6c103140 msgr2=0x7f0e6c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6c103140 0x7f0e6c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 -- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1015b0 msgr2=0x7f0e6c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1015b0 0x7f0e6c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e63fff700 1 -- 172.21.15.7:0/2186224779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64009c40 con 0x7f0e6c1027a0 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.977+0000 7f0e6b7fe700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1015b0 0x7f0e6c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.978+0000 7f0e63fff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c199c00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0e54003730 tx=0x7f0e5400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.982+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5401d070 con 0x7f0e6c1027a0 2024-01-11T18:14:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.982+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e54005050 con 0x7f0e6c1027a0 2024-01-11T18:14:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.982+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e540095f0 con 0x7f0e6c1027a0 2024-01-11T18:14:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.982+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004b10 con 0x7f0e6c1027a0 2024-01-11T18:14:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.983+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c1a1a80 con 0x7f0e6c1027a0 2024-01-11T18:14:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.984+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0683f0 con 0x7f0e6c1027a0 2024-01-11T18:14:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.986+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f0e540051c0 con 0x7f0e6c1027a0 2024-01-11T18:14:14.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.987+0000 7f0e697fa700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0e580a1f60 0x7f0e580a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:14.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.987+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0e540cea20 con 0x7f0e6c1027a0 2024-01-11T18:14:14.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.987+0000 7f0e6b7fe700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0e580a1f60 0x7f0e580a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:14.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.988+0000 7f0e6b7fe700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0e580a1f60 0x7f0e580a4420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c102600 tx=0x7f0e5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:14.990+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f0e540978e0 con 0x7f0e6c1027a0 2024-01-11T18:14:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.234+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0e6c10cfe0 con 0x7f0e6c1027a0 2024-01-11T18:14:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.235+0000 7f0e697fa700 1 -- 172.21.15.7:0/2186224779 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+579 (secure 0 0 0) 0x7f0e5409af00 con 0x7f0e6c1027a0 2024-01-11T18:14:15.225 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3570192633 2024-01-12T17:50:24.497330+0000 2024-01-11T18:14:15.225 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/867047662 2024-01-12T17:50:24.497330+0000 2024-01-11T18:14:15.225 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1912965293 2024-01-12T17:50:24.497330+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3811163237 2024-01-12T17:49:57.975302+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3086228555 2024-01-12T17:49:57.975302+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/867047662 2024-01-12T17:50:24.497330+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4071850608 2024-01-12T17:50:24.497330+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1981932676 2024-01-12T17:49:57.975302+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1981932676 2024-01-12T17:49:57.975302+0000 2024-01-11T18:14:15.226 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/113093279 2024-01-12T17:49:57.975302+0000 2024-01-11T18:14:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0e580a1f60 msgr2=0x7f0e580a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0e580a1f60 0x7f0e580a4420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c102600 tx=0x7f0e5c009260 comp rx=0 tx=0).stop 2024-01-11T18:14:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 msgr2=0x7f0e6c199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c199c00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0e54003730 tx=0x7f0e5400f690 comp rx=0 tx=0).stop 2024-01-11T18:14:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 shutdown_connections 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f0e580a1f60 0x7f0e580a4420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6c103140 0x7f0e6c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c1015b0 0x7f0e6c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 --2- 172.21.15.7:0/2186224779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e6c1027a0 0x7f0e6c199c00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 >> 172.21.15.7:0/2186224779 conn(0x7f0e6c0fcb00 msgr2=0x7f0e6c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 shutdown_connections 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:15.238+0000 7f0e721ff700 1 -- 172.21.15.7:0/2186224779 wait complete. 2024-01-11T18:14:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-01-11T18:14:15.247 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T18:14:15.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:15.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /tmp/cephfs 2024-01-11T18:14:15.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:15 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/111297238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T18:14:15.475 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:15 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/818725998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T18:14:15.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T18:14:15.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:15.478 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-01-11T18:14:15.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:15 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/111297238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T18:14:15.488 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:15 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/818725998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T18:14:15.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:15 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/111297238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-11T18:14:15.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:15 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/818725998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-01-11T18:14:15.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:15.566 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T18:14:15.666 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-01-11T18:14:15.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:15.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-01-11T18:14:15.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:15.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T18:14:15.873 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-11T18:14:15.873 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T18:14:15.873 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-11T18:14:15.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16351428k,nr_inodes=4087857,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/4aa34edbc6b1f6692bf1dd734d2332a806cc62efd19730b92558f6e1b6a6a77a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/4aa34edbc6b1f6692bf1dd734d2332a806cc62efd19730b92558f6e1b6a6a77a/diff,workdir=/var/lib/containers/storage/overlay/4aa34edbc6b1f6692bf1dd734d2332a806cc62efd19730b92558f6e1b6a6a77a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/24c492caba31f4f736ce8a9ea91234f395deaed4ec31d26bd98bd45dfc26b297/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/24c492caba31f4f736ce8a9ea91234f395deaed4ec31d26bd98bd45dfc26b297/diff,workdir=/var/lib/containers/storage/overlay/24c492caba31f4f736ce8a9ea91234f395deaed4ec31d26bd98bd45dfc26b297/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3276240k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/4d556f2f30f6d85f6ed961ffb5fadce021856cbaa7a64140cbe115c29d8567b5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/4d556f2f30f6d85f6ed961ffb5fadce021856cbaa7a64140cbe115c29d8567b5/diff,workdir=/var/lib/containers/storage/overlay/4d556f2f30f6d85f6ed961ffb5fadce021856cbaa7a64140cbe115c29d8567b5/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d3102359743f78fabe7b95c64c6f9e7672db3c18179f4479327ed6cef3fbf35a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/d3102359743f78fabe7b95c64c6f9e7672db3c18179f4479327ed6cef3fbf35a/diff,workdir=/var/lib/containers/storage/overlay/d3102359743f78fabe7b95c64c6f9e7672db3c18179f4479327ed6cef3fbf35a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d67a1f995691072f157f7513f7f8b23ee7e1bea647dd9523c2bf08ac31ef1572/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/d67a1f995691072f157f7513f7f8b23ee7e1bea647dd9523c2bf08ac31ef1572/diff,workdir=/var/lib/containers/storage/overlay/d67a1f995691072f157f7513f7f8b23ee7e1bea647dd9523c2bf08ac31ef1572/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e66f4cad987f927e295c31f2aa3ff2d5a21f3846844acf0b8d5c1497add178b3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/e66f4cad987f927e295c31f2aa3ff2d5a21f3846844acf0b8d5c1497add178b3/diff,workdir=/var/lib/containers/storage/overlay/e66f4cad987f927e295c31f2aa3ff2d5a21f3846844acf0b8d5c1497add178b3/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/955a220b4ab39693075b37c675a2d917c9ced1c9dd7b6c39c87134bb67e1e2bc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/955a220b4ab39693075b37c675a2d917c9ced1c9dd7b6c39c87134bb67e1e2bc/diff,workdir=/var/lib/containers/storage/overlay/955a220b4ab39693075b37c675a2d917c9ced1c9dd7b6c39c87134bb67e1e2bc/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/33bbfa977eb92da0ee4d41e79f4b6cd5fb33d58ed03b493cf509ef2ae2a45a4d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/33bbfa977eb92da0ee4d41e79f4b6cd5fb33d58ed03b493cf509ef2ae2a45a4d/diff,workdir=/var/lib/containers/storage/overlay/33bbfa977eb92da0ee4d41e79f4b6cd5fb33d58ed03b493cf509ef2ae2a45a4d/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/50072246631df260b9b5bed694e36e3be62a2d98b2f3be076c38371644ab0781/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/50072246631df260b9b5bed694e36e3be62a2d98b2f3be076c38371644ab0781/diff,workdir=/var/lib/containers/storage/overlay/50072246631df260b9b5bed694e36e3be62a2d98b2f3be076c38371644ab0781/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f878d6dc0c5ac5abc5a76b26cf749a01691338296d8380a566e1db3ecec1de4b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J4BMDPGY7QXYLPH7ADBGFSTSBN:/var/lib/containers/storage/overlay/l/UPVBQDPYMGCWYZIYP5XCTZXYKB,upperdir=/var/lib/containers/storage/overlay/f878d6dc0c5ac5abc5a76b26cf749a01691338296d8380a566e1db3ecec1de4b/diff,workdir=/var/lib/containers/storage/overlay/f878d6dc0c5ac5abc5a76b26cf749a01691338296d8380a566e1db3ecec1de4b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2024-01-11T18:14:15.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:15.922 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T18:14:16.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T18:14:16.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T18:14:16.247 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-11T18:14:16.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:14:16.250 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-01-11T18:14:16.261 INFO:tasks.cephadm:Teardown begin 2024-01-11T18:14:16.261 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T18:14:16.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 -- 172.21.15.7:0/3529890039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 msgr2=0x7f78a41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/3529890039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41019b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78a00099c0 tx=0x7f78a0009cd0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 -- 172.21.15.7:0/3529890039 shutdown_connections 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/3529890039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4103120 0x7f78a41099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/3529890039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41019b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/3529890039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a4102780 0x7f78a4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 -- 172.21.15.7:0/3529890039 >> 172.21.15.7:0/3529890039 conn(0x7f78a40fcb00 msgr2=0x7f78a40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 -- 172.21.15.7:0/3529890039 shutdown_connections 2024-01-11T18:14:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.477+0000 7f78ac5ff700 1 -- 172.21.15.7:0/3529890039 wait complete. 2024-01-11T18:14:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 Processor -- start 2024-01-11T18:14:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 -- start start 2024-01-11T18:14:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102780 0x7f78a4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a4103120 0x7f78a419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419f070 con 0x7f78a4101590 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419f1b0 con 0x7f78a4102780 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78ac5ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419f340 con 0x7f78a4103120 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45082/0 (socket says 172.21.15.7:45082) 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 -- 172.21.15.7:0/4123114621 learned_addr learned my addr 172.21.15.7:0/4123114621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aab9c700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a4103120 0x7f78a419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78a9b9a700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102780 0x7f78a4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 -- 172.21.15.7:0/4123114621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a4103120 msgr2=0x7f78a419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a4103120 0x7f78a419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 -- 172.21.15.7:0/4123114621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102780 msgr2=0x7f78a4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102780 0x7f78a4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.480+0000 7f78aa39b700 1 -- 172.21.15.7:0/4123114621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00095f0 con 0x7f78a4101590 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.481+0000 7f78aa39b700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41974f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f78a0003730 tx=0x7f78a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.481+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a001d070 con 0x7f78a4101590 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.481+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a0005050 con 0x7f78a4101590 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.481+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a419f5c0 con 0x7f78a4101590 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.481+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004a50 con 0x7f78a4101590 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.481+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a419f9e0 con 0x7f78a4101590 2024-01-11T18:14:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.482+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a40683f0 con 0x7f78a4101590 2024-01-11T18:14:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.485+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f78a00051c0 con 0x7f78a4101590 2024-01-11T18:14:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.485+0000 7f789b7fe700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78900a1ee0 0x7f78900a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.486+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f78a00cea40 con 0x7f78a4101590 2024-01-11T18:14:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.486+0000 7f78a9b9a700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78900a1ee0 0x7f78900a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.486+0000 7f78a9b9a700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78900a1ee0 0x7f78900a43a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7894009c60 tx=0x7f7894009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.490+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f78a00977e0 con 0x7f78a4101590 2024-01-11T18:14:16.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:16 smithi016 ceph-mon[43195]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-01-11T18:14:16.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:16 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2186224779' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:16 smithi038 ceph-mon[43582]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-01-11T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:16 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2186224779' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T18:14:16.559 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:16 smithi007 ceph-mon[33361]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-01-11T18:14:16.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:16 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2186224779' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-11T18:14:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.752+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78a4108a10 con 0x7f78a4101590 2024-01-11T18:14:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.753+0000 7f789b7fe700 1 -- 172.21.15.7:0/4123114621 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f78a009ae00 con 0x7f78a4101590 2024-01-11T18:14:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:14:16.743 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":15897,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3100471640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3100471640},{"type":"v1","addr":"172.21.15.16:6837","nonce":3100471640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T18:13:56.388677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":45,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":43,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T18:14:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78900a1ee0 msgr2=0x7f78900a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78900a1ee0 0x7f78900a43a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7894009c60 tx=0x7f7894009260 comp rx=0 tx=0).stop 2024-01-11T18:14:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 msgr2=0x7f78a41974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41974f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f78a0003730 tx=0x7f78a000f690 comp rx=0 tx=0).stop 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 shutdown_connections 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f78900a1ee0 0x7f78900a43a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78a4103120 0x7f78a419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a4101590 0x7f78a41974f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 --2- 172.21.15.7:0/4123114621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102780 0x7f78a4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 >> 172.21.15.7:0/4123114621 conn(0x7f78a40fcb00 msgr2=0x7f78a4105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 shutdown_connections 2024-01-11T18:14:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.756+0000 7f78ac5ff700 1 -- 172.21.15.7:0/4123114621 wait complete. 2024-01-11T18:14:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-11T18:14:16.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T18:14:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 -- 172.21.15.7:0/3049353558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8102780 msgr2=0x7f77b8102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 --2- 172.21.15.7:0/3049353558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8102780 0x7f77b8102be0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009990 tx=0x7f77a8009ca0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 -- 172.21.15.7:0/3049353558 shutdown_connections 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 --2- 172.21.15.7:0/3049353558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8103120 0x7f77b81099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 --2- 172.21.15.7:0/3049353558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b8101590 0x7f77b81019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 --2- 172.21.15.7:0/3049353558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8102780 0x7f77b8102be0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 -- 172.21.15.7:0/3049353558 >> 172.21.15.7:0/3049353558 conn(0x7f77b80fcb00 msgr2=0x7f77b80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 -- 172.21.15.7:0/3049353558 shutdown_connections 2024-01-11T18:14:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.984+0000 7f77c01ff700 1 -- 172.21.15.7:0/3049353558 wait complete. 2024-01-11T18:14:16.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.985+0000 7f77c01ff700 1 Processor -- start 2024-01-11T18:14:16.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.985+0000 7f77c01ff700 1 -- start start 2024-01-11T18:14:16.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.985+0000 7f77c01ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8101590 0x7f77b81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77c01ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 0x7f77b8199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77c01ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b8103120 0x7f77b81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77c01ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81a1240 con 0x7f77b8102780 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77c01ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81a1380 con 0x7f77b8103120 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77c01ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81a1510 con 0x7f77b8101590 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 0x7f77b8199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bdf9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8101590 0x7f77b81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 0x7f77b8199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45086/0 (socket says 172.21.15.7:45086) 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 -- 172.21.15.7:0/264371825 learned_addr learned my addr 172.21.15.7:0/264371825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77be79c700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b8103120 0x7f77b81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 -- 172.21.15.7:0/264371825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8101590 msgr2=0x7f77b81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8101590 0x7f77b81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 -- 172.21.15.7:0/264371825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b8103120 msgr2=0x7f77b81a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:16.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b8103120 0x7f77b81a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bd79a700 1 -- 172.21.15.7:0/264371825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80095f0 con 0x7f77b8102780 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.986+0000 7f77bdf9b700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8101590 0x7f77b81996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.987+0000 7f77bd79a700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 0x7f77b8199c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f77a80098e0 tx=0x7f77a8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.987+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a801d070 con 0x7f77b8102780 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.987+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b81a1790 con 0x7f77b8102780 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.987+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a8022470 con 0x7f77b8102780 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.987+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8017650 con 0x7f77b8102780 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.987+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b81a1c10 con 0x7f77b8102780 2024-01-11T18:14:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.988+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b80683f0 con 0x7f77b8102780 2024-01-11T18:14:16.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.990+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f77a80225e0 con 0x7f77b8102780 2024-01-11T18:14:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.991+0000 7f77aeffd700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f77a40a1ee0 0x7f77a40a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.992+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f77a80cf7a0 con 0x7f77b8102780 2024-01-11T18:14:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.992+0000 7f77bdf9b700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f77a40a1ee0 0x7f77a40a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:16.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.992+0000 7f77bdf9b700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f77a40a1ee0 0x7f77a40a43a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f77b81025e0 tx=0x7f77b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:16.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:16.994+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f77a80985d0 con 0x7f77b8102780 2024-01-11T18:14:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.257+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77b81a1f50 con 0x7f77b8102780 2024-01-11T18:14:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.258+0000 7f77aeffd700 1 -- 172.21.15.7:0/264371825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+11317 (secure 0 0 0) 0x7f77a809bbf0 con 0x7f77b8102780 2024-01-11T18:14:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:14:17.247 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2806336658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2806336658},{"type":"v1","addr":"172.21.15.7:6835","nonce":2806336658}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1543402898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1543402898},{"type":"v1","addr":"172.21.15.7:6837","nonce":1543402898}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/727242421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":727242421},{"type":"v1","addr":"172.21.15.7:6839","nonce":727242421}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":15897,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3100471640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3100471640},{"type":"v1","addr":"172.21.15.16:6837","nonce":3100471640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":24487,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3639639157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3639639157},{"type":"v1","addr":"172.21.15.38:6833","nonce":3639639157}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24488,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/189478188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":189478188},{"type":"v1","addr":"172.21.15.16:6833","nonce":189478188}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-11T17:56:08.339093+0000","modified":"2024-01-11T18:13:56.388677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24601,"mds_2":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/4278122554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4278122554},{"type":"v1","addr":"172.21.15.16:6835","nonce":4278122554}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6835/1587519060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1587519060},{"type":"v1","addr":"172.21.15.38:6835","nonce":1587519060}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/2455039829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2455039829},{"type":"v1","addr":"172.21.15.38:6837","nonce":2455039829}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":45,"addr":"172.21.15.38:6839/443773776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":443773776},{"type":"v1","addr":"172.21.15.38:6839","nonce":443773776}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24601":{"gid":24601,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":43,"addr":"172.21.15.7:6841/898637958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":898637958},{"type":"v1","addr":"172.21.15.7:6841","nonce":898637958}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/1892769619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1892769619},{"type":"v1","addr":"172.21.15.16:6839","nonce":1892769619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f77a40a1ee0 msgr2=0x7f77a40a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f77a40a1ee0 0x7f77a40a43a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f77b81025e0 tx=0x7f77b4009260 comp rx=0 tx=0).stop 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 msgr2=0x7f77b8199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 0x7f77b8199c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f77a80098e0 tx=0x7f77a8005720 comp rx=0 tx=0).stop 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 shutdown_connections 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f77a40a1ee0 0x7f77a40a43a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77b8101590 0x7f77b81996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b8103120 0x7f77b81a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 --2- 172.21.15.7:0/264371825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77b8102780 0x7f77b8199c00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.260+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 >> 172.21.15.7:0/264371825 conn(0x7f77b80fcb00 msgr2=0x7f77b8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:17.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.261+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 shutdown_connections 2024-01-11T18:14:17.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.261+0000 7f77c01ff700 1 -- 172.21.15.7:0/264371825 wait complete. 2024-01-11T18:14:17.251 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-11T18:14:17.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.486+0000 7f6fb95ff700 1 -- 172.21.15.7:0/2116505113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 msgr2=0x7f6fb4102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.486+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/2116505113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4102be0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80099c0 tx=0x7f6fa8009cd0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 -- 172.21.15.7:0/2116505113 shutdown_connections 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/2116505113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 0x7f6fb41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/2116505113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb4103120 0x7f6fb41099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/2116505113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4102be0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 -- 172.21.15.7:0/2116505113 >> 172.21.15.7:0/2116505113 conn(0x7f6fb40fcb00 msgr2=0x7f6fb40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:17.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 -- 172.21.15.7:0/2116505113 shutdown_connections 2024-01-11T18:14:17.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.487+0000 7f6fb95ff700 1 -- 172.21.15.7:0/2116505113 wait complete. 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 Processor -- start 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 -- start start 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 0x7f6fb40753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb4103120 0x7f6fb4073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40746e0 con 0x7f6fb4102780 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4074850 con 0x7f6fb4103120 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb95ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41a1560 con 0x7f6fb4101590 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 0x7f6fb40753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45102/0 (socket says 172.21.15.7:45102) 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb27fc700 1 -- 172.21.15.7:0/3919453198 learned_addr learned my addr 172.21.15.7:0/3919453198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.488+0000 7f6fb37fe700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb4103120 0x7f6fb4073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb2ffd700 1 -- 172.21.15.7:0/3919453198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb4103120 msgr2=0x7f6fb4073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb2ffd700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb4103120 0x7f6fb4073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb2ffd700 1 -- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 msgr2=0x7f6fb4073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb2ffd700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb2ffd700 1 -- 172.21.15.7:0/3919453198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80095f0 con 0x7f6fb4101590 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb27fc700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb2ffd700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 0x7f6fb40753c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400e9c0 tx=0x7f6fa400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4004220 con 0x7f6fb4101590 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb41a1840 con 0x7f6fb4101590 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa4004380 con 0x7f6fb4101590 2024-01-11T18:14:17.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4003be0 con 0x7f6fb4101590 2024-01-11T18:14:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.489+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb41a1d50 con 0x7f6fb4101590 2024-01-11T18:14:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.490+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4108a60 con 0x7f6fb4101590 2024-01-11T18:14:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.493+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f6fa4003d40 con 0x7f6fb4101590 2024-01-11T18:14:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.494+0000 7f6f9bfff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f9c0a1ee0 0x7f6f9c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.494+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6fa4014070 con 0x7f6fb4101590 2024-01-11T18:14:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.494+0000 7f6fb27fc700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f9c0a1ee0 0x7f6f9c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.495+0000 7f6fb27fc700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f9c0a1ee0 0x7f6f9c0a43a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80095c0 tx=0x7f6fa8009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:17.484 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:17 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4123114621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T18:14:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.496+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f6fa4096e40 con 0x7f6fb4101590 2024-01-11T18:14:17.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:17 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4123114621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T18:14:17.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:17 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4123114621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T18:14:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.715+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fb40683f0 con 0x7f6fb4101590 2024-01-11T18:14:17.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.717+0000 7f6f9bfff700 1 -- 172.21.15.7:0/3919453198 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24382 (secure 0 0 0) 0x7f6fa409a460 con 0x7f6fb4101590 2024-01-11T18:14:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T18:14:17.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"a76d0c8a-b0a9-11ee-95ab-87774f69a715","created":"2024-01-11T17:49:24.248673+0000","modified":"2024-01-11T18:02:43.786606+0000","last_up_change":"2024-01-11T17:55:28.929384+0000","last_in_change":"2024-01-11T17:55:11.855577+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T17:52:31.107037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T17:56:06.693316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T17:56:07.769186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-11T17:56:11.302707+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e34d29ca-d1a5-4771-a752-5acca1c7d041","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6803","nonce":953692014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6805","nonce":953692014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6809","nonce":953692014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":953692014},{"type":"v1","addr":"172.21.15.7:6807","nonce":953692014}]},"public_addr":"172.21.15.7:6803/953692014","cluster_addr":"172.21.15.7:6805/953692014","heartbeat_back_addr":"172.21.15.7:6809/953692014","heartbeat_front_addr":"172.21.15.7:6807/953692014","state":["exists","up"]},{"osd":1,"uuid":"7b918edd-6f35-4186-83f6-8dea129fa4d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6801","nonce":2714939694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6803","nonce":2714939694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6807","nonce":2714939694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2714939694},{"type":"v1","addr":"172.21.15.16:6805","nonce":2714939694}]},"public_addr":"172.21.15.16:6801/2714939694","cluster_addr":"172.21.15.16:6803/2714939694","heartbeat_back_addr":"172.21.15.16:6807/2714939694","heartbeat_front_addr":"172.21.15.16:6805/2714939694","state":["exists","up"]},{"osd":2,"uuid":"85ec6460-ec16-4142-abff-60b2b9d83550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6801","nonce":2733698254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6803","nonce":2733698254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6807","nonce":2733698254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2733698254},{"type":"v1","addr":"172.21.15.38:6805","nonce":2733698254}]},"public_addr":"172.21.15.38:6801/2733698254","cluster_addr":"172.21.15.38:6803/2733698254","heartbeat_back_addr":"172.21.15.38:6807/2733698254","heartbeat_front_addr":"172.21.15.38:6805/2733698254","state":["exists","up"]},{"osd":3,"uuid":"ad9f9282-a8c3-43df-954d-8bc2a5854cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6811","nonce":1132146736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6813","nonce":1132146736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6817","nonce":1132146736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1132146736},{"type":"v1","addr":"172.21.15.7:6815","nonce":1132146736}]},"public_addr":"172.21.15.7:6811/1132146736","cluster_addr":"172.21.15.7:6813/1132146736","heartbeat_back_addr":"172.21.15.7:6817/1132146736","heartbeat_front_addr":"172.21.15.7:6815/1132146736","state":["exists","up"]},{"osd":4,"uuid":"3740f4c2-4680-4dce-b64a-65e8405a98e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6809","nonce":1933820349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6811","nonce":1933820349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6815","nonce":1933820349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1933820349},{"type":"v1","addr":"172.21.15.16:6813","nonce":1933820349}]},"public_addr":"172.21.15.16:6809/1933820349","cluster_addr":"172.21.15.16:6811/1933820349","heartbeat_back_addr":"172.21.15.16:6815/1933820349","heartbeat_front_addr":"172.21.15.16:6813/1933820349","state":["exists","up"]},{"osd":5,"uuid":"e6f9e392-3e69-4e4a-ae9c-07206a3c8f6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6809","nonce":3973290350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6811","nonce":3973290350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6815","nonce":3973290350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3973290350},{"type":"v1","addr":"172.21.15.38:6813","nonce":3973290350}]},"public_addr":"172.21.15.38:6809/3973290350","cluster_addr":"172.21.15.38:6811/3973290350","heartbeat_back_addr":"172.21.15.38:6815/3973290350","heartbeat_front_addr":"172.21.15.38:6813/3973290350","state":["exists","up"]},{"osd":6,"uuid":"731711fe-aa80-4c77-a022-b1a16f24306b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6819","nonce":1282115079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6821","nonce":1282115079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6825","nonce":1282115079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1282115079},{"type":"v1","addr":"172.21.15.7:6823","nonce":1282115079}]},"public_addr":"172.21.15.7:6819/1282115079","cluster_addr":"172.21.15.7:6821/1282115079","heartbeat_back_addr":"172.21.15.7:6825/1282115079","heartbeat_front_addr":"172.21.15.7:6823/1282115079","state":["exists","up"]},{"osd":7,"uuid":"e1e933c6-92f2-4974-83cf-633e3530103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6817","nonce":760747298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6819","nonce":760747298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6823","nonce":760747298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":760747298},{"type":"v1","addr":"172.21.15.16:6821","nonce":760747298}]},"public_addr":"172.21.15.16:6817/760747298","cluster_addr":"172.21.15.16:6819/760747298","heartbeat_back_addr":"172.21.15.16:6823/760747298","heartbeat_front_addr":"172.21.15.16:6821/760747298","state":["exists","up"]},{"osd":8,"uuid":"e3bb6d44-a03b-4e41-a286-3f6e2104397d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6817","nonce":659578485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6819","nonce":659578485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6823","nonce":659578485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":659578485},{"type":"v1","addr":"172.21.15.38:6821","nonce":659578485}]},"public_addr":"172.21.15.38:6817/659578485","cluster_addr":"172.21.15.38:6819/659578485","heartbeat_back_addr":"172.21.15.38:6823/659578485","heartbeat_front_addr":"172.21.15.38:6821/659578485","state":["exists","up"]},{"osd":9,"uuid":"fb7e8378-c57e-4c86-9184-fe68617c8169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6827","nonce":1883138205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6829","nonce":1883138205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6833","nonce":1883138205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1883138205},{"type":"v1","addr":"172.21.15.7:6831","nonce":1883138205}]},"public_addr":"172.21.15.7:6827/1883138205","cluster_addr":"172.21.15.7:6829/1883138205","heartbeat_back_addr":"172.21.15.7:6833/1883138205","heartbeat_front_addr":"172.21.15.7:6831/1883138205","state":["exists","up"]},{"osd":10,"uuid":"bf5a2cb2-377f-4650-9971-178ba7b1b176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6825","nonce":2304237438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6827","nonce":2304237438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6831","nonce":2304237438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2304237438},{"type":"v1","addr":"172.21.15.16:6829","nonce":2304237438}]},"public_addr":"172.21.15.16:6825/2304237438","cluster_addr":"172.21.15.16:6827/2304237438","heartbeat_back_addr":"172.21.15.16:6831/2304237438","heartbeat_front_addr":"172.21.15.16:6829/2304237438","state":["exists","up"]},{"osd":11,"uuid":"5dc65b78-877e-4fc7-8aef-6d5d8af09353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6825","nonce":3838329833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6827","nonce":3838329833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6831","nonce":3838329833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3838329833},{"type":"v1","addr":"172.21.15.38:6829","nonce":3838329833}]},"public_addr":"172.21.15.38:6825/3838329833","cluster_addr":"172.21.15.38:6827/3838329833","heartbeat_back_addr":"172.21.15.38:6831/3838329833","heartbeat_front_addr":"172.21.15.38:6829/3838329833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:51:48.330656+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:07.873700+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:26.373845+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:52:48.330810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:08.148657+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:27.000617+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:53:49.384707+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:07.678141+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:26.599690+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:54:48.917842+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:08.244812+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T17:55:27.121517+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3570192633":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6801/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/1912965293":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/3811163237":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/3086228555":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6800/867047662":"2024-01-12T17:50:24.497330+0000","172.21.15.7:0/4071850608":"2024-01-12T17:50:24.497330+0000","172.21.15.7:6800/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:6801/1981932676":"2024-01-12T17:49:57.975302+0000","172.21.15.7:0/113093279":"2024-01-12T17:49:57.975302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T18:14:17.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f9c0a1ee0 msgr2=0x7f6f9c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f9c0a1ee0 0x7f6f9c0a43a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80095c0 tx=0x7f6fa8009530 comp rx=0 tx=0).stop 2024-01-11T18:14:17.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 msgr2=0x7f6fb40753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 0x7f6fb40753c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400e9c0 tx=0x7f6fa400ecd0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 shutdown_connections 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f6f9c0a1ee0 0x7f6f9c0a43a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fb4101590 0x7f6fb40753c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb4103120 0x7f6fb4073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 --2- 172.21.15.7:0/3919453198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb4102780 0x7f6fb4073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 >> 172.21.15.7:0/3919453198 conn(0x7f6fb40fcb00 msgr2=0x7f6fb4105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 shutdown_connections 2024-01-11T18:14:17.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.720+0000 7f6fb95ff700 1 -- 172.21.15.7:0/3919453198 wait complete. 2024-01-11T18:14:17.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 -- 172.21.15.7:0/878737218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060104940 msgr2=0x7f4060106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 --2- 172.21.15.7:0/878737218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060104940 0x7f4060106d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4050009990 tx=0x7f4050009ca0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 -- 172.21.15.7:0/878737218 shutdown_connections 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 --2- 172.21.15.7:0/878737218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 0x7f4060104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 --2- 172.21.15.7:0/878737218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060107270 0x7f40601096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 --2- 172.21.15.7:0/878737218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060104940 0x7f4060106d30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 -- 172.21.15.7:0/878737218 >> 172.21.15.7:0/878737218 conn(0x7f40600fba00 msgr2=0x7f40600fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 -- 172.21.15.7:0/878737218 shutdown_connections 2024-01-11T18:14:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.946+0000 7f40683ff700 1 -- 172.21.15.7:0/878737218 wait complete. 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.947+0000 7f40683ff700 1 Processor -- start 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.947+0000 7f40683ff700 1 -- start start 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f40683ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 0x7f4060199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f40683ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 0x7f4060199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f40683ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060107270 0x7f40601a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f40683ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601a11f0 con 0x7f4060107270 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f40683ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601a1330 con 0x7f4060104940 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f40683ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601a14c0 con 0x7f4060102010 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f406699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060107270 0x7f40601a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f406599a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 0x7f4060199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f406619b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 0x7f4060199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f406599a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 0x7f4060199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58074/0 (socket says 172.21.15.7:58074) 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.948+0000 7f406599a700 1 -- 172.21.15.7:0/2096055356 learned_addr learned my addr 172.21.15.7:0/2096055356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406599a700 1 -- 172.21.15.7:0/2096055356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 msgr2=0x7f4060199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406599a700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 0x7f4060199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406599a700 1 -- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060107270 msgr2=0x7f40601a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406599a700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060107270 0x7f40601a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406599a700 1 -- 172.21.15.7:0/2096055356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500095f0 con 0x7f4060104940 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406619b700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 0x7f4060199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:17.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f406599a700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 0x7f4060199bb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4050005910 tx=0x7f4050009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405001d070 con 0x7f4060104940 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40601a1740 con 0x7f4060104940 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4050021410 con 0x7f4060104940 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050005720 con 0x7f4060104940 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.949+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40601a1b60 con 0x7f4060104940 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.950+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4060108e70 con 0x7f4060104940 2024-01-11T18:14:17.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.952+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f4050021580 con 0x7f4060104940 2024-01-11T18:14:17.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.954+0000 7f40577fe700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f404c0a1ee0 0x7f404c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:17.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.954+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f40500cf750 con 0x7f4060104940 2024-01-11T18:14:17.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.954+0000 7f406619b700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f404c0a1ee0 0x7f404c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:17.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.955+0000 7f406619b700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f404c0a1ee0 0x7f404c0a43a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f405c011490 tx=0x7f405c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:17.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:17.956+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f40500984f0 con 0x7f4060104940 2024-01-11T18:14:18.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.216+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f406010ce40 con 0x7f4060104940 2024-01-11T18:14:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.225+0000 7f40577fe700 1 -- 172.21.15.7:0/2096055356 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f405009bb10 con 0x7f4060104940 2024-01-11T18:14:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f404c0a1ee0 msgr2=0x7f404c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f404c0a1ee0 0x7f404c0a43a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f405c011490 tx=0x7f405c009260 comp rx=0 tx=0).stop 2024-01-11T18:14:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 msgr2=0x7f4060199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 0x7f4060199bb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4050005910 tx=0x7f4050009580 comp rx=0 tx=0).stop 2024-01-11T18:14:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 shutdown_connections 2024-01-11T18:14:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f404c0a1ee0 0x7f404c0a43a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4060102010 0x7f4060199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060107270 0x7f40601a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 --2- 172.21.15.7:0/2096055356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4060104940 0x7f4060199bb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 >> 172.21.15.7:0/2096055356 conn(0x7f40600fba00 msgr2=0x7f40601017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 shutdown_connections 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.228+0000 7f40683ff700 1 -- 172.21.15.7:0/2096055356 wait complete. 2024-01-11T18:14:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T18:14:18.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T18:14:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 -- 172.21.15.7:0/3078455148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 msgr2=0x7efc580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 --2- 172.21.15.7:0/3078455148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc580744f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009990 tx=0x7efc4c009ca0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 -- 172.21.15.7:0/3078455148 shutdown_connections 2024-01-11T18:14:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 --2- 172.21.15.7:0/3078455148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58109470 0x7efc5810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 --2- 172.21.15.7:0/3078455148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58075650 0x7efc58073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 --2- 172.21.15.7:0/3078455148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc580744f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 -- 172.21.15.7:0/3078455148 >> 172.21.15.7:0/3078455148 conn(0x7efc580fcb00 msgr2=0x7efc580fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 -- 172.21.15.7:0/3078455148 shutdown_connections 2024-01-11T18:14:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.462+0000 7efc5dfff700 1 -- 172.21.15.7:0/3078455148 wait complete. 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.463+0000 7efc5dfff700 1 Processor -- start 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 -- start start 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc58199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 0x7efc58199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58109470 0x7efc581a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581a11c0 con 0x7efc58075650 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581a1300 con 0x7efc58109470 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc5dfff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581a1490 con 0x7efc58074070 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 0x7efc58199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 0x7efc58199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45150/0 (socket says 172.21.15.7:45150) 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc56ffd700 1 -- 172.21.15.7:0/1360256661 learned_addr learned my addr 172.21.15.7:0/1360256661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc577fe700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc58199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.464+0000 7efc57fff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58109470 0x7efc581a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc56ffd700 1 -- 172.21.15.7:0/1360256661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 msgr2=0x7efc58199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc56ffd700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc58199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc56ffd700 1 -- 172.21.15.7:0/1360256661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58109470 msgr2=0x7efc581a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc56ffd700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58109470 0x7efc581a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc56ffd700 1 -- 172.21.15.7:0/1360256661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48009520 con 0x7efc58075650 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc577fe700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc58199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc57fff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58109470 0x7efc581a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc56ffd700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 0x7efc58199b80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098e0 tx=0x7efc4c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c01d070 con 0x7efc58075650 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c0095f0 con 0x7efc58075650 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4c022470 con 0x7efc58075650 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.465+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c017650 con 0x7efc58075650 2024-01-11T18:14:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.466+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc581a19b0 con 0x7efc58075650 2024-01-11T18:14:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.466+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc58108230 con 0x7efc58075650 2024-01-11T18:14:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.468+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7efc4c022850 con 0x7efc58075650 2024-01-11T18:14:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.470+0000 7efc54ff9700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc400a1ee0 0x7efc400a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.470+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9307+0+0 (secure 0 0 0) 0x7efc4c0cf330 con 0x7efc58075650 2024-01-11T18:14:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.470+0000 7efc577fe700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc400a1ee0 0x7efc400a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.470+0000 7efc577fe700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc400a1ee0 0x7efc400a43a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efc48009fd0 tx=0x7efc48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.473+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7efc4c097fd0 con 0x7efc58075650 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: ignoring --setuser ceph since I am not root 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: ignoring --setgroup ceph since I am not root 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-mds[80231]: -- 172.21.15.16:0/3752646313 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56124fd2d520 con 0x56124fd02800 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: starting mds.b at 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-mon[43195]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/264371825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T18:14:18.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/3919453198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:14:18.486 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e[79529]: ignoring --setuser ceph since I am not root 2024-01-11T18:14:18.486 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e[79529]: ignoring --setgroup ceph since I am not root 2024-01-11T18:14:18.486 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-mds[79552]: -- 172.21.15.16:0/2920605717 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d9d903d520 con 0x55d9d9012800 2024-01-11T18:14:18.486 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:14:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e[79529]: starting mds.e at 2024-01-11T18:14:18.515 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f[79809]: ignoring --setuser ceph since I am not root 2024-01-11T18:14:18.515 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f[79809]: ignoring --setgroup ceph since I am not root 2024-01-11T18:14:18.515 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-mds[79832]: -- 172.21.15.38:0/1358925239 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ffa734d520 con 0x55ffa7323000 2024-01-11T18:14:18.515 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f[79809]: starting mds.f at 2024-01-11T18:14:18.515 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: ignoring --setuser ceph since I am not root 2024-01-11T18:14:18.515 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: ignoring --setgroup ceph since I am not root 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-mds[80516]: -- 172.21.15.38:0/2915629789 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5577ae4f7520 con 0x5577ae4cd000 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: starting mds.i at 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-mon[43582]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/264371825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/3919453198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c[80150]: ignoring --setuser ceph since I am not root 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c[80150]: ignoring --setgroup ceph since I am not root 2024-01-11T18:14:18.516 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-mds[80173]: -- 172.21.15.38:0/3412169237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5609226d9520 con 0x5609226ae800 2024-01-11T18:14:18.517 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:14:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c[80150]: starting mds.c at 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-mon[33361]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/264371825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/3919453198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T18:14:18.217+0000 7fec68a56700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: ignoring --setuser ceph since I am not root 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: ignoring --setgroup ceph since I am not root 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-mds[81185]: -- 172.21.15.7:0/2342830353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fcfe11d520 con 0x55fcfe0f5000 2024-01-11T18:14:18.560 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:14:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: starting mds.a at 2024-01-11T18:14:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:18.736+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efc5802d380 con 0x7efc58075650 2024-01-11T18:14:19.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.223+0000 7efc54ff9700 1 -- 172.21.15.7:0/1360256661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc4c09b5f0 con 0x7efc58075650 2024-01-11T18:14:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc400a1ee0 msgr2=0x7efc400a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc400a1ee0 0x7efc400a43a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efc48009fd0 tx=0x7efc48009260 comp rx=0 tx=0).stop 2024-01-11T18:14:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 msgr2=0x7efc58199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 0x7efc58199b80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098e0 tx=0x7efc4c005720 comp rx=0 tx=0).stop 2024-01-11T18:14:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 shutdown_connections 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7efc400a1ee0 0x7efc400a43a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc58074070 0x7efc58199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc58109470 0x7efc581a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 --2- 172.21.15.7:0/1360256661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc58075650 0x7efc58199b80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 >> 172.21.15.7:0/1360256661 conn(0x7efc580fcb00 msgr2=0x7efc58105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 shutdown_connections 2024-01-11T18:14:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.227+0000 7efc5dfff700 1 -- 172.21.15.7:0/1360256661 wait complete. 2024-01-11T18:14:19.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 -- 172.21.15.7:0/885010146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 msgr2=0x7f75b4102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 --2- 172.21.15.7:0/885010146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4102c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75a80099c0 tx=0x7f75a8009cd0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 -- 172.21.15.7:0/885010146 shutdown_connections 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 --2- 172.21.15.7:0/885010146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b4103140 0x7f75b41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 --2- 172.21.15.7:0/885010146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b41015b0 0x7f75b41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 --2- 172.21.15.7:0/885010146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4102c00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.465+0000 7f75b99ff700 1 -- 172.21.15.7:0/885010146 >> 172.21.15.7:0/885010146 conn(0x7f75b40fcb00 msgr2=0x7f75b40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.466+0000 7f75b99ff700 1 -- 172.21.15.7:0/885010146 shutdown_connections 2024-01-11T18:14:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.466+0000 7f75b99ff700 1 -- 172.21.15.7:0/885010146 wait complete. 2024-01-11T18:14:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.467+0000 7f75b99ff700 1 Processor -- start 2024-01-11T18:14:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.467+0000 7f75b99ff700 1 -- start start 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b99ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 0x7f75b41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b99ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b99ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b4103140 0x7f75b41a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b99ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41a1240 con 0x7f75b41027a0 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b99ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41a1380 con 0x7f75b4103140 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b99ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41a1510 con 0x7f75b41015b0 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 0x7f75b41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 0x7f75b41996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.7:56312/0 (socket says 172.21.15.7:56312) 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b2ffd700 1 -- 172.21.15.7:0/1484628909 learned_addr learned my addr 172.21.15.7:0/1484628909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b37fe700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b4103140 0x7f75b41a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b27fc700 1 -- 172.21.15.7:0/1484628909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 msgr2=0x7f75b41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b27fc700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 0x7f75b41996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b27fc700 1 -- 172.21.15.7:0/1484628909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b4103140 msgr2=0x7f75b41a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b27fc700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b4103140 0x7f75b41a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b27fc700 1 -- 172.21.15.7:0/1484628909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80095f0 con 0x7f75b41027a0 2024-01-11T18:14:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.468+0000 7f75b2ffd700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 0x7f75b41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T18:14:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.469+0000 7f75b27fc700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4199c00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fd0 tx=0x7f75a800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.469+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a801d070 con 0x7f75b41027a0 2024-01-11T18:14:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.469+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a800bda0 con 0x7f75b41027a0 2024-01-11T18:14:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.469+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b41a1790 con 0x7f75b41027a0 2024-01-11T18:14:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.469+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8004b00 con 0x7f75b41027a0 2024-01-11T18:14:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.469+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b41a1bb0 con 0x7f75b41027a0 2024-01-11T18:14:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.470+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b40683f0 con 0x7f75b41027a0 2024-01-11T18:14:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.472+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f75a8004c60 con 0x7f75b41027a0 2024-01-11T18:14:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.473+0000 7f75b89fd700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f759c0a1ee0 0x7f759c0a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.474+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f75a80ceb60 con 0x7f75b41027a0 2024-01-11T18:14:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.474+0000 7f75b2ffd700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f759c0a1ee0 0x7f759c0a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.474+0000 7f75b2ffd700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f759c0a1ee0 0x7f759c0a43a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f75a4005950 tx=0x7f75a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.477+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f75a8097800 con 0x7f75b41027a0 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/2096055356' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: osdmap e90: 12 total, 12 up, 12 in 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-01-11T18:14:19.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:19 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1360256661' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/2096055356' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: osdmap e90: 12 total, 12 up, 12 in 2024-01-11T18:14:19.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-01-11T18:14:19.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:19 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1360256661' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/2096055356' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: osdmap e90: 12 total, 12 up, 12 in 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-01-11T18:14:19.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:19 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1360256661' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-11T18:14:19.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:19.694+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75b41a2070 con 0x7f75b41027a0 2024-01-11T18:14:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.225+0000 7f75b89fd700 1 -- 172.21.15.7:0/1484628909 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f75a809ae20 con 0x7f75b41027a0 2024-01-11T18:14:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f759c0a1ee0 msgr2=0x7f759c0a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f759c0a1ee0 0x7f759c0a43a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f75a4005950 tx=0x7f75a4009260 comp rx=0 tx=0).stop 2024-01-11T18:14:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 msgr2=0x7f75b4199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4199c00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fd0 tx=0x7f75a800fa00 comp rx=0 tx=0).stop 2024-01-11T18:14:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 shutdown_connections 2024-01-11T18:14:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f759c0a1ee0 0x7f759c0a43a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b41015b0 0x7f75b41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b4103140 0x7f75b41a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 --2- 172.21.15.7:0/1484628909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75b41027a0 0x7f75b4199c00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.230+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 >> 172.21.15.7:0/1484628909 conn(0x7f75b40fcb00 msgr2=0x7f75b4073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.232+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 shutdown_connections 2024-01-11T18:14:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.232+0000 7f75b99ff700 1 -- 172.21.15.7:0/1484628909 wait complete. 2024-01-11T18:14:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T18:14:20.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T18:14:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.473+0000 7f3760bff700 1 -- 172.21.15.7:0/2398627104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1027a0 msgr2=0x7f375c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.473+0000 7f3760bff700 1 --2- 172.21.15.7:0/2398627104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1027a0 0x7f375c102c00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3750009920 tx=0x7f3750009c30 comp rx=0 tx=0).stop 2024-01-11T18:14:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 -- 172.21.15.7:0/2398627104 shutdown_connections 2024-01-11T18:14:20.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 --2- 172.21.15.7:0/2398627104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f375c1015b0 0x7f375c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 --2- 172.21.15.7:0/2398627104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c103140 0x7f375c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 --2- 172.21.15.7:0/2398627104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1027a0 0x7f375c102c00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 -- 172.21.15.7:0/2398627104 >> 172.21.15.7:0/2398627104 conn(0x7f375c0fcb00 msgr2=0x7f375c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 -- 172.21.15.7:0/2398627104 shutdown_connections 2024-01-11T18:14:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.474+0000 7f3760bff700 1 -- 172.21.15.7:0/2398627104 wait complete. 2024-01-11T18:14:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 Processor -- start 2024-01-11T18:14:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 -- start start 2024-01-11T18:14:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 0x7f375c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c1027a0 0x7f375c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f375c103140 0x7f375c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a12b0 con 0x7f375c1015b0 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a13f0 con 0x7f375c1027a0 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f3760bff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a1580 con 0x7f375c103140 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f375a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 0x7f375c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.475+0000 7f375a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 0x7f375c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45188/0 (socket says 172.21.15.7:45188) 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 -- 172.21.15.7:0/879565593 learned_addr learned my addr 172.21.15.7:0/879565593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f3759d9b700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c1027a0 0x7f375c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375ad9d700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f375c103140 0x7f375c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 -- 172.21.15.7:0/879565593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f375c103140 msgr2=0x7f375c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f375c103140 0x7f375c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 -- 172.21.15.7:0/879565593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c1027a0 msgr2=0x7f375c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c1027a0 0x7f375c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 -- 172.21.15.7:0/879565593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c009520 con 0x7f375c1015b0 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f375a59c700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 0x7f375c199730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f374c00ea50 tx=0x7f374c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00cbd0 con 0x7f375c1015b0 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f374c00cd30 con 0x7f375c1015b0 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.476+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3750009520 con 0x7f375c1015b0 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.477+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c018680 con 0x7f375c1015b0 2024-01-11T18:14:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.477+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c1a1be0 con 0x7f375c1015b0 2024-01-11T18:14:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.478+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c0683f0 con 0x7f375c1015b0 2024-01-11T18:14:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.480+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f374c010430 con 0x7f375c1015b0 2024-01-11T18:14:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.482+0000 7f374b7fe700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f37440a1ee0 0x7f37440a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.482+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8811+0+0 (secure 0 0 0) 0x7f374c014070 con 0x7f375c1015b0 2024-01-11T18:14:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.482+0000 7f3759d9b700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f37440a1ee0 0x7f37440a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.483+0000 7f3759d9b700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f37440a1ee0 0x7f37440a43a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f375000fa40 tx=0x7f3750019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.486+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f374c096ed0 con 0x7f375c1015b0 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: Cluster is now healthy 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1360256661' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6834/3999661829,v1:172.21.15.38:6837/3999661829] up:boot 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6834/705842138,v1:172.21.15.16:6838/705842138] up:boot 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.16:6835/1522041663,v1:172.21.15.16:6839/1522041663] up:boot 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6836/1829582591,v1:172.21.15.38:6839/1829582591] up:boot 2024-01-11T18:14:20.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.38:6835/3546798881,v1:172.21.15.38:6838/3546798881] up:boot 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: mds.? [v2:172.21.15.7:6840/3135438805,v1:172.21.15.7:6841/3135438805] up:boot 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: fsmap 12 up:standby 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-11T18:14:20.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:20 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1484628909' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: Cluster is now healthy 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1360256661' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6834/3999661829,v1:172.21.15.38:6837/3999661829] up:boot 2024-01-11T18:14:20.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6834/705842138,v1:172.21.15.16:6838/705842138] up:boot 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.16:6835/1522041663,v1:172.21.15.16:6839/1522041663] up:boot 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6836/1829582591,v1:172.21.15.38:6839/1829582591] up:boot 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.38:6835/3546798881,v1:172.21.15.38:6838/3546798881] up:boot 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: mds.? [v2:172.21.15.7:6840/3135438805,v1:172.21.15.7:6841/3135438805] up:boot 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: fsmap 12 up:standby 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-11T18:14:20.516 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-11T18:14:20.517 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:20 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1484628909' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: Cluster is now healthy 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1360256661' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6834/3999661829,v1:172.21.15.38:6837/3999661829] up:boot 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6834/705842138,v1:172.21.15.16:6838/705842138] up:boot 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.16:6835/1522041663,v1:172.21.15.16:6839/1522041663] up:boot 2024-01-11T18:14:20.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6836/1829582591,v1:172.21.15.38:6839/1829582591] up:boot 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.38:6835/3546798881,v1:172.21.15.38:6838/3546798881] up:boot 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: mds.? [v2:172.21.15.7:6840/3135438805,v1:172.21.15.7:6841/3135438805] up:boot 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: fsmap 12 up:standby 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='mgr.14152 172.21.15.7:0/3138737544' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-11T18:14:20.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:20 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1484628909' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:20.704+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f375c10ce10 con 0x7f375c1015b0 2024-01-11T18:14:21.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.231+0000 7f374b7fe700 1 -- 172.21.15.7:0/879565593 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f374c09a4f0 con 0x7f375c1015b0 2024-01-11T18:14:21.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f37440a1ee0 msgr2=0x7f37440a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:21.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f37440a1ee0 0x7f37440a43a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f375000fa40 tx=0x7f3750019040 comp rx=0 tx=0).stop 2024-01-11T18:14:21.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 msgr2=0x7f375c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:21.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 0x7f375c199730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f374c00ea50 tx=0x7f374c00ed60 comp rx=0 tx=0).stop 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 shutdown_connections 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f37440a1ee0 0x7f37440a43a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f375c103140 0x7f375c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f375c1015b0 0x7f375c199730 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 --2- 172.21.15.7:0/879565593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c1027a0 0x7f375c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 >> 172.21.15.7:0/879565593 conn(0x7f375c0fcb00 msgr2=0x7f375c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 shutdown_connections 2024-01-11T18:14:21.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.236+0000 7f3760bff700 1 -- 172.21.15.7:0/879565593 wait complete. 2024-01-11T18:14:21.227 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T18:14:21.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-01-11T18:14:21.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 -- 172.21.15.7:0/3326501333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc103140 msgr2=0x7f67dc1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 --2- 172.21.15.7:0/3326501333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc103140 0x7f67dc1099f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67d400b0d0 tx=0x7f67d400b3e0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 -- 172.21.15.7:0/3326501333 shutdown_connections 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 --2- 172.21.15.7:0/3326501333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc103140 0x7f67dc1099f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 --2- 172.21.15.7:0/3326501333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc1015b0 0x7f67dc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 --2- 172.21.15.7:0/3326501333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67dc1027a0 0x7f67dc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 -- 172.21.15.7:0/3326501333 >> 172.21.15.7:0/3326501333 conn(0x7f67dc0fcb00 msgr2=0x7f67dc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 -- 172.21.15.7:0/3326501333 shutdown_connections 2024-01-11T18:14:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.492+0000 7f67e19ff700 1 -- 172.21.15.7:0/3326501333 wait complete. 2024-01-11T18:14:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.493+0000 7f67e19ff700 1 Processor -- start 2024-01-11T18:14:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.493+0000 7f67e19ff700 1 -- start start 2024-01-11T18:14:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67e19ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc1015b0 0x7f67dc199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67e19ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 0x7f67dc199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67e19ff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67dc103140 0x7f67dc1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67e19ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc1a1310 con 0x7f67dc1027a0 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67e19ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc1a1450 con 0x7f67dc1015b0 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67e19ff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc1a15e0 con 0x7f67dc103140 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 0x7f67dc199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 0x7f67dc199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45204/0 (socket says 172.21.15.7:45204) 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 -- 172.21.15.7:0/4093173820 learned_addr learned my addr 172.21.15.7:0/4093173820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67daffd700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc1015b0 0x7f67dc199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67db7fe700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67dc103140 0x7f67dc1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 -- 172.21.15.7:0/4093173820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67dc103140 msgr2=0x7f67dc1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67dc103140 0x7f67dc1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 -- 172.21.15.7:0/4093173820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc1015b0 msgr2=0x7f67dc199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc1015b0 0x7f67dc199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.494+0000 7f67da7fc700 1 -- 172.21.15.7:0/4093173820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009d00 con 0x7f67dc1027a0 2024-01-11T18:14:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.495+0000 7f67da7fc700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 0x7f67dc199cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9a0 tx=0x7f67d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.495+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0009790 con 0x7f67dc1027a0 2024-01-11T18:14:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.495+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d0010430 con 0x7f67dc1027a0 2024-01-11T18:14:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.495+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc1a18c0 con 0x7f67dc1027a0 2024-01-11T18:14:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.495+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0018680 con 0x7f67dc1027a0 2024-01-11T18:14:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.495+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc1a1dd0 con 0x7f67dc1027a0 2024-01-11T18:14:21.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:21 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/1484628909' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-11T18:14:21.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:21 smithi016 ceph-mon[43195]: osdmap e91: 12 total, 12 up, 12 in 2024-01-11T18:14:21.487 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:21 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/879565593' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.496+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc0683f0 con 0x7f67dc1027a0 2024-01-11T18:14:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.498+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 136624+0+0 (secure 0 0 0) 0x7f67d00105a0 con 0x7f67dc1027a0 2024-01-11T18:14:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.499+0000 7f67e09fd700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f67c40a1ee0 0x7f67c40a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T18:14:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.499+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8252+0+0 (secure 0 0 0) 0x7f67d0014070 con 0x7f67dc1027a0 2024-01-11T18:14:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.500+0000 7f67daffd700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f67c40a1ee0 0x7f67c40a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T18:14:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.500+0000 7f67daffd700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f67c40a1ee0 0x7f67c40a43a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f67dc102600 tx=0x7f67cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T18:14:21.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.503+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181097 (secure 0 0 0) 0x7f67d0052380 con 0x7f67dc1027a0 2024-01-11T18:14:21.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:21 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/1484628909' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-11T18:14:21.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:21 smithi038 ceph-mon[43582]: osdmap e91: 12 total, 12 up, 12 in 2024-01-11T18:14:21.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:21 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/879565593' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:21.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:21 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/1484628909' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-11T18:14:21.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:21 smithi007 ceph-mon[33361]: osdmap e91: 12 total, 12 up, 12 in 2024-01-11T18:14:21.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:21 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/879565593' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:21.717+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67dc10ce20 con 0x7f67dc1027a0 2024-01-11T18:14:22.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.237+0000 7f67e09fd700 1 -- 172.21.15.7:0/4093173820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) v1 ==== 178+0+0 (secure 0 0 0) 0x7f67d009a670 con 0x7f67dc1027a0 2024-01-11T18:14:22.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f67c40a1ee0 msgr2=0x7f67c40a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:22.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f67c40a1ee0 0x7f67c40a43a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f67dc102600 tx=0x7f67cc009260 comp rx=0 tx=0).stop 2024-01-11T18:14:22.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 msgr2=0x7f67dc199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 0x7f67dc199cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9a0 tx=0x7f67d000ecb0 comp rx=0 tx=0).stop 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 shutdown_connections 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:6800/314922909,v1:172.21.15.7:6801/314922909] conn(0x7f67c40a1ee0 0x7f67c40a43a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67dc103140 0x7f67dc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc1015b0 0x7f67dc199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 --2- 172.21.15.7:0/4093173820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67dc1027a0 0x7f67dc199cd0 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9a0 tx=0x7f67d000ecb0 comp rx=0 tx=0).stop 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.242+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 >> 172.21.15.7:0/4093173820 conn(0x7f67dc0fcb00 msgr2=0x7f67dc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T18:14:22.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.243+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 shutdown_connections 2024-01-11T18:14:22.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T18:14:22.243+0000 7f67e19ff700 1 -- 172.21.15.7:0/4093173820 wait complete. 2024-01-11T18:14:22.236 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2024-01-11T18:14:22.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-11T18:14:22.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-11T18:14:22.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-11T18:14:22.469 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-01-11T18:14:22.470 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-11T18:14:22.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:22 smithi016 ceph-mon[43195]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T18:14:22.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:22 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/879565593' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-01-11T18:14:22.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:22 smithi016 ceph-mon[43195]: osdmap e92: 12 total, 12 up, 12 in 2024-01-11T18:14:22.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:22 smithi016 ceph-mon[43195]: from='client.? 172.21.15.7:0/4093173820' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:22.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:22 smithi038 ceph-mon[43582]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T18:14:22.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:22 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/879565593' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-01-11T18:14:22.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:22 smithi038 ceph-mon[43582]: osdmap e92: 12 total, 12 up, 12 in 2024-01-11T18:14:22.515 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:22 smithi038 ceph-mon[43582]: from='client.? 172.21.15.7:0/4093173820' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:22.534 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-11T18:14:22.556 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-11T18:14:22.559 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:22 smithi007 ceph-mon[33361]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-01-11T18:14:22.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:22 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/879565593' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-01-11T18:14:22.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:22 smithi007 ceph-mon[33361]: osdmap e92: 12 total, 12 up, 12 in 2024-01-11T18:14:22.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:22 smithi007 ceph-mon[33361]: from='client.? 172.21.15.7:0/4093173820' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-11T18:14:22.579 INFO:tasks.cephadm:Stopping all daemons... 2024-01-11T18:14:22.579 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-01-11T18:14:22.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a 2024-01-11T18:14:23.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:22 smithi007 systemd[1]: Stopping Ceph mon.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:23.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T18:14:23.264+0000 7fec6e261700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:23.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a[33338]: 2024-01-11T18:14:23.264+0000 7fec6e261700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-01-11T18:14:23.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jan 11 18:14:23 smithi007 bash[107084]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-a 2024-01-11T18:14:24.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.a.service' 2024-01-11T18:14:24.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:24.188 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-01-11T18:14:24.189 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-01-11T18:14:24.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.b 2024-01-11T18:14:24.485 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:24 smithi016 systemd[1]: Stopping Ceph mon.b for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:24.985 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-b[43171]: 2024-01-11T18:14:24.585+0000 7ff42a89a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:24.985 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-b[43171]: 2024-01-11T18:14:24.585+0000 7ff42a89a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-01-11T18:14:24.985 INFO:journalctl@ceph.mon.b.smithi016.stdout:Jan 11 18:14:24 smithi016 bash[85889]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-b 2024-01-11T18:14:25.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.b.service' 2024-01-11T18:14:25.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:25.271 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-01-11T18:14:25.271 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-01-11T18:14:25.272 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.c 2024-01-11T18:14:25.640 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:25 smithi038 systemd[1]: Stopping Ceph mon.c for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:25.921 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-c[43559]: 2024-01-11T18:14:25.651+0000 7fa5be2c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:25.921 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-c[43559]: 2024-01-11T18:14:25.651+0000 7fa5be2c9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-01-11T18:14:26.263 INFO:journalctl@ceph.mon.c.smithi038.stdout:Jan 11 18:14:25 smithi038 bash[86267]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mon-c 2024-01-11T18:14:26.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mon.c.service' 2024-01-11T18:14:26.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:26.328 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-01-11T18:14:26.329 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-01-11T18:14:26.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x 2024-01-11T18:14:26.560 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 18:14:26 smithi007 systemd[1]: Stopping Ceph mgr.x for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:27.030 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 18:14:26 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T18:14:26.714+0000 7fdc6d105700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:27.030 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 18:14:26 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x[33612]: 2024-01-11T18:14:26.714+0000 7fdc6d105700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-01-11T18:14:27.310 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jan 11 18:14:27 smithi007 bash[107216]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-x 2024-01-11T18:14:27.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.x.service' 2024-01-11T18:14:27.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:27.581 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-01-11T18:14:27.581 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-01-11T18:14:27.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.y 2024-01-11T18:14:27.949 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 18:14:27 smithi016 systemd[1]: Stopping Ceph mgr.y for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:28.234 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 18:14:28 smithi016 bash[86037]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-y 2024-01-11T18:14:28.235 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Jan 11 18:14:28 smithi016 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-01-11T18:14:28.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.y.service' 2024-01-11T18:14:28.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:28.336 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-01-11T18:14:28.336 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-01-11T18:14:28.336 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.z 2024-01-11T18:14:28.716 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 18:14:28 smithi038 systemd[1]: Stopping Ceph mgr.z for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:28.975 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 18:14:28 smithi038 bash[86417]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mgr-z 2024-01-11T18:14:28.976 INFO:journalctl@ceph.mgr.z.smithi038.stdout:Jan 11 18:14:28 smithi038 systemd[1]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-01-11T18:14:29.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mgr.z.service' 2024-01-11T18:14:29.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:29.276 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-01-11T18:14:29.276 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-01-11T18:14:29.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.0 2024-01-11T18:14:29.653 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 18:14:29 smithi007 systemd[1]: Stopping Ceph osd.0 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:30.060 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 18:14:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0[48084]: 2024-01-11T18:14:29.663+0000 7f7d8af97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:30.060 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 18:14:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0[48084]: 2024-01-11T18:14:29.663+0000 7f7d8af97700 -1 osd.0 93 *** Got signal Terminated *** 2024-01-11T18:14:30.060 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 18:14:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0[48084]: 2024-01-11T18:14:29.663+0000 7f7d8af97700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:14:36.060 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jan 11 18:14:35 smithi007 bash[107357]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-0 2024-01-11T18:14:37.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.0.service' 2024-01-11T18:14:37.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:37.827 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-01-11T18:14:37.827 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-01-11T18:14:37.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.1 2024-01-11T18:14:38.235 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 18:14:37 smithi016 systemd[1]: Stopping Ceph osd.1 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:38.235 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 18:14:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1[48283]: 2024-01-11T18:14:38.087+0000 7f95b0444700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:38.235 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 18:14:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1[48283]: 2024-01-11T18:14:38.087+0000 7f95b0444700 -1 osd.1 93 *** Got signal Terminated *** 2024-01-11T18:14:38.235 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 18:14:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1[48283]: 2024-01-11T18:14:38.087+0000 7f95b0444700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:14:44.234 INFO:journalctl@ceph.osd.1.smithi016.stdout:Jan 11 18:14:43 smithi016 bash[86181]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-1 2024-01-11T18:14:45.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.1.service' 2024-01-11T18:14:45.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:45.188 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-01-11T18:14:45.188 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-01-11T18:14:45.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.2 2024-01-11T18:14:45.515 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 18:14:45 smithi038 systemd[1]: Stopping Ceph osd.2 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:45.515 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 18:14:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2[49334]: 2024-01-11T18:14:45.452+0000 7f5409b7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:45.515 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 18:14:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2[49334]: 2024-01-11T18:14:45.452+0000 7f5409b7c700 -1 osd.2 93 *** Got signal Terminated *** 2024-01-11T18:14:45.515 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 18:14:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2[49334]: 2024-01-11T18:14:45.452+0000 7f5409b7c700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:14:51.515 INFO:journalctl@ceph.osd.2.smithi038.stdout:Jan 11 18:14:51 smithi038 bash[86564]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-2 2024-01-11T18:14:52.641 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.2.service' 2024-01-11T18:14:52.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:14:52.694 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-01-11T18:14:52.694 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-01-11T18:14:52.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.3 2024-01-11T18:14:53.060 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 18:14:52 smithi007 systemd[1]: Stopping Ceph osd.3 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:14:53.560 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 18:14:53 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T18:14:53.092+0000 7fb75faea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:14:53.560 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 18:14:53 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T18:14:53.092+0000 7fb75faea700 -1 osd.3 93 *** Got signal Terminated *** 2024-01-11T18:14:53.560 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 18:14:53 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T18:14:53.092+0000 7fb75faea700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:14:57.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:14:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:14:56.933+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:14:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:14:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:14:56.705+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:14:57.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:14:56 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:14:56.884+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:14:57.559 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 18:14:57 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3[56334]: 2024-01-11T18:14:57.232+0000 7fb7580dd700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.617149+0000 front 2024-01-11T18:14:33.617108+0000 (oldest deadline 2024-01-11T18:14:57.017775+0000) 2024-01-11T18:14:57.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:14:57 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:14:57.215+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:14:57.939 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:14:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:14:57.672+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:14:58.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:14:57 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:14:57.912+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:14:58.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:14:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:14:57.950+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:14:58.265 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:14:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:14:58.000+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:14:58.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:14:58 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:14:58.175+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:14:58.963 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:14:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:14:58.704+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:14:59.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:14:58 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:14:58.900+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:14:59.265 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:14:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:14:59.032+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:14:59.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:14:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:14:58.974+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:14:59.310 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jan 11 18:14:59 smithi007 bash[107620]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-3 2024-01-11T18:14:59.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:14:59 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:14:59.147+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:14:59.981 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:14:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:14:59.700+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:00.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:14:59 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:14:59.899+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:00.264 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:14:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:14:59.992+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:00.265 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:00 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:00.081+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:00.309 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:00 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:00.141+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:00.932 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:00 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:00.655+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:01.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.3.service' 2024-01-11T18:15:01.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:01.145 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-01-11T18:15:01.145 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-01-11T18:15:01.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.4 2024-01-11T18:15:01.176 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:00 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:00.929+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:01.264 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:00 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:00.943+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:01.265 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:01 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:01.073+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:01.309 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:01 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:01.110+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:01.485 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 18:15:01 smithi016 systemd[1]: Stopping Ceph osd.4 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:01.485 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 18:15:01 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4[56313]: 2024-01-11T18:15:01.409+0000 7f79e62a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:01.485 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 18:15:01 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4[56313]: 2024-01-11T18:15:01.409+0000 7f79e62a1700 -1 osd.4 93 *** Got signal Terminated *** 2024-01-11T18:15:01.485 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 18:15:01 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4[56313]: 2024-01-11T18:15:01.409+0000 7f79e62a1700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:01.918 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:01 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:01.647+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:02.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:01 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:01.953+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:02.264 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:02 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:02.025+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:02.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:01 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:01.929+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:02.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:02.099+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:02.886 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:02 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:02.625+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:03.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:02 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:02.905+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:03.264 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:03 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:03.016+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:03.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:02 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:02.897+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:03.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:03 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:03.117+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:03.956 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:03 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:03.864+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:03.956 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:03 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:03.645+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:04.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:03 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:03.865+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:04.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:03 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:03.865+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:04.264 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:03 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:03.967+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:04.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:04 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:04.157+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:05.000 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:04 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:04.624+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:05.000 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:04 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:04.844+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:05.234 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:05 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:05.105+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:05.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:04 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:04.883+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:05.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:04 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:04.883+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:05.264 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:05 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:05.011+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:05.309 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:05 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:05.142+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:05.310 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:05 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:05.056+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:06.004 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:05 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:05.658+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:06.004 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:05 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:05.847+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:06.145 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:05 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:05.880+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:06.145 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:05 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:05.880+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:06.265 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:06 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:06.015+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:06.265 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:06 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:06.015+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:06.310 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:06 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:06.007+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:06.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:06 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:06.138+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:06.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:06 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:06.155+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:07.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:06 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:06.850+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:07.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:06 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:06.701+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:07.156 INFO:journalctl@ceph.osd.4.smithi016.stdout:Jan 11 18:15:07 smithi016 bash[86444]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-4 2024-01-11T18:15:07.157 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:06 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:06.896+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:07.157 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:06 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:06.896+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:07.310 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:06 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:06.961+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:07.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:07 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:07.160+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:07.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:07 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:07.160+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:07.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:07 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:07.166+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:07.514 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:07 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:07.064+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:07.515 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:07 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:07.064+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:08.014 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:07 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:07.874+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:08.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:07 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:07.676+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:08.156 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:07 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:07.902+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:08.156 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:07 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:07.902+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:08.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:07 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:07.941+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:08.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:08 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:08.146+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:08.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:08 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:08.146+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:08.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.4.service' 2024-01-11T18:15:08.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:08.406 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-01-11T18:15:08.406 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-01-11T18:15:08.406 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.5 2024-01-11T18:15:08.439 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:08.048+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:08.439 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:08.048+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:08.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:08.166+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:08.705 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:08 smithi038 systemd[1]: Stopping Ceph osd.5 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:08.705 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:08.683+0000 7fea5e82b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:08.705 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:08.683+0000 7fea5e82b700 -1 osd.5 93 *** Got signal Terminated *** 2024-01-11T18:15:08.705 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:08.683+0000 7fea5e82b700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:09.014 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:08.834+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:09.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:08 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:08.716+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:09.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:08.920+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:09.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:08 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:08.920+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:09.235 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:09.121+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:09.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:08 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:08.954+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:09.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:09 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:09.180+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:09.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:09 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:09.180+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:09.514 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:09 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:09.088+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:09.514 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:09 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:09.088+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:10.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:09 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:09.797+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:10.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:09 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:09.709+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:10.234 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:10 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:10.083+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:10.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:09.920+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:10.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:09 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:09.920+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:10.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:10 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:09.999+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:10.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:10 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:10.218+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:10.310 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:10 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:10.218+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:10.514 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:10 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:10.048+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:10.515 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:10 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:10.048+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:11.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:10 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:10.779+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:11.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:10 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:10.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:11.235 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:11 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:11.041+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:11.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:10 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:10.925+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:11.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:10 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:10.925+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:11.249 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:10 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:10.986+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:11.515 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:11 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:11.076+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:11.515 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:11 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:11.076+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:11.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:11 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:11.259+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:11.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:11 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:11.259+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:12.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:11 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:11.724+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:12.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:11 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:11.823+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:12.235 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:12 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:12.076+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:12.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:11 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:11.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:12.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:11 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:11.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:12.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:11 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:11.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:12.271 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:11 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:11.969+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:12.514 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:12 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:12.041+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:12.515 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:12 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:12.041+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:12.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:12 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:12.282+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:12.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:12 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:12.282+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:12.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:12 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:12.282+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:13.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:12 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:12.848+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:13.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:12 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:12.737+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:13.235 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:13 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:13.071+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:13.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:12 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:12.985+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:13.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:12 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:12.985+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:13.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:12 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:12.985+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:13.259 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:13 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:13.001+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:13.514 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:13 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:13.069+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:34.184914+0000 front 2024-01-11T18:14:34.184994+0000 (oldest deadline 2024-01-11T18:14:57.684781+0000) 2024-01-11T18:15:13.515 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:13 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5[58043]: 2024-01-11T18:15:13.069+0000 7fea56e1e700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.186076+0000 front 2024-01-11T18:14:41.186134+0000 (oldest deadline 2024-01-11T18:15:05.886118+0000) 2024-01-11T18:15:13.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:13 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:13.270+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:13.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:13 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:13.270+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:13.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:13 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:13.270+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:14.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:13 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:13.867+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:14.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:13 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:13.773+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:14.234 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:14 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:14.119+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:14.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:13 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:13.966+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:14.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:13 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:13.966+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:14.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:13 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:13.966+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:14.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:13 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:13.996+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:14.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:13 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:13.996+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:14.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:14 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:14.319+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:14.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:14 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:14.319+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:14.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:14 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:14.319+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:14.730 INFO:journalctl@ceph.osd.5.smithi038.stdout:Jan 11 18:15:14 smithi038 bash[86825]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-5 2024-01-11T18:15:15.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:14 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:14.741+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:15.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:14 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:14.835+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:15.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:14 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:14.835+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:15.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:14 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:14.950+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:15.310 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:14 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:14.950+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:15.484 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:15 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:15.008+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:15.484 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:15 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:15.008+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:15.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:15 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:15.008+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:15.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:15 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:15.097+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:15.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:15 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:15.097+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:15.662 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.5.service' 2024-01-11T18:15:15.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:15.754 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-01-11T18:15:15.755 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-01-11T18:15:15.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.6 2024-01-11T18:15:15.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:15.349+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:15.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:15.349+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:15.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:15.349+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:16.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:15 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:15.750+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:16.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:15 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:15.845+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:16.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:15 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:15.845+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:16.136 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:15 smithi007 systemd[1]: Stopping Ceph osd.6 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:16.137 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:15.952+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:16.137 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:15.952+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:16.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:16 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:16.133+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:16.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:16 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:16.133+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:16.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:16 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:16.051+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:16.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:16 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:16.051+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:16.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:16 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:16.051+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:16.559 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:16.146+0000 7f1bd518d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:16.560 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:16.146+0000 7f1bd518d700 -1 osd.6 93 *** Got signal Terminated *** 2024-01-11T18:15:16.560 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:16.146+0000 7f1bd518d700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:16.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:16.391+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:16.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:16.391+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:16.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:16.391+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:17.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:16 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:16.873+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:17.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:16 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:16.873+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:17.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:16 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:16.781+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:17.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:16 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:16.781+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:17.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:16.957+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:17.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:16 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:16.957+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:17.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:17 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:17.110+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:17.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:17 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:17.110+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:17.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:17 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:17.082+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:17.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:17 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:17.082+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:17.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:17 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:17.082+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:17.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:17.388+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:17.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:17.388+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:17.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:17.388+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:18.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:17 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:17.880+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:18.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:17 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:17.880+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:18.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:17 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:17.831+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:18.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:17 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:17.831+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:18.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:17.987+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:18.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:17 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:17.987+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:18.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:18.150+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:18.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:18.150+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:18.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:18.058+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:18.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:18.058+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:18.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:18 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:18.058+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:18.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:18.421+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:18.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:18.421+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:18.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:18 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:18.421+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:19.264 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:18.887+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:19.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:18.887+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:19.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:18.872+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:19.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:18 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:18.872+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:19.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:19.034+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:19.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:19.034+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:19.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:19 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:19.192+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:19.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:19 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:19.192+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:19.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:19 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:19.044+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:19.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:19 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:19.044+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:19.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:19 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:19.044+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:19.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:19.419+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:19.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:19.419+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:19.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:19.419+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:20.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:19 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:19.852+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:20.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:19 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:19.852+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:20.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:19 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:19.898+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:20.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:19 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:19.898+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:20.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:19 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:19.898+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:20.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:19.992+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:20.310 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:19 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:19.992+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:20.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:20.166+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:20.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:20.166+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:20.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:20.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:20.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:20.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:20.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:20.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:20.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:20.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:20.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:20 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:20.444+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:20.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:20 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:20.444+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:20.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:20 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:20.444+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:21.264 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:20 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:20.814+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:21.265 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:20 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:20.814+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:21.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:20 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:20.892+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:21.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:20 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:20.892+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:21.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:20 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:20.892+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:21.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:21.007+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.515576+0000 front 2024-01-11T18:14:42.515626+0000 (oldest deadline 2024-01-11T18:15:04.215558+0000) 2024-01-11T18:15:21.309 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6[64882]: 2024-01-11T18:15:21.007+0000 7f1bcd780700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.820184+0000 front 2024-01-11T18:14:49.820215+0000 (oldest deadline 2024-01-11T18:15:13.221515+0000) 2024-01-11T18:15:21.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:21.207+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:21.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:21.207+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:21.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:21.207+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:21.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:21.020+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:21.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:21.020+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:21.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:21.020+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:21.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:21 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:21.020+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:21.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:21.446+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:21.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:21.446+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:21.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:21.446+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:21.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:21 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:21.446+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:22.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:21 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:21.769+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:22.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:21 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:21.769+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:22.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:21 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:21.915+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:22.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:21 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:21.915+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:22.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:21 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:21.915+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:22.310 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jan 11 18:15:22 smithi007 bash[107886]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-6 2024-01-11T18:15:22.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:22.234+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:22.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:22.234+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:22.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:22.234+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:22.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:22.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:22.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:22.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.006+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:22.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:22.408+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:22.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:22.408+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:22.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:22.408+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:22.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:22 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:22.408+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:23.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:22 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:22.736+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:23.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:22 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:22.736+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:23.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:22 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:22.908+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:23.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:22 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:22.908+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:23.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:22 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:22.908+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:23.207 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.958+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:23.208 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.958+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:23.208 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.958+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:23.208 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:22 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:22.958+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:23.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:23.218+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:23.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:23.218+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:23.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:23.218+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:23.609 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:23.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:23.609 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:23.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:23.609 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:23.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:23.609 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:23 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:23.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:23.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.6.service' 2024-01-11T18:15:23.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:23.842 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-01-11T18:15:23.842 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-01-11T18:15:23.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.7 2024-01-11T18:15:24.014 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:23.690+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:24.014 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:23.690+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:24.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:23.690+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:24.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:23.913+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:24.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:23.913+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:24.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:23 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:23.913+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:24.213 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:23 smithi016 systemd[1]: Stopping Ceph osd.7 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:24.114+0000 7fb59fc13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:24.114+0000 7fb59fc13700 -1 osd.7 93 *** Got signal Terminated *** 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:24.114+0000 7fb59fc13700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:24.224+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:24.224+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:23.916+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:23.916+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:23.916+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:24.214 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:23.916+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:24.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:24.224+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:24.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:24.363+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:24.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:24.363+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:24.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:24.363+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:24.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:24 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:24.363+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:24.949 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:24.663+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:24.950 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:24.663+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:24.950 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:24.663+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:25.201 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:24.935+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:25.201 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:24.935+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:25.201 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:24.935+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:25.201 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:24 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:24.935+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:25.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:24.960+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:25.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:24.960+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:25.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:24 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:24.960+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:25.484 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:25.211+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:25.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:25.211+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:25.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:25.211+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:25.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:25 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:25.358+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:25.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:25 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:25.358+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:25.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:25 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:25.358+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:25.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:25 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:25.358+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:25.960 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:25.629+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:25.960 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:25.629+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:25.960 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:25.629+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:26.226 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:25.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:26.226 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:25.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:26.226 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:25.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:26.226 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:25 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:25.971+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:26.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:25.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:26.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:25.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:26.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:25 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:25.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:26.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:26.236+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:26.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:26.236+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:26.485 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:26.236+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:26.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:26 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:26.395+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:26.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:26 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:26.395+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:26.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:26 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:26.395+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:26.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:26 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:26.395+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:26.973 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:26.658+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:26.974 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:26.658+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:26.974 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:26.658+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:27.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:26.923+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:27.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:26.923+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:27.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:26.923+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:27.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:26 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:26.923+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:27.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:26.984+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:27.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:26.984+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:27.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:26 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:26.984+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:27.734 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:27.268+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:27.734 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:27.268+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:27.734 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:27.268+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:27.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:27 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:27.351+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:27.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:27 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:27.351+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:27.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:27 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:27.351+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:27.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:27 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:27.351+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:28.004 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:27.662+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:28.004 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:27.662+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:28.004 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:27.662+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:28.004 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:27 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:27.662+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:28.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:27.892+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:28.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:27.892+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:28.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:27.892+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:28.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:27.892+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:28.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.014+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:28.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.014+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:28.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.014+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:28.734 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:28.315+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:41.445446+0000 front 2024-01-11T18:14:41.445621+0000 (oldest deadline 2024-01-11T18:15:04.345574+0000) 2024-01-11T18:15:28.734 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:28.315+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:50.147147+0000 front 2024-01-11T18:14:50.147018+0000 (oldest deadline 2024-01-11T18:15:14.847156+0000) 2024-01-11T18:15:28.735 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:28.315+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.948250+0000 front 2024-01-11T18:14:55.948274+0000 (oldest deadline 2024-01-11T18:15:20.648367+0000) 2024-01-11T18:15:28.735 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7[64304]: 2024-01-11T18:15:28.315+0000 7fb59c20e700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.651905+0000 front 2024-01-11T18:15:05.651869+0000 (oldest deadline 2024-01-11T18:15:27.352083+0000) 2024-01-11T18:15:28.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:28 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:28.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:28.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:28 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:28.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:28.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:28 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:28.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:28.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:28 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:28.368+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:28.969 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:28.640+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:28.969 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:28.640+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:28.969 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:28.640+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:28.969 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:28.640+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:29.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:28.869+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:29.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:28.869+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:29.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:28.869+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:29.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:28 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:28.869+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:29.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.979+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:29.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.979+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:29.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.979+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:29.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:28 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:28.979+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:29.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:29.380+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:29.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:29.380+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:29.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:29.380+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:29.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:29 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:29.380+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:30.005 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:29 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:29.642+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:30.005 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:29 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:29.642+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:30.005 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:29 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:29.642+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:30.005 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:29 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:29.642+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:30.234 INFO:journalctl@ceph.osd.7.smithi016.stdout:Jan 11 18:15:29 smithi016 bash[86700]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-7 2024-01-11T18:15:30.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:29.863+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:30.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:29.863+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:30.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:29.863+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:30.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:29.863+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:30.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:29 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:29.863+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:30.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:30.016+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:30.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:30.016+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:30.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:30.016+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:30.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:30.016+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:30.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:30 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:30.348+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:30.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:30 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:30.348+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:30.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:30 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:30.348+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:30.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:30 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:30.348+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:30.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:30 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:30.348+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:31.014 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:30.656+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:31.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:30.656+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:31.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:30.656+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:31.015 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:30 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:30.656+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:31.199 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:30.867+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:31.199 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:30.867+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:31.199 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:30.867+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:31.200 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:30.867+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:31.200 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:30 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:30.867+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:31.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.7.service' 2024-01-11T18:15:31.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:31.260 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-01-11T18:15:31.260 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-01-11T18:15:31.260 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.8 2024-01-11T18:15:31.302 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:31.057+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:31.302 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:31.057+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:31.302 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:31.057+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:31.302 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:31.057+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:31.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:31.301+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:31.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:31.301+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:31.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:31.301+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:31.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:31.301+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:31.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:31 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:31.301+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:31.670 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 systemd[1]: Stopping Ceph osd.8 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:31.671 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.521+0000 7f59415bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:31.671 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.521+0000 7f59415bf700 -1 osd.8 93 *** Got signal Terminated *** 2024-01-11T18:15:31.671 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.521+0000 7f59415bf700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:32.014 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.681+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:32.167 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.681+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:32.168 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.681+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:32.168 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:31 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:31.681+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:32.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:31.893+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:32.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:31.893+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:32.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:31.893+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:32.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:31.893+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:32.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:31.893+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:32.515 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.028+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:32.515 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.028+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:32.515 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.028+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:32.515 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.028+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:32.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:32.288+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:32.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:32.288+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:32.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:32.288+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:32.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:32.288+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:32.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:32 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:32.288+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:32.979 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:32.729+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:32.979 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:32.729+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:32.979 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:32.729+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:32.979 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:32.729+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:33.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:32.864+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:33.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:32.864+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:33.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:32.864+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:33.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:32.864+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:33.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:32 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:32.864+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:33.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.989+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:33.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.989+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:33.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.989+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:33.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:32 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:32.989+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:33.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:33.256+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:33.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:33.256+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:33.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:33.256+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:33.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:33.256+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:33.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:33 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:33.256+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:33.997 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:33 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:33.724+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:33.998 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:33 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:33.724+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:33.998 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:33 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:33.724+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:33.998 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:33 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:33.724+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:34.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:33 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:33.842+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:34.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:33 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:33.842+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:34.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:33 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:33.842+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:34.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:33 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:33.842+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:34.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:33 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:33.842+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:34.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.008+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:34.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.008+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:34.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.008+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:34.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.008+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:34.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:34.286+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:34.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:34.286+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:34.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:34.286+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:34.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:34.286+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:34.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:34 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:34.286+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:34.958 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:34.693+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:34.958 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:34.693+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:34.958 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:34.693+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:34.958 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:34.693+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:35.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:34 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:34.811+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:35.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:34 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:34.811+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:35.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:34 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:34.811+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:35.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:34 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:34.811+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:35.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:34 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:34.811+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:35.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.968+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:35.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.968+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:35.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.968+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:35.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:34 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:34.968+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:35.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:35.278+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:35.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:35.278+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:35.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:35.278+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:35.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:35.278+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:35.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:35 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:35.278+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:35.926 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:35.682+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.729949+0000 front 2024-01-11T18:14:33.729987+0000 (oldest deadline 2024-01-11T18:14:56.629890+0000) 2024-01-11T18:15:35.926 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:35.682+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.333425+0000 front 2024-01-11T18:14:49.333398+0000 (oldest deadline 2024-01-11T18:15:14.033572+0000) 2024-01-11T18:15:35.926 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:35.682+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:54.034089+0000 front 2024-01-11T18:14:54.034226+0000 (oldest deadline 2024-01-11T18:15:23.434736+0000) 2024-01-11T18:15:35.926 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:35.682+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:06.236805+0000 front 2024-01-11T18:15:06.236853+0000 (oldest deadline 2024-01-11T18:15:27.237220+0000) 2024-01-11T18:15:35.926 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8[66023]: 2024-01-11T18:15:35.682+0000 7f593dbba700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:13.038494+0000 front 2024-01-11T18:15:13.038529+0000 (oldest deadline 2024-01-11T18:15:35.338748+0000) 2024-01-11T18:15:36.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:35 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:35.812+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:36.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:35 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:35.812+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:36.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:35 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:35.812+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:36.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:35 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:35.812+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:36.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:35 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:35.812+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:36.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:35.936+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:36.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:35.936+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:36.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:35.936+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:36.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:35 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:35.936+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:36.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:36.304+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:36.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:36.304+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:36.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:36.304+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:36.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:36.304+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:36.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:36.304+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:36.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:36 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:36.304+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:37.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:36 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:36.810+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:37.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:36 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:36.810+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:37.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:36 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:36.810+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:37.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:36 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:36.810+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:37.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:36 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:36.810+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:37.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:36 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:36.810+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:37.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:36 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:36.937+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:37.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:36 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:36.937+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:37.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:36 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:36.937+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:37.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:36 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:36.937+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:37.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:37.317+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:37.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:37.317+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:37.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:37.317+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:37.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:37.317+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:37.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:37.317+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:37.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:37 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:37.317+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:37.764 INFO:journalctl@ceph.osd.8.smithi038.stdout:Jan 11 18:15:37 smithi038 bash[87079]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-8 2024-01-11T18:15:38.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:37 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:37.847+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:38.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:37 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:37.847+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:38.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:37 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:37.847+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:38.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:37 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:37.847+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:38.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:37 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:37.847+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:38.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:37 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:37.847+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:38.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:37 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:37.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:38.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:37 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:37.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:38.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:37 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:37.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:38.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:37 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:37.971+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:38.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.8.service' 2024-01-11T18:15:38.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:38.327+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:38.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:38.327+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:38.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:38.327+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:38.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:38.327+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:38.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:38.327+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:38.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:38.327+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:38.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:38.866 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-01-11T18:15:38.866 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-01-11T18:15:38.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.9 2024-01-11T18:15:39.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:38.817+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:39.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:38.817+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:39.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:38.817+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:39.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:38.817+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:39.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:38.817+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:39.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:38 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:38.817+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:39.257 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:38 smithi007 systemd[1]: Stopping Ceph osd.9 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:39.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:38 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:38.931+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:39.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:38 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:38.931+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:39.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:38 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:38.931+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:39.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:38 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:38.931+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:39.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:38 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:38.931+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:39.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.268+0000 7f643a98f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:39.559 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.268+0000 7f643a98f700 -1 osd.9 93 *** Got signal Terminated *** 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.268+0000 7f643a98f700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.336+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.336+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.336+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.336+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.336+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:39.560 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:39 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:39.336+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:40.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:39 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:39.861+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:40.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:39 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:39.861+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:40.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:39 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:39.861+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:40.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:39 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:39.861+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:40.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:39 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:39.861+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:40.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:39 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:39.861+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:40.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:39.884+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:40.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:39.884+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:40.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:39.884+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:40.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:39.884+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:40.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:39.884+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:40.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:40.353+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:40.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:40.353+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:40.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:40.353+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:40.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:40.353+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:40.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:40.353+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:40.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:40 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:40.353+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:41.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:40.834+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:41.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:40.834+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:41.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:40.834+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:41.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:40.834+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:41.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:40.834+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:41.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:40 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:40.834+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:41.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:40 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:40.855+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:41.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:40 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:40.855+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:41.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:40 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:40.855+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:41.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:40 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:40.855+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:41.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:40 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:40.855+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:41.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:41 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:41.367+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:41.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:41 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:41.367+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:41.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:41 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:41.367+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:41.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:41 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:41.367+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:41.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:41 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:41.367+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:41.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:41 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:41.367+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:42.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:41 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:41.796+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:42.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:41 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:41.796+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:42.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:41 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:41.796+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:42.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:41 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:41.796+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:42.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:41 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:41.796+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:42.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:41 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:41.796+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:42.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:41 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:41.836+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:42.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:41 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:41.836+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:42.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:41 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:41.836+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:42.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:41 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:41.836+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:42.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:41 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:41.836+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:42.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:42.375+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:42.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:42.375+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:42.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:42.375+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:42.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:42.375+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:42.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:42.375+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:42.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:42 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:42.375+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:43.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:43.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:43.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:43.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:43.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:43.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:43.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:42 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:42.843+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:43.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:42.847+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:43.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:42.847+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:43.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:42.847+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:43.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:42.847+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:43.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:42 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:42.847+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:43.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:43.425+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:33.059214+0000 front 2024-01-11T18:14:33.059243+0000 (oldest deadline 2024-01-11T18:14:57.159338+0000) 2024-01-11T18:15:43.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:43.425+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:42.261549+0000 front 2024-01-11T18:14:42.261623+0000 (oldest deadline 2024-01-11T18:15:06.361745+0000) 2024-01-11T18:15:43.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:43.425+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.263446+0000 front 2024-01-11T18:14:49.263411+0000 (oldest deadline 2024-01-11T18:15:11.563684+0000) 2024-01-11T18:15:43.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:43.425+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:56.265215+0000 front 2024-01-11T18:14:56.265245+0000 (oldest deadline 2024-01-11T18:15:20.965496+0000) 2024-01-11T18:15:43.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:43.425+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:04.967872+0000 front 2024-01-11T18:15:04.967910+0000 (oldest deadline 2024-01-11T18:15:30.267936+0000) 2024-01-11T18:15:43.810 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:43 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9[72787]: 2024-01-11T18:15:43.425+0000 7f6436f8a700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.268720+0000 front 2024-01-11T18:15:10.268606+0000 (oldest deadline 2024-01-11T18:15:35.568627+0000) 2024-01-11T18:15:44.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:44.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:44.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:44.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:44.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:44.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:44.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:43 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:43.840+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:44.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:43.827+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:44.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:43.827+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:44.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:43.827+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:44.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:43.827+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:44.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:43.827+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:45.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:44 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:44.794+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:45.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:44 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:44.793+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:45.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:44 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:44.793+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:45.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:44 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:44.793+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:45.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:44 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:44.793+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:45.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:44 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:44.793+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:45.809 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jan 11 18:15:45 smithi007 bash[108146]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-9 2024-01-11T18:15:46.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:46.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:46.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:46.234 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:46.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:46.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:46.235 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:45 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:45.752+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:46.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:45.812+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:46.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:45.812+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:46.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:45.812+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:46.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:45.812+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:46.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:45 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:45.812+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:46.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:46.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:46.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:46.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:46.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:46.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:46.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:46 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:46.716+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:46.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.9.service' 2024-01-11T18:15:47.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:47.083 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-01-11T18:15:47.083 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-01-11T18:15:47.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.10 2024-01-11T18:15:47.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:46 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:46.842+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:47.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:46 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:46.842+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:47.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:46 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:46.842+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:47.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:46 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:46.842+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:47.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:46 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:46.842+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:47.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:46 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:46.842+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:47.484 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 systemd[1]: Stopping Ceph osd.10 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:47.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.344+0000 7f62f71f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:47.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.344+0000 7f62f71f6700 -1 osd.10 93 *** Got signal Terminated *** 2024-01-11T18:15:47.485 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.344+0000 7f62f71f6700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:47.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:47.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:47.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:47.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:47.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:47.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:47.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:47 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:47.719+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:48.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:47.868+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:48.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:47.868+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:48.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:47.868+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:48.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:47.868+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:48.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:47.868+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:48.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:47.868+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:48.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:48.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:48.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:48.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:48.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:48.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:48.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:48 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:48.729+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:49.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:48 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:48.859+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:49.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:48 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:48.859+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:49.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:48 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:48.859+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:49.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:48 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:48.859+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:49.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:48 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:48.859+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:49.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:48 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:48.859+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:49.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:49.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:49.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:49.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:49.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:49.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:49.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:49 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:49.688+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:50.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:49.849+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:50.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:49.849+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:50.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:49.849+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:50.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:49.849+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:50.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:49.849+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:50.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:49 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:49.849+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:50.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:50.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:50.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:50.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:50.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:50.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:50.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:50 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:50.653+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:51.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:51.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:51.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:51.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:51.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:51.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:51.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:50 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:50.871+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:51.984 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:30.788852+0000 front 2024-01-11T18:14:30.788834+0000 (oldest deadline 2024-01-11T18:14:56.688779+0000) 2024-01-11T18:15:51.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-01-11T18:14:40.189952+0000 front 2024-01-11T18:14:40.189981+0000 (oldest deadline 2024-01-11T18:15:03.690178+0000) 2024-01-11T18:15:51.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:46.492018+0000 front 2024-01-11T18:14:46.492192+0000 (oldest deadline 2024-01-11T18:15:11.092704+0000) 2024-01-11T18:15:51.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:52.793998+0000 front 2024-01-11T18:14:52.793924+0000 (oldest deadline 2024-01-11T18:15:19.794396+0000) 2024-01-11T18:15:51.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:05.095244+0000 front 2024-01-11T18:15:05.095289+0000 (oldest deadline 2024-01-11T18:15:29.195416+0000) 2024-01-11T18:15:51.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:11.496556+0000 front 2024-01-11T18:15:11.496612+0000 (oldest deadline 2024-01-11T18:15:36.796622+0000) 2024-01-11T18:15:51.985 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:51 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10[73072]: 2024-01-11T18:15:51.613+0000 7f62ef7e9700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:19.097782+0000 front 2024-01-11T18:15:19.097863+0000 (oldest deadline 2024-01-11T18:15:42.597694+0000) 2024-01-11T18:15:52.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:52.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:52.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:52.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:52.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:52.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:52.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:51 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:51.837+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:53.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:53.264 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:53.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:53.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:53.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:53.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:53.265 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:52 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:52.795+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:53.484 INFO:journalctl@ceph.osd.10.smithi016.stdout:Jan 11 18:15:53 smithi016 bash[86962]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-10 2024-01-11T18:15:54.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:54.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:54.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:54.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:54.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:54.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:54.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:53 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:53.747+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:54.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.10.service' 2024-01-11T18:15:54.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:15:54.652 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-01-11T18:15:54.652 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-01-11T18:15:54.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.11 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 systemd[1]: Stopping Ceph osd.11 for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.707+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.708+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.708+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.708+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.708+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.708+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.708+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.927+0000 7fe4e7623700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.927+0000 7fe4e7623700 -1 osd.11 93 *** Got signal Terminated *** 2024-01-11T18:15:55.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:54 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:54.927+0000 7fe4e7623700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-11T18:15:56.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:56.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:56.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:56.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:56.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:56.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:56.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:55 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:55.743+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:57.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:57.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:56 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:56.729+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:58.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:58.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:58.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:58.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:58.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:58.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:58.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:57 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:57.711+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:15:59.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:15:59.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:15:59.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:15:59.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:15:59.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:15:59.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:15:59.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:58 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:58.661+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:16:00.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.651+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-01-11T18:14:31.386838+0000 front 2024-01-11T18:14:31.386858+0000 (oldest deadline 2024-01-11T18:14:56.086648+0000) 2024-01-11T18:16:00.014 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.651+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6804 osd.2 since back 2024-01-11T18:14:49.890895+0000 front 2024-01-11T18:14:49.890983+0000 (oldest deadline 2024-01-11T18:15:15.791098+0000) 2024-01-11T18:16:00.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.651+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-01-11T18:14:55.792011+0000 front 2024-01-11T18:14:55.791901+0000 (oldest deadline 2024-01-11T18:15:19.291823+0000) 2024-01-11T18:16:00.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.651+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-01-11T18:15:03.393338+0000 front 2024-01-11T18:15:03.393224+0000 (oldest deadline 2024-01-11T18:15:28.693351+0000) 2024-01-11T18:16:00.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.651+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6812 osd.5 since back 2024-01-11T18:15:10.394208+0000 front 2024-01-11T18:15:10.394235+0000 (oldest deadline 2024-01-11T18:15:37.995130+0000) 2024-01-11T18:16:00.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.652+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-01-11T18:15:20.896464+0000 front 2024-01-11T18:15:20.896569+0000 (oldest deadline 2024-01-11T18:15:46.196361+0000) 2024-01-11T18:16:00.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.652+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-01-11T18:15:26.196857+0000 front 2024-01-11T18:15:26.196919+0000 (oldest deadline 2024-01-11T18:15:50.197302+0000) 2024-01-11T18:16:00.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:15:59 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11[73988]: 2024-01-11T18:15:59.652+0000 7fe4dfc16700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6820 osd.8 since back 2024-01-11T18:15:35.398653+0000 front 2024-01-11T18:15:35.398693+0000 (oldest deadline 2024-01-11T18:15:58.898755+0000) 2024-01-11T18:16:01.015 INFO:journalctl@ceph.osd.11.smithi038.stdout:Jan 11 18:16:00 smithi038 bash[87341]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-osd-11 2024-01-11T18:16:01.988 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@osd.11.service' 2024-01-11T18:16:02.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:02.041 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-01-11T18:16:02.041 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-01-11T18:16:02.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.a 2024-01-11T18:16:02.438 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:16:02 smithi007 systemd[1]: Stopping Ceph mds.a for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:02.809 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:16:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T18:16:02.447+0000 7f76c337b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:02.809 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:16:02 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a[81148]: 2024-01-11T18:16:02.447+0000 7f76c337b700 -1 mds.a *** got signal Terminated *** 2024-01-11T18:16:06.809 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jan 11 18:16:06 smithi007 bash[108409]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-a 2024-01-11T18:16:06.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.a.service' 2024-01-11T18:16:07.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:07.049 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-01-11T18:16:07.049 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-01-11T18:16:07.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.d 2024-01-11T18:16:07.309 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 18:16:07 smithi007 systemd[1]: Stopping Ceph mds.d for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:07.809 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 18:16:07 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-d[80185]: 2024-01-11T18:16:07.421+0000 7fdd077d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:07.809 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 18:16:07 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-d[80185]: 2024-01-11T18:16:07.421+0000 7fdd077d6700 -1 mds.d *** got signal Terminated *** 2024-01-11T18:16:10.555 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jan 11 18:16:10 smithi007 bash[108550]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-d 2024-01-11T18:16:10.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.d.service' 2024-01-11T18:16:10.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:10.671 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-01-11T18:16:10.671 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-01-11T18:16:10.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.g 2024-01-11T18:16:11.035 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 18:16:10 smithi007 systemd[1]: Stopping Ceph mds.g for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:11.309 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 18:16:11 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-g[80760]: 2024-01-11T18:16:11.045+0000 7f938ee89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:11.309 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 18:16:11 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-g[80760]: 2024-01-11T18:16:11.045+0000 7f938ee89700 -1 mds.g *** got signal Terminated *** 2024-01-11T18:16:15.059 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jan 11 18:16:14 smithi007 bash[108688]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-g 2024-01-11T18:16:15.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.g.service' 2024-01-11T18:16:15.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:15.285 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-01-11T18:16:15.285 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-01-11T18:16:15.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.j 2024-01-11T18:16:15.559 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 18:16:15 smithi007 systemd[1]: Stopping Ceph mds.j for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:16.059 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 18:16:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-j[79800]: 2024-01-11T18:16:15.661+0000 7fb80029c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:16.059 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 18:16:15 smithi007 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-j[79800]: 2024-01-11T18:16:15.661+0000 7fb80029c700 -1 mds.j *** got signal Terminated *** 2024-01-11T18:16:19.809 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jan 11 18:16:19 smithi007 bash[108827]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-j 2024-01-11T18:16:19.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.j.service' 2024-01-11T18:16:19.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:19.998 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-01-11T18:16:19.998 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-01-11T18:16:19.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.b 2024-01-11T18:16:20.484 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:16:20 smithi016 systemd[1]: Stopping Ceph mds.b for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:20.484 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:16:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: 2024-01-11T18:16:20.257+0000 7f2ff91fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:20.484 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:16:20 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b[80208]: 2024-01-11T18:16:20.257+0000 7f2ff91fb700 -1 mds.b *** got signal Terminated *** 2024-01-11T18:16:22.734 INFO:journalctl@ceph.mds.b.smithi016.stdout:Jan 11 18:16:22 smithi016 bash[87219]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-b 2024-01-11T18:16:22.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.b.service' 2024-01-11T18:16:22.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:22.783 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-01-11T18:16:22.783 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-01-11T18:16:22.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.e 2024-01-11T18:16:23.061 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:16:22 smithi016 systemd[1]: Stopping Ceph mds.e for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:23.484 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:16:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e[79529]: 2024-01-11T18:16:23.072+0000 7f80d8602700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:23.484 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:16:23 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e[79529]: 2024-01-11T18:16:23.072+0000 7f80d8602700 -1 mds.e *** got signal Terminated *** 2024-01-11T18:16:26.734 INFO:journalctl@ceph.mds.e.smithi016.stdout:Jan 11 18:16:26 smithi016 bash[87357]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-e 2024-01-11T18:16:26.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.e.service' 2024-01-11T18:16:26.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:26.787 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-01-11T18:16:26.787 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-01-11T18:16:26.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h 2024-01-11T18:16:27.061 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:16:26 smithi016 systemd[1]: Stopping Ceph mds.h for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:27.484 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:16:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h[82872]: 2024-01-11T18:16:27.072+0000 7f573c116700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:27.484 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:16:27 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h[82872]: 2024-01-11T18:16:27.072+0000 7f573c116700 -1 mds.h *** got signal Terminated *** 2024-01-11T18:16:31.484 INFO:journalctl@ceph.mds.h.smithi016.stdout:Jan 11 18:16:31 smithi016 bash[87495]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-h 2024-01-11T18:16:31.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.h.service' 2024-01-11T18:16:31.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:31.560 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-01-11T18:16:31.560 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-01-11T18:16:31.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.k 2024-01-11T18:16:31.853 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 18:16:31 smithi016 systemd[1]: Stopping Ceph mds.k for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:32.234 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 18:16:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-k[79179]: 2024-01-11T18:16:31.863+0000 7f6bf7b0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:32.234 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 18:16:31 smithi016 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-k[79179]: 2024-01-11T18:16:31.863+0000 7f6bf7b0a700 -1 mds.k *** got signal Terminated *** 2024-01-11T18:16:36.234 INFO:journalctl@ceph.mds.k.smithi016.stdout:Jan 11 18:16:36 smithi016 bash[87635]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-k 2024-01-11T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.k.service' 2024-01-11T18:16:36.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:36.368 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-01-11T18:16:36.368 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-01-11T18:16:36.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.c 2024-01-11T18:16:36.765 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:16:36 smithi038 systemd[1]: Stopping Ceph mds.c for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:36.765 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:16:36 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c[80150]: 2024-01-11T18:16:36.630+0000 7f5c4fc92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:36.765 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:16:36 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c[80150]: 2024-01-11T18:16:36.630+0000 7f5c4fc92700 -1 mds.c *** got signal Terminated *** 2024-01-11T18:16:38.715 INFO:journalctl@ceph.mds.c.smithi038.stdout:Jan 11 18:16:38 smithi038 bash[87596]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-c 2024-01-11T18:16:38.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.c.service' 2024-01-11T18:16:38.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:38.902 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-01-11T18:16:38.903 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-01-11T18:16:38.903 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.f 2024-01-11T18:16:39.264 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:16:38 smithi038 systemd[1]: Stopping Ceph mds.f for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:39.264 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:16:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f[79809]: 2024-01-11T18:16:39.204+0000 7fd546656700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:39.264 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:16:39 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f[79809]: 2024-01-11T18:16:39.204+0000 7fd546656700 -1 mds.f *** got signal Terminated *** 2024-01-11T18:16:42.764 INFO:journalctl@ceph.mds.f.smithi038.stdout:Jan 11 18:16:42 smithi038 bash[87734]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-f 2024-01-11T18:16:42.785 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.f.service' 2024-01-11T18:16:42.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:42.823 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-01-11T18:16:42.823 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-01-11T18:16:42.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.i 2024-01-11T18:16:43.111 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:16:42 smithi038 systemd[1]: Stopping Ceph mds.i for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:43.514 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:16:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T18:16:43.120+0000 7f76ada93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:43.514 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:16:43 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i[80493]: 2024-01-11T18:16:43.120+0000 7f76ada93700 -1 mds.i *** got signal Terminated *** 2024-01-11T18:16:46.764 INFO:journalctl@ceph.mds.i.smithi038.stdout:Jan 11 18:16:46 smithi038 bash[87873]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-i 2024-01-11T18:16:46.835 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.i.service' 2024-01-11T18:16:46.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:46.878 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-01-11T18:16:46.878 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-01-11T18:16:46.878 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.l 2024-01-11T18:16:47.264 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 18:16:46 smithi038 systemd[1]: Stopping Ceph mds.l for a76d0c8a-b0a9-11ee-95ab-87774f69a715... 2024-01-11T18:16:47.764 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 18:16:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-l[79463]: 2024-01-11T18:16:47.295+0000 7fb5e598e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-11T18:16:47.764 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 18:16:47 smithi038 ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-l[79463]: 2024-01-11T18:16:47.295+0000 7fb5e598e700 -1 mds.l *** got signal Terminated *** 2024-01-11T18:16:51.264 INFO:journalctl@ceph.mds.l.smithi038.stdout:Jan 11 18:16:51 smithi038 bash[88021]: ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715-mds-l 2024-01-11T18:16:51.334 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a76d0c8a-b0a9-11ee-95ab-87774f69a715@mds.l.service' 2024-01-11T18:16:51.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-11T18:16:51.373 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-01-11T18:16:51.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --force --keep-logs 2024-01-11T18:16:51.694 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T18:16:54.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --force --keep-logs 2024-01-11T18:16:55.186 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T18:16:56.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --force --keep-logs 2024-01-11T18:16:57.074 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T18:16:58.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-11T18:16:58.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-11T18:16:58.830 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-11T18:16:58.867 INFO:tasks.cephadm:Archiving crash dumps... 2024-01-11T18:16:58.870 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi007/crash 2024-01-11T18:16:58.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash -- . 2024-01-11T18:16:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash: Cannot open: No such file or directory 2024-01-11T18:16:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-01-11T18:16:58.960 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi016/crash 2024-01-11T18:16:58.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash -- . 2024-01-11T18:16:59.015 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash: Cannot open: No such file or directory 2024-01-11T18:16:59.015 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-01-11T18:16:59.017 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi038/crash 2024-01-11T18:16:59.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash -- . 2024-01-11T18:16:59.070 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/crash: Cannot open: No such file or directory 2024-01-11T18:16:59.070 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-01-11T18:16:59.071 INFO:tasks.cephadm:Checking cluster log for badness... 2024-01-11T18:16:59.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-01-11T18:16:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:grep: /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph.log: No such file or directory 2024-01-11T18:16:59.161 INFO:tasks.cephadm:Compressing logs... 2024-01-11T18:16:59.161 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-11T18:16:59.166 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-11T18:16:59.168 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-11T18:16:59.211 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-01-11T18:16:59.212 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-11T18:16:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-11T18:16:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-01-11T18:16:59.220 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.5.log 2024-01-11T18:16:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.i.log 2024-01-11T18:16:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.c.log 2024-01-11T18:16:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.f.log 2024-01-11T18:16:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.z.log 2024-01-11T18:16:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.11.log 2024-01-11T18:16:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.2.log 2024-01-11T18:16:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log 2024-01-11T18:16:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.l.log 2024-01-11T18:16:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.c.log 2024-01-11T18:16:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.8.log 2024-01-11T18:16:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.7.log 2024-01-11T18:16:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.1.log 2024-01-11T18:16:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.h.log 2024-01-11T18:16:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.7.log: /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log 2024-01-11T18:16:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.k.log 2024-01-11T18:16:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.e.log 2024-01-11T18:16:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.b.log 2024-01-11T18:16:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.b.log 2024-01-11T18:16:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.y.log 2024-01-11T18:16:59.228 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.l.log: /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.c.log: /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.8.log: 92.9% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.z.log.gz 2024-01-11T18:16:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.10.log 2024-01-11T18:16:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.4.log 2024-01-11T18:16:59.229 INFO:teuthology.orchestra.run.smithi038.stderr: 94.3% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.l.log.gz 2024-01-11T18:16:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.10.log: /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.4.log: 94.2% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.k.log.gz 2024-01-11T18:16:59.233 INFO:teuthology.orchestra.run.smithi016.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-11T18:16:59.233 INFO:teuthology.orchestra.run.smithi016.stderr: 92.8% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.y.log.gz 2024-01-11T18:16:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-01-11T18:16:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-11T18:16:59.250 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-01-11T18:16:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.a.log 2024-01-11T18:16:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.g.log 2024-01-11T18:16:59.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.j.log 2024-01-11T18:16:59.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.g.log: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-01-11T18:16:59.254 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.a.log 2024-01-11T18:16:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.9.log 2024-01-11T18:16:59.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.3.log 2024-01-11T18:16:59.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log 2024-01-11T18:16:59.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.3.log: 94.2% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.g.log.gz 2024-01-11T18:16:59.260 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.0.log 2024-01-11T18:16:59.260 INFO:teuthology.orchestra.run.smithi007.stderr: 94.3% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.j.log.gz 2024-01-11T18:16:59.260 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.d.log 2024-01-11T18:16:59.261 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.6.log 2024-01-11T18:16:59.262 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.x.log 2024-01-11T18:16:59.266 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.6.log: /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.x.log: 94.2% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.d.log.gz 2024-01-11T18:16:59.278 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log.gz 2024-01-11T18:16:59.486 INFO:teuthology.orchestra.run.smithi007.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-11T18:16:59.675 INFO:teuthology.orchestra.run.smithi038.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-11T18:16:59.860 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mgr.x.log.gz 2024-01-11T18:17:00.121 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log.gz 2024-01-11T18:17:00.179 INFO:teuthology.orchestra.run.smithi016.stderr: 91.7% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.b.log.gz 2024-01-11T18:17:00.204 INFO:teuthology.orchestra.run.smithi016.stderr: 94.8% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.h.log.gz 2024-01-11T18:17:00.204 INFO:teuthology.orchestra.run.smithi038.stderr: 94.3% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.c.log.gz 2024-01-11T18:17:00.328 INFO:teuthology.orchestra.run.smithi016.stderr: 92.5% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.e.log.gz 2024-01-11T18:17:00.354 INFO:teuthology.orchestra.run.smithi038.stderr: 91.7% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.c.log.gz 2024-01-11T18:17:00.643 INFO:teuthology.orchestra.run.smithi038.stderr: 93.4% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.f.log.gz 2024-01-11T18:17:00.653 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-volume.log.gz 2024-01-11T18:17:01.743 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mon.a.log.gz 2024-01-11T18:17:02.543 INFO:teuthology.orchestra.run.smithi038.stderr: 93.8% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.i.log.gz 2024-01-11T18:17:04.371 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.b.log.gz 2024-01-11T18:17:06.143 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-mds.a.log.gz 2024-01-11T18:18:31.728 INFO:teuthology.orchestra.run.smithi016.stderr: 90.5% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.10.log.gz 2024-01-11T18:18:42.094 INFO:teuthology.orchestra.run.smithi016.stderr: 90.4% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.1.log.gz 2024-01-11T18:18:48.573 INFO:teuthology.orchestra.run.smithi038.stderr: 90.7% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.11.log.gz 2024-01-11T18:18:52.338 INFO:teuthology.orchestra.run.smithi038.stderr: 90.5% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.8.log.gz 2024-01-11T18:18:57.270 INFO:teuthology.orchestra.run.smithi016.stderr: 90.4% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.7.log.gz 2024-01-11T18:19:07.847 INFO:teuthology.orchestra.run.smithi038.stderr: 90.6% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.2.log.gz 2024-01-11T18:19:36.223 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.4.log.gz 2024-01-11T18:19:36.225 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-01-11T18:19:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:real 2m37.039s 2024-01-11T18:19:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:user 5m13.572s 2024-01-11T18:19:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m10.693s 2024-01-11T18:20:19.112 INFO:teuthology.orchestra.run.smithi038.stderr: 90.7% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.5.log.gz 2024-01-11T18:20:19.115 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-01-11T18:20:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:real 3m19.927s 2024-01-11T18:20:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:user 5m46.841s 2024-01-11T18:20:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m12.251s 2024-01-11T18:20:38.467 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.6.log.gz 2024-01-11T18:20:46.657 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.0.log.gz 2024-01-11T18:20:51.304 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.3.log.gz 2024-01-11T18:21:31.444 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/a76d0c8a-b0a9-11ee-95ab-87774f69a715/ceph-osd.9.log.gz 2024-01-11T18:21:31.449 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-01-11T18:21:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:real 4m32.246s 2024-01-11T18:21:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:user 7m6.992s 2024-01-11T18:21:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m47.460s 2024-01-11T18:21:31.450 INFO:tasks.cephadm:Archiving logs... 2024-01-11T18:21:31.451 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi007/log 2024-01-11T18:21:31.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-11T18:22:46.728 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi016/log 2024-01-11T18:22:46.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-11T18:23:41.801 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi038/log 2024-01-11T18:23:41.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-11T18:24:41.211 INFO:tasks.cephadm:Removing cluster... 2024-01-11T18:24:41.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --force 2024-01-11T18:24:41.711 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T18:24:43.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --force 2024-01-11T18:24:44.187 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T18:24:44.976 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a76d0c8a-b0a9-11ee-95ab-87774f69a715 --force 2024-01-11T18:24:45.360 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: a76d0c8a-b0a9-11ee-95ab-87774f69a715 2024-01-11T18:24:46.166 INFO:tasks.cephadm:Removing cephadm ... 2024-01-11T18:24:46.166 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-11T18:24:46.216 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-11T18:24:46.249 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-11T18:24:46.276 INFO:tasks.cephadm:Teardown complete 2024-01-11T18:24:46.277 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-11T18:24:46.287 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-11T18:24:46.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-11T18:24:46.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-11T18:24:46.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-11T18:24:46.448 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-11T18:24:46.459 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-11T18:24:46.459 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T18:24:46.464 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T18:24:46.467 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T18:24:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2024-01-11T18:24:46.488 INFO:teuthology.orchestra.run.smithi038.stderr:bash: ntpq: command not found 2024-01-11T18:24:46.502 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2024-01-11T18:24:46.511 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T18:24:46.512 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-01-11T18:24:46.512 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 -2130us[-2130us] +/- 95ms 2024-01-11T18:24:46.512 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -372us[ -410us] +/- 66ms 2024-01-11T18:24:46.512 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +2388us[+2388us] +/- 80ms 2024-01-11T18:24:46.512 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T18:24:46.530 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T18:24:46.530 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-01-11T18:24:46.530 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 -2133us[-2133us] +/- 95ms 2024-01-11T18:24:46.530 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -362us[ -349us] +/- 66ms 2024-01-11T18:24:46.530 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +2380us[+2393us] +/- 80ms 2024-01-11T18:24:46.530 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T18:24:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T18:24:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-01-11T18:24:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 -2153us[-2153us] +/- 95ms 2024-01-11T18:24:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -371us[ -338us] +/- 66ms 2024-01-11T18:24:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +2368us[+2368us] +/- 80ms 2024-01-11T18:24:46.569 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T18:24:46.570 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-11T18:24:46.580 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-11T18:24:46.581 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-11T18:24:46.617 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-11T18:24:46.813 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-11T18:24:46.921 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-11T18:24:47.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T18:24:47.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:24:47.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T18:24:47.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:24:47.155 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T18:24:47.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:24:47.193 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-11T18:24:47.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-11T18:24:47.242 INFO:teuthology.task.internal:Duration was 2858.695609 seconds 2024-01-11T18:24:47.242 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-11T18:24:47.309 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-11T18:24:47.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-11T18:24:47.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-11T18:24:47.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-11T18:24:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T18:24:47.433 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T18:24:47.484 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T18:24:47.887 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-11T18:24:47.887 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T18:24:47.887 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-11T18:24:47.949 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-01-11T18:24:47.949 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-11T18:24:47.978 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-01-11T18:24:47.978 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-11T18:24:48.009 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-11T18:24:48.009 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-11T18:24:48.014 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-11T18:24:48.021 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-11T18:24:48.139 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-11T18:24:48.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-11T18:24:48.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-11T18:24:48.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-11T18:24:48.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-11T18:24:48.368 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-11T18:24:48.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-11T18:24:48.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-11T18:24:48.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-11T18:24:48.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-11T18:24:48.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-11T18:24:48.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-11T18:24:48.611 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-11T18:24:48.652 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-01-11T18:24:48.663 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-01-11T18:24:48.688 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-01-11T18:24:48.734 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-11T18:24:48.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:24:48.776 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-11T18:24:48.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:24:48.797 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-11T18:24:48.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T18:24:48.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-11T18:24:48.830 INFO:teuthology.task.internal:Transferring archived files... 2024-01-11T18:24:48.831 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi007 2024-01-11T18:24:48.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-11T18:24:48.917 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi016 2024-01-11T18:24:48.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-11T18:24:48.968 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480/remote/smithi038 2024-01-11T18:24:48.968 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-11T18:24:49.018 INFO:teuthology.task.internal:Removing archive directory... 2024-01-11T18:24:49.018 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-11T18:24:49.023 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-11T18:24:49.025 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-11T18:24:49.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-11T18:24:49.085 INFO:teuthology.task.internal:Not uploading archives. 2024-01-11T18:24:49.085 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-11T18:24:49.096 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-11T18:24:49.096 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-11T18:24:49.107 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-11T18:24:49.109 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-11T18:24:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 11 18:24 /home/ubuntu/cephtest 2024-01-11T18:24:49.134 INFO:teuthology.orchestra.run.smithi038.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 11 18:24 /home/ubuntu/cephtest 2024-01-11T18:24:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 11 18:24 /home/ubuntu/cephtest 2024-01-11T18:24:49.150 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-11T18:24:49.163 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-11T18:24:49.212 INFO:teuthology.nuke:Checking targets against current locks 2024-01-11T18:24:49.240 DEBUG:teuthology.nuke:shortname: smithi007 2024-01-11T18:24:49.240 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T18:24:49.263 DEBUG:teuthology.nuke:shortname: smithi016 2024-01-11T18:24:49.263 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T18:24:49.268 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 17:26:24.097693', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUxAXgF7Uz1hRGAu0N2ik0dSn7OJa3/bAmjoGACrbzZl+U+CY4bMsuLpWV//adPrDYu7cZq670RzaMGa6anUYs='} 2024-01-11T18:24:49.283 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 17:26:24.099694', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKW56sAh1/aT9PL4UA9l9+2lC/6OkMwIfx/Fs+Ncf/f+DdUMaC8HzkGNfq7EKlzd6mXt9WSTQecBYwFGnSH1OQw='} 2024-01-11T18:24:49.288 DEBUG:teuthology.nuke:shortname: smithi038 2024-01-11T18:24:49.288 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T18:24:49.314 INFO:teuthology.orchestra.console.smithi007:Power off 2024-01-11T18:24:49.314 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-11T18:24:49.325 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-10_15:00:23-fs-wip-vshankar-testing-20240103.072409-1-testing-default-smithi/7511480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 17:26:24.101444', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKOnrNojzKmJ4frmXVXgx8FtpXvRrJRpgot+brkW8uFuelrNSuPXdazsVnH79I3ILG7iDrmIr9UFqZmBNhyAkNQ='} 2024-01-11T18:24:49.338 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-01-11T18:24:49.338 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:49.360 INFO:teuthology.orchestra.console.smithi016:Power off 2024-01-11T18:24:49.360 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-11T18:24:49.373 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-11T18:24:49.385 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2024-01-11T18:24:49.386 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:49.402 INFO:teuthology.orchestra.console.smithi038:Power off 2024-01-11T18:24:49.402 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-11T18:24:49.411 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-01-11T18:24:49.425 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-01-11T18:24:49.425 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:49.449 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-01-11T18:24:53.374 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:53.413 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:53.451 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:53.483 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-11T18:24:53.523 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-01-11T18:24:53.560 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-01-11T18:24:57.488 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:57.524 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:57.561 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T18:24:57.599 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-11T18:24:57.634 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-01-11T18:24:57.671 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-01-11T18:24:57.701 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-01-11T18:24:57.735 INFO:teuthology.orchestra.console.smithi016:Power off completed 2024-01-11T18:24:57.772 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-01-11T18:24:57.878 INFO:teuthology.run:Summary data: client.0-kernel-sha1: c719ee96ba7847b7d8dd469c35de92fb2f0dfd6b description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 2858.695608854294 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2024-01-11T18:24:57.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-11T18:24:57.958 INFO:teuthology.run:FAIL